{"id":39816,"date":"2018-09-06T12:12:08","date_gmt":"2018-09-06T12:12:08","guid":{"rendered":"http:\/\/www.labri.fr\/perso\/barla\/blog\/?p=39816"},"modified":"2019-08-14T16:00:04","modified_gmt":"2019-08-14T16:00:04","slug":"critical-contours-w-s-zucker","status":"publish","type":"post","link":"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=39816","title":{"rendered":"Shading structure w\/ Steven &#038; Ben"},"content":{"rendered":"<p id=\"top\" \/><h3>About Critical contours<\/h3>\n<p style=\"padding-left: 30px;\">Masataka visited our research team (in Bordeaux, France) a few months ago, and he pointed me to your paper on critical contours. I&#8217;ve read it once, and I&#8217;ll probably have to come back to it to understand the maths completely (I wanted to get to the end of the paper first). But I already have some comments, remarks and questions, so here we go!<\/p>\n<p>Again, thanks for the multiple readings. There&#8217;s a version in SIAM J. Image Science now, which might be slightly cleaner.<br \/><a class=\"moz-txt-link-freetext\" href=\"https:\/\/epubs.siam.org\/doi\/abs\/10.1137\/17M1145525\">https:\/\/epubs.siam.org\/doi\/abs\/10.1137\/17M1145525<\/a><\/p>\n<p style=\"padding-left: 30px;\">I really liked this observation about other approaches (data- or regularization-driven) being too <b>brittle<\/b> because one image constraints a family of shapes. One thing I kept wondering though is why you elected to use contours as invariants across the family of shapes, and not <b>regions<\/b> for instance?<\/p>\n<p>Regions and contours are connected via the &#8220;shading to contour limit&#8221; &#8212; and it is these contours that form part of the Morse-Smale complex. We worked pretty seriously to understand this, because it identifies part of the shape from contour problem with (part of) the shape from shading problem. In effect, it is these contours that serve to represent the key shaded regions on which shape inference is grounded (in our view).<\/p>\n<p style=\"padding-left: 30px;\">I completely understand why occluding contours should be part of these invariants (being themselves discontinuities of visibility). But for locii around and in the prolongation of internal contours, or even at the bottom of ridges or the top of valleys, regions might suffice. When artists draw curves, isn&#8217;t it because they restrict themselves to a specific style? Actually they might (and often do) use hatching to convey regions&#8230;<\/p>\n<p>A nice consequence of the topological nature of our argument is that it provides some flexibility &#8212; it leads to thinking about equivalence class structures rather than specific instances. I wish that we had something deeper to say about artists&#8217; style.<\/p>\n<p>You&#8217;re right that sometimes it&#8217;s the bottom of a ridge that matters, etc; and this is exactly what we are working on now. The critical contours are a kind of extremal curve in slant, and the shading paper only focused on maxima. There&#8217;s a lot more to be said &#8212; stay tuned.<\/p>\n<p style=\"padding-left: 30px;\">One thing that I really liked in the paper is equation 2: with this the image formation process permits much more interesting appearances.<br \/>I was wondering why you did not show more <b>complex F functions<\/b>, for instance to deal with shiny objects; is it a limitation or simply that you have not yet done this? With Romain Vergne, George-Pierre Bonneau and Roland Fleming, we have published a paper at Siggraph a couple years ago, relying on the same equation. We did consider more complex F functions (called LitSpheres or MatCaps in Graphics), but our mathematical analysis did not go as far as yours&#8230; Still you might be interested to have a look at it (and ask questions if you feel like it :-)): <a class=\"moz-txt-link-freetext\" href=\"https:\/\/hal.inria.fr\/hal-01307571\/file\/paper-light.pdf\">https:\/\/hal.inria.fr\/hal-01307571\/file\/paper-light.pdf<\/a><\/p>\n<p>Thanks for the reference &#8212; we&#8217;ll definitely have a look at it.<\/p>\n<p style=\"padding-left: 30px;\">On a more conceptual level, I am not sure that the human visual system really has to <b>reconstruct<\/b> a structure and fill it in.<\/p>\n<p>We agree completely! In fact, there&#8217;s some evidence from neurophysiology that this can be the case. (We have a figure on this issue using stimuli from Ed Connor in an encyclopaedia article coming out soon.)\u00a0 Moreover, it may be that it&#8217;s only\u00a0when the intermediate structure is required that it is reconstructed (or filled in). Tons to do on this issue!<\/p>\n<p style=\"padding-left: 30px;\">I&#8217;m not convinced that this is necessary (or even efficient) in usual everyday tasks, where some surface details may safely be ignored. Of course this is not the case for tasks such as painting and drawing; but these require imagination and careful observation, not mentioning training&#8230; So even if we are <b>capable<\/b> of inferring a structure and fill it in for specific purposes (art, research, etc), it does not mean this is the natural way to go! I know this is a tough (philosophical?) question, but do you think that the human visual system actually goes through the reconstruction of a complex Morse-like structure?<\/p>\n<p>So far we only have evidence for a portion of the MS complex &#8212; and there is no doubt that other components matter as well (e.g. occluding contours). The important thing about MS is that it&#8217;s global, and shape inferences involve global aspects. It is in this sense that we feel Morse-Smale is &#8216;pointing&#8217; in the right direction.<\/p>\n<h3>About hue flow<\/h3>\n<p>There&#8217;s actually (much) more to the color\/shading story than we&#8217;ve published thus far (which is my fault, largely). In particular, we have a notion of &#8216;hue frequency&#8217;, which is the number of cycles around the hue<br \/>circle as intensity runs from black &#8211;&gt; white. The color\/shading effect\u00a0holds when the hue freq is in the range 1 &#8211; 3 cycles; below this it looks\u00a0like a lighting effect and above it a texture effect.<\/p>\n<p>I think that your movie illustrates this change over a neighborhood. Does this make sense to you?<\/p>\n<p style=\"padding-left: 30px;\">Yes it completely makes sense: I noticed the impact of frequency while I was playing with the shader!<\/p>\n<p style=\"padding-left: 30px;\">If I had to guess, I&#8217;d say this is because when shading and hue are correlated but with similar frequencies, there is an ambiguity between shading and reflectance: the image can be reasonably interpreted as &#8220;colored shading&#8221;. When the frequency increases, the resulting color fringes then become attributed to reflectance variations, but the smooth intensity variations due to shading are not perceived on their own, but rather as part of the reflectance changes (reflectance colors are simply darker in some regions). Of course, when you decorrelate hue and shading, the resulting flows run across each others, providing strong enough visual cues to see shading and reflectance separately.<\/p>\n<p style=\"padding-left: 30px;\">What I also find interesting is that with prominent shape features in sight (as in the end of the video I sent you), even when hue and shading are correlated, and no matter how you adjust the frequency of hue variations, you will always be able to see shading (and thus shape) clearly. I&#8217;d say this is due both to the presence of internal occluding contours, and rather strong shading variations, which leave no leeway for an interpretation in terms of reflectance changes. Maybe it&#8217;s worth investigating as well?<\/p>\n<p>Yes; it&#8217;s definitely worth investigating. And, once again, you&#8217;ve hit upon another subtlety in the color\/shading interactions that also &#8216;accounts for&#8217; what you&#8217;ve noticed. In particular, the shading and color variations cannot be too concentrated in the image &#8212; they must span a bit of space. This, I believe, is because the physiological foundation for this &#8212; Liz Johnson&#8217;s oriented double opponent cells &#8212; are sensitive only in the range of about 1 &#8211; 4 cycles\/degree. (See the right had figure in the attached slide). When the neighborhood around the critical contour is too small, like in the &#8216;prominent shape features&#8217; in your video, the cells simply do not respond.\u00a0We&#8217;ve never done any psychophysics directly to address this, but it really needs to be done.<\/p>\n<p>[Ben]\u00a0It would be interesting to understand why the sharp critical contours are so dominant. I think the last large feature (central bump) you showed on your movie may be occluding other parts of the surface (e.g. the critical contour on the right side). This discontinuity may be &#8216;unmaskable&#8217;. If you note the left side of that same bump, it does seem like it is masked fairly well at some frequency.<\/p>\n<p style=\"padding-left: 30px;\">You&#8217;re right, the left side of the bump does appear masked, definitely. Some &#8220;closure&#8221; effect may be at work when one looks at the entire shape though. Indeed, the entire contour of the large feature appears closed even when it is masked locally. I guess the relationship to the occluding contours is another important visual aspect&#8230;<\/p>\n<p>We definitely think that there is contour completion necessary. Our theory is:<\/p>\n<p>1. Full Orientation Field<br \/>2. Parallel compressed orientations yield &#8216;critical contours&#8217;; these are possible open and detached<br \/>3. Contour completion gives closed critical contours<br \/>4. Closed critical contours yield 3D perception of protrusions, bumps, etc.<\/p>\n<p style=\"padding-left: 30px;\">\u00a0<\/p>\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>About Critical contours Masataka visited our research team (in Bordeaux, France) a few months ago, and he pointed me to your paper on critical contours. I&#8217;ve read it once, and I&#8217;ll probably have to come back to it to understand the maths completely (I wanted to get to the end of the paper first). But &#8230; <a title=\"Shading structure w\/ Steven &#038; Ben\" class=\"read-more\" href=\"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=39816\" aria-label=\"Read more about Shading structure w\/ Steven &#038; Ben\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[],"class_list":["post-39816","post","type-post","status-publish","format-standard","hentry","category-discuss"],"_links":{"self":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/39816","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=39816"}],"version-history":[{"count":5,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/39816\/revisions"}],"predecessor-version":[{"id":40137,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/39816\/revisions\/40137"}],"wp:attachment":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=39816"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=39816"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=39816"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}