{"id":40496,"date":"2021-09-03T21:03:49","date_gmt":"2021-09-03T21:03:49","guid":{"rendered":"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=40496"},"modified":"2021-09-07T09:00:49","modified_gmt":"2021-09-07T09:00:49","slug":"reverse-hierarchy-w-alex-schmid","status":"publish","type":"post","link":"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=40496","title":{"rendered":"Reverse hierarchy w\/ Alex Schmid"},"content":{"rendered":"<p id=\"top\" \/>\n<p class=\"has-text-align-right\"><em>Look forward to hearing your thoughts! Maybe we are due for another chat about ideas soon \ud83d\ude42<br>I&#8217;m giving a lab talk next Tuesday (presenting stuff from my proposal) and trying to organise my thoughts about the seemingly&nbsp;&#8220;separate&#8221; processing and subsequent binding of things like colour\/motion\/form, and reconcile this with my stance that object colour\/motion\/form\/material are jointly&nbsp;computed from the same image structure in a holistic way and are inseparable. I think the coupled computations idea can fit nicely with the theories presented in the symposium &#8211; all &#8220;perceptual access&#8221; is &#8220;high level&#8221; (even seemingly low-level phenomena like pop-out visual search), and we only have perceptual access after we reach some holistic level of processing.<\/em><\/p>\n\n\n\n<p>I&#8217;ve watched the first two presentations, and especially liked the second one.<br>The idea of a fast bottom-up perception, followed by selective top-down scrutiny adapted to a certain task sounds convincing.<br>Also the idea that we always start from a global gist, but with the initial bottom-up process being unconscious, is quite plausible ecologically speaking.<br><br>There seems to be a good amount of evidence for that theory; I&#8217;d be curious to know whether there&#8217;s evidence that&#8217;s seemingly <em>against <\/em>it though&#8230;<br><br>The typical experiments that they show focus on <em>things <\/em>(objects, letters, faces, shapes) in more or less complex scenes.<br>In contrast, material (or shape) perception seems different since focus is rather on a single object, from which we want to discern material (or shape) properties.<br><br>Let me do a brain dump of what it makes me come to mind in this context \ud83d\ude42 :<br><br>For shape\/material perception, the fast bottom-up (unconscious) &#8220;gist perception&#8221; could consist in a process that identifies occluding contours (when available), from which top-down &#8220;scrutiny&#8221; could work inward.<br>Contours themselves might not need to be perceived in details, which reminds me the way some artists start by &#8220;blocking out&#8221; shapes in early painting phases: <a href=\"https:\/\/images.app.goo.gl\/c9ZRt4x9TrNDrxsT6\">https:\/\/images.app.goo.gl\/c9ZRt4x9TrNDrxsT6<\/a><br>Here not only approximate contours, but also overall colors are roughly identified; only later, during the top-down scrutiny phase, may vision start to rely on refined image features in a task-dependent way.<br><br>Regarding material perception, some categories might be sufficiently distinct from others (say based on their color, think of Gold) to be perceived directly from &#8220;gist perception&#8221;.<br>Other categories might require more attention and scrutiny, potentially because their image feature signatures are close enough to other categories from which they must be distinguished (according to the current task).<br><br>I wonder to which extent material and shape perception are intertwined: maybe the more we scrutinize, the more we attend to image features that are diagnostic of either material or shape ?\u00a0<br>For instance, the compression\/stretching of a reflection is determined by curvature anisotropy, while its amount of blur depends on surface micro-roughness (both can be affected by lighting as well).<br>In practice, I think we rarely pay such a close attention to details (exceptions might include some zealous artists, vision\/graphics researchers, and subjects in perception experiments ;-)).<br>A more common example of &#8220;scrutiny perception&#8221; might be to follow the fold of a dress (guided by directions of reflection elongations) and to pay attention to feature signatures (including blur, but also color, contrast, etc) to recognize that the dress is made of, say, satin. At this level of scrutiny refinement, I guess material and shape might still be intertwined to some extent&#8230;<\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>This is where I think material and shape are completely intertwined, in that they mutually constrain one another and are not &#8220;separately computed&#8217;.<\/em><\/p>\n\n\n\n<p>There&#8217;s something that bugs me though: we are able (in most cases) to make separate judgments about shape and material, or even about gloss and color. It means that when prompted, we can scrutinize further, and let (or force?) ourselves to pay attention to different aspects of image features. However, this might not be the default regime of everyday life perception, but rather a more or less disciplined way to look at the world.<br><br>One might say (as Koenderink does in his recent writings) that we must &#8220;learn to see&#8221;, as do artists since they have to learn how to convey the right image features. However, there&#8217;s a difference between an artist and a person who&#8217;s asked to make specific judgments about shape or material: the artists put their scrutiny to the test by trying to reproduce image features through painting or drawing, which likely allows them to <em>realize <\/em>which aspects of image features are effective, and which aspects can be approximated. This is an active learning process.<br><br>In contrast, making separate judgments about shape and material does not involve any trial and error (at least for an adult): it seems rather natural, even though not that common. How can we reconcile that if we assume that shape and material properties appear intertwined in image features? The only way I can make sense of it is that some aspects of image features are intrinsically more governed by shape, while others are more governed by material, as in the example of highlight elongation (shape) and blur\/contrast (material) I mentioned previously.<br><br>What is important, according to the reverse hierarchical theory, is that this ability of making separate judgments does not have to be an intermediate step before we can recognize the object we have in front of us. It&#8217;s the other way around: scrutiny requires more time and energy, such as when one is resting and inspecting an object, out of sheer curiosity. There might have been a bias toward that rather uncommon behavior <em>(scrutiny) <\/em>in the material perception literature.<br><br>If we want to get away from such a bias, we might have to consider larger patterns of image features. What are the patterns that govern the perception of a piece of cloth, or of a patch of ground soaked by the rain, or of a smooth object that can fit in one&#8217;s hand? How can these patterns be organized hierarchically in a bottom up fashion, starting from image features, so that the highest (and most approximate) level of description could be assumed to be quickly available to <em>gist <\/em>perception, while the lowest (and closest to actual shape and material) actually require prolonged <em>scrutiny<\/em>, and likely involve personal experience or even active learning (as with artists)?<br><br>I think the story becomes even more complex when you put motion and action into the mix: indeed, <em>scrutiny <\/em>usually assumes a static scene\/object and a separate observer.<br>What if the object is moving by itself? Or in the field of view due to <em>locomotion<\/em>? Or manipulated in one own&#8217;s hand so that it can be <em>inspected <\/em>?<br>I suspect that these conditions involve rather different perception regimes, and my guess is that <em>scrutiny<\/em> is only one possible such regime; whereas <em>gist <\/em>might be a default strategy.<\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>For navigation\/locomotion, I think gist plays more of a role (or at least it&#8217;s the default strategy), but for manipulating something in one&#8217;s hand, scrutiny becomes important. Though it does seem automatic in many cases&#8230; It&#8217;s probably not though?<br><\/em><\/p>\n\n\n\n<p>Indeed, <em>inspection<\/em> might not necessarily require scrutiny (one can inspect an object by turning it around without paying close attention to all details of image features). I find it useful to think in terms of different vision regimes, even though they are probably strongly biased by language&#8230; I&#8217;ve recently compiled a list of terms to think about it. Here it is, roughly ordered based on the implication of the observer in the act of seeing. <\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>Interestingly, I&#8217;ve had some personal insights into this kind of stuff lately. I can&#8217;t remember if we talked about it yet (I&#8217;ve told Katja), but I was relatively recently diagnosed with ADHD (maybe you already got this impression of me\u00a0\ud83d\ude43). Of course I&#8217;ve had it my whole life without knowing, but since &#8220;diagnosis&#8221; I&#8217;ve had insights into how I visually process\/attend to things versus most other people,\u00a0and the framework presented in the symposium session allowed me to better articulate what I was experiencing. <\/em><\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>Let me try:<br>When I take medication to help with attentional regulation, it allows me to have control over what I focus on, whereas previously my focus would be random and unpredictable. Without medication, I have approximately 2 modes: completely scattered, or I focus for 10 hours (or 10 days) straight on a project, nothing in between. With the medication, I can *way* more easily choose what to focus on and switch between different kinds of tasks. That&#8217;s the behavioural outcome, but what it *feels* like is, without the medication I have no control over the scrutiny aspect of things &#8211; whatever I&#8217;m processing, whether it&#8217;s scattered or deep focus, is nearly completely bottom up, whereas with the meds suddenly I have proper top down scrutiny ability, and visually I can actually parse a scene into different elements, such that objects are separate from one another and not a giant blur of colour\/shapes\/motion (same with audio input, and internal thoughts &#8211; external\/internal noise becomes able to be voluntarily scrutinised); it&#8217;s the scrutiny of different aspects any kind of input from a scene at will. Kind of like putting glasses on. I think with non-ADHD folk, the conscious\/top down scrutiny is usually so effortless that it seems automatic, even though it&#8217;s not, whereas I was experiencing all of the gist with either lack of scrutiny (scattered, broad, attending to everything at once) or no control over where the scrutiny goes (hyperfocus on a single thing).\u00a0The insight comes from now having had both experiences. (PS: Now I&#8217;m finding special populations very interesting&#8230;\u00a0\ud83d\ude02\u00a0It is possible that I am part of more than one special population, but we can chat about this over zoom some time if you&#8217;re interested! \ud83d\ude09).<\/em><\/p>\n\n\n\n<p>Very interesting indeed! I don&#8217;t really like the term &#8220;disorder&#8221; actually: it might only be a different way of being, one that may not always be adapted to our modern way of living, but that might have gone unnoticed (or have provided a clear advantage) in different times&#8230; I don&#8217;t really know. It seems that what&#8217;s effortless in non-ADHD folks (those lousy neuro-typical people ;-)) is the switching between different regimes: it&#8217;s seamless (as in my experience). This is what makes it seem that all the details of the objects were perceived all the time even before I started paying attention to them, even though it&#8217;s not the case. There&#8217;s a connection with foveal\/peripheral vision here, but it goes even further as it extends in time.<br><br>This reminds me of a\u00a0 paper on &#8220;sensori-motor contingencies&#8221; (<a href=\"http:\/\/nivea.psycho.univ-paris5.fr\/Manuscripts\/ORegan;Noe.BBS.pdf\">http:\/\/nivea.psycho.univ-paris5.fr\/Manuscripts\/ORegan;Noe.BBS.pdf<\/a>), which in a nutshell argues that the complex representations we seem to have in the brain are nothing but illusions: what really happens are prediction processes (the contingencies) that change based on the task at hand. These processes might be good candidate mechanisms to explain how a subject goes from <em>gist <\/em>to <em>scrutiny: <\/em>by making (very fast) predictions that require to be checked through visual attention to (coarse or fine) parts of the visual field, with conscious perception emerging when those predictions are found to be compliant with observation. In contrast, surprise may happen in case of prediction failures, and multi-stable stimuli might also be explained by the switching of predictions.<br><br>I wonder also how this relates to something I&#8217;ve recently read about: the default mode network (DMN). I&#8217;ve read about it in a very surprising paper where researchers use psychedelic drugs to study different states of consciousness, the Enthropic Brain Theory (<a href=\"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnhum.2014.00020\/full\">https:\/\/www.frontiersin.org\/articles\/10.3389\/fnhum.2014.00020\/full<\/a>). I&#8217;ve started asking for references about the DMN to colleagues working in connection with neurosciences, but you might know more about it actually!<br><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Look forward to hearing your thoughts! Maybe we are due for another chat about ideas soon \ud83d\ude42I&#8217;m giving a lab talk next Tuesday (presenting stuff from my proposal) and trying to organise my thoughts about the seemingly&nbsp;&#8220;separate&#8221; processing and subsequent binding of things like colour\/motion\/form, and reconcile this with my stance that object colour\/motion\/form\/material are &#8230; <a title=\"Reverse hierarchy w\/ Alex Schmid\" class=\"read-more\" href=\"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=40496\" aria-label=\"Read more about Reverse hierarchy w\/ Alex Schmid\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[],"class_list":["post-40496","post","type-post","status-publish","format-standard","hentry","category-discuss"],"_links":{"self":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/40496","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=40496"}],"version-history":[{"count":2,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/40496\/revisions"}],"predecessor-version":[{"id":40500,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/40496\/revisions\/40500"}],"wp:attachment":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=40496"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=40496"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=40496"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}