{"id":40979,"date":"2023-07-02T21:31:16","date_gmt":"2023-07-02T21:31:16","guid":{"rendered":"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=40979"},"modified":"2024-02-03T20:07:37","modified_gmt":"2024-02-03T20:07:37","slug":"mental-imagery-w-thomans-naselar","status":"publish","type":"post","link":"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=40979","title":{"rendered":"Mental imagery w\/ Thomas Naselaris"},"content":{"rendered":"<p id=\"top\" \/>\n<p>Are there really images in the brain? If not, what is meant by &#8220;image&#8221;? If yes, how do you get around the classical problem of infinite regression (who&#8217;s watching the mental image? The homonculus problem\u2026).<\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>I use the label &#8220;mental image&#8221; to describe a visual representation in the brain that is independent of retinal input. I think the term &#8220;mental image&#8221; is fine because people with mental imagery (like myself) have vision-like experiences that are independent of retinal input. But &#8220;mental image&#8221; is really a technical term. It means, &#8220;I can build a model of visual processing that can predict the brain activity that I recorded when I instructed a subject to imagine&#8221;.<\/em><\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>This technical definition doesn&#8217;t run into any problem with infinite regress, because it doesn&#8217;t posit a &#8220;mental screen&#8221; and a &#8220;mental viewer of the image&#8221;, etc. It&#8217;s just a technical term that is inspired by the subjective experience of mental imagery, but is not beholden to it.<\/em><\/p>\n\n\n\n<p>My own intuition is that brain activity associated to mental imagery is also active when seeing the world; I imagine it as a dynamic symbolic representation (with spatial relationships) that &#8220;resonates&#8221; with visual information picked up by our eyes in a feedback loop. When we close our eyes, some of us are able to stimulate that symbolic representation by the sheer force of our imagination\/memory, but like an impulse response, I&#8217;d imagine it quickly dies out as it does not &#8220;resonate&#8221; with the perceived world.<\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>Per my technical definition of mental imagery, we definitely imagine while we see. The story about the interaction between simultaneous imagery and vision is just developing. There are certainly interactions between seen and mental image representations. We are working quite hard on a computational theory that makes predictions about the effects on brain activity of mental and seen images that do or do not cohere. According to the theory (so far), it seems that \u00a0these interactions are\u00a0complicated. There could be  resonance&#8221;-like phenomena; there could be lots of other\u00a0kinds of interactions too. But it&#8217;s too early for me to say, as we have not yet done the right experiments.<\/em><\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>As for &#8220;symbolic&#8221; representations&#8211;I don&#8217;t know. I find it easiest and most useful to define certain kinds of brain activity in terms of the models that accurately predict them. We build models of activity in the visual cortex when it is being activated by seen images, and we find that these models generalize to brain activity that is recorded when subjects are imagining. So, I feel comfortable referring to mental images as &#8220;visual representations&#8221;. That said, there are parts of visual cortex where language models are also predictive. Like, I can show many pictures to the brain, and then predict response by extracting natural-language captions from the pictures, mapping them into a semantic feature space (like the ones learned by &#8220;CLIP&#8221;, for example), and regressing those features onto brain activity. Does this mean the activity encoded &#8220;symbolic&#8221; representations? I don&#8217;t know, but don&#8217;t think it matters too much what label you use if you are clear about the model you are referring to.<\/em><\/p>\n\n\n\n<p>This is of course entirely speculative. But the reason why I see it as plausible is because it could explain why it is so hard to focus on details in mental imagery, which is perhaps related to the larger spatial receptive field responses you presented us in your talk. <\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>That&#8217;s a really interesting idea. So like, mental images are hard to sustain, because if they don&#8217;t resonate with what we&#8217;re seeing they will eventually extinguish? I think that&#8217;s an interesting and plausible hypothesis.<\/em><\/p>\n\n\n\n<p>Why a purely imaginary focusing is possible with macrodoses of psychedelics then becomes particularly intriguing, as it would mean that the feedback loop that keep the outside world and internal symbolic representation in check is actually enforced by the brain (through something like the Default Mode Network?).<\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>Also interesting. The hard work is defining exactly what the drugs do to amplify\/sustain the activity.\u00a0<\/em><\/p>\n\n\n\n<p>As you also mentioned when talking about artistic practice, I believe that feedback loops are essential to understanding visual perception (which is closer to my field of expertise). However, if mental imagery is what happens when you cut out that loop to remove visual feedback, then trying to understand it by re-instating a loop (as you suggested at the end of your talk) might be misleading: people might instead learn to use the AI-generated image feedback to perform a novel task, which is perhaps similar to what happens in Brain Computer Interface research.<\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>Yes, I was proposing to &#8220;close the loop&#8221; with the brain (or the mental image), as in a BCI. Because mental images themselves are under-specified, I think the most accurate pixel-space characterization of any one mental image is as a *distribution* over pictures. If we close the loop, the seen and mental image could be continuously updated until they converge (in principle).\u00a0<\/em><\/p>\n\n\n\n<p>I wonder whether you side with these intuitions, and how they could be tested rigorously.<br>In any case, thanks again for your very stimulating talk!<\/p>\n\n\n\n<p class=\"has-text-align-right\"><em>Yes, I think they can all be tested, and all will be tested over the decades! It will be fun to find out!<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Are there really images in the brain? If not, what is meant by &#8220;image&#8221;? If yes, how do you get around the classical problem of infinite regression (who&#8217;s watching the mental image? The homonculus problem\u2026). I use the label &#8220;mental image&#8221; to describe a visual representation in the brain that is independent of retinal input. &#8230; <a title=\"Mental imagery w\/ Thomas Naselaris\" class=\"read-more\" href=\"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=40979\" aria-label=\"Read more about Mental imagery w\/ Thomas Naselaris\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[],"class_list":["post-40979","post","type-post","status-publish","format-standard","hentry","category-discuss"],"_links":{"self":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/40979","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=40979"}],"version-history":[{"count":3,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/40979\/revisions"}],"predecessor-version":[{"id":41020,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/40979\/revisions\/41020"}],"wp:attachment":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=40979"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=40979"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=40979"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}