{"id":3761,"date":"2010-07-08T14:16:55","date_gmt":"2010-07-08T14:16:55","guid":{"rendered":"http:\/\/www.labri.fr\/perso\/barla\/blog\/?p=3761"},"modified":"2012-07-09T16:04:27","modified_gmt":"2012-07-09T16:04:27","slug":"probabilistic-models-of-the-brain","status":"publish","type":"post","link":"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=3761","title":{"rendered":"Probabilistic Models of the Brain"},"content":{"rendered":"<p id=\"top\" \/><em>Rajesh Rao, Brnuo Olshausen and Michael Lewicki (Ed.)<br \/>\n29 mars 2010<\/em><\/p>\n<p><em><br \/>\n<\/em><\/p>\n<h2>Introduction<\/h2>\n<ul>\n<li>Barlow \ufb01rst proposed a self-organizing strategy for sensory nervous systems based on the principle of redundancy reduction, the idea that neurons should encode information in such a way as to minimize statistical dependencies. The alluring aspect of this approach is that it does not require that one pre-suppose a speci\ufb01c goal for sensory processing, such as \u201cedge-detection\u201d or \u201ccontour extraction\u201d. Rather the emphasis is on formulating a general goal for sensory processing from which speci\ufb01c coding strategies such as edge detection of contour integration could be derived. &#8211; p.2<\/li>\n<li>How broadly can the principle of statistically ef\ufb01cient coding of natural signals be applied before some other principle, such as task-related adaptation, becomes more important ? &#8211; p.9<\/li>\n<\/ul>\n<h2>Bayesian modelling of Visual Perception<\/h2>\n<ul>\n<li>We can foresee that priors will be conditional on context. When more than one prior can be applied in a certain context, then these priors will have to compete. &#8211; p.34 <em>How could Bayesian models be tweaked to serve as a model of Minsky&#8217;s frames?<\/em><\/li>\n<li>Different tasks might direct the attention of the visual system to different components of the stimulus and their associated priors. &#8211; p.34<\/li>\n<\/ul>\n<h2>Vision, Psychophysics and Bayes<\/h2>\n<ul>\n<li>Of course, human vision is not optimal, but starting from the optimal observer yields a coherent research strategy in which models can be modi\ufb01ed to discard the same kinds and amount of information as the human visual system. &#8211; p.39<\/li>\n<li>The immediate focus of interest is how much human performance differs from ideal, because it is the differences which are diagnostic of the kinds of mechanisms used by the human visual system. &#8211; p.40<\/li>\n<li>The modelling problem naturally breaks in two : determining how a useful signal (e.g., objects, distances, shapes, etc) get encoded into intensity changes in the image, and second, determining the limits to decoding the image to infer the signals. &#8211; p.41 <em>Sounds like a praise for 1st order analysis!<\/em><\/li>\n<\/ul>\n<h2>Velocity likelihoods in biological and machine vision<\/h2>\n<ul>\n<li>A Bayesian model with a prior favoring slow speeds can explain a range of percepts in human vision. &#8211; p.80<\/li>\n<li>To a \ufb01rst approximation, complex cells in V1 can be modeled as squared outputs of spatiotemporal oriented \ufb01lters. Again, to \ufb01rst approximation, MT pattern cells can be modeled as pooling these squared responses over space. This is consistent with the idea that a population of velocity tuned cells in area MT represent the likelihood of a velocity. &#8211; p.94<\/li>\n<\/ul>\n<h2>Natural image statistics for cortical orientation map development<\/h2>\n<ul>\n<li>It has been proposed that the goal of coding should be to detect the underlying cause of the input by reducing the redundancy in the neuronal activities. This is achieved to some degree in the visual system that processes natural images projected onto the retina. The images are typically highly redundant, because they contain correlation in space and time. Some of the redundancy is already reduced in the retina : the On-Off properties of retinal ganglion cells, e.g., effectively serve as local spatial decorrelation \ufb01lters. In a subsequent step, simple cells in the primary visual cortex decorrelate the input even further &#8211; they respond to typical features (oriented bars) that correspond to input activity correlations of higher order, i.e., between many neurons at a time. It is thus a reasonnable goal for simple cell development to reduce the redundancy of neurons\u2019 responses in a natural viewing environment. &#8211; p.182<\/li>\n<li>We suggest a competitive mechanism for cortical dynamics and predict a sparse representation for the simple cells. From a functional point of view, a sparse code is a good strategy to represent visual stimuli, because the structure of natural images is sparse. -p.197<\/li>\n<li>We observe that our simulations converge much faster if we use an annealing scheme that starts with weak competition slowly strengthening over time. This might also happen in the cortex : activity patterns \ufb01rst reaching it long before eye opening when the local circuitry may not yet be able to support strong competition. &#8211; p.199<\/li>\n<\/ul>\n<h2>Natural image statistics and divisive normalization<\/h2>\n<ul>\n<li>Statistical dependency appears in pairs of coef\ufb01cients at nearby spatial positions, orientations and scales. The dependency decreases or pairs that differ markedly in one or more of these attributes. &#8211; p.205<\/li>\n<li>By considering speci\ufb01c types of changes to the visual environment, we \ufb01nd that image statistics serve to distinguish between two canonical types of adaptation, which we refer to as contrast and pattern adaptation. Each type of adaptation arises from a speci\ufb01c change in image statistics. In response to the statistical change, normalization parameters are altered in predictable ways, which then leads to a change in the contrast response function. &#8211; p.211<\/li>\n<li>Re-scaling the contrast changes the normalization constant, which shifts the contrast response function to the right ; this sequence of events will be called contrast adaptation. &#8211; p.211<\/li>\n<li>Increased dependency between a neuron and its neighbor increases the corresponding normalization weight, which in turn lowers the saturation level of the curve. This sequence of events will be called pattern adaptation. p.212<\/li>\n<li>A sensible assumption might be that each stage of processing in the system takes the response of the previous stage and attempts to eliminate as much statistical redundancy as possible, within the limits of its computational capabilities. This \u201csuccessive whitening\u201d version of the theory naturally leads one to ask : How far can this bottom-up theory go to explain higher levels of sensory processing ? Surely, at some level, the tasks that the organism performs must have some in\ufb02uence on the design of the system. &#8211; p.217<\/li>\n<\/ul>\n<h2>A probabilistic network model of population responses<\/h2>\n<ul>\n<li>We have proposed a novel network model of population responses that can support a variety of response pro\ufb01les, and veridically encodes multiple values. The signi\ufb01cant contribution of this model lies in the proposal that the role of the population is to represent the full range of information about orientation present in the input. &#8211; p.238<\/li>\n<li>The differences in behavior between our model and previous ones may be primarily traced to two features of the models : the relative roles of feedforward and recurrent inputs, and the recurrent weight pro\ufb01le. p.239<\/li>\n<li>Various orientation selectivity models can be characterized by their assumptions about what information underlies the population responses in V1, which is re\ufb02ected in their balance between feedforward and recurrent inputs, and in the recurrent weight pro\ufb01le. &#8211; p.240<\/li>\n<li>It is possible to devise a network model that optimizes the recurrent weights and balances these two forces to preserve information about multiple values in the input. &#8211; p.240<\/li>\n<li>The distribution population coding provides a natural framework for understanding the information encoded in a population response, and for formulating objectives for preserving and manipulating this information. p.240<\/li>\n<\/ul>\n<h2>Sparse codes and spikes<\/h2>\n<ul>\n<li>In order for the cortex to perform inference on retinal images, it must somehow implement a generative model for explaining the signals coming from optic nerve \ufb01bers in terms of hypotheses about the state of the world. I shall propose here that the neurons in the primary visual cortex, area V1, form the \ufb01rst stage in this generative modeling process by modeling the structure of images in terms of a linear superposition of basis functions. One can think of these basis functions as simple \u201cfeature vocabulary\u201d for describing images in terms of additive functions. &#8211; p.257\u00a0 <em>Additiveness seems to be an important feature of sensory coding simply to make decoding easier! However, although the basis functions themselves are added, it may not correspond to adding portions of the original signal, right?<\/em><\/li>\n<li>It is suggested that the spike trains of sensory neurons essentially serve as a sparse code in time, which in turn forms a more ef\ufb01cient and meaningful representation of image structure. Thus, a single principle may be able to account for both the receptive properties of neurons and the spiking nature of neural activity. &#8211; p.258<\/li>\n<li>Importantly, the basis set is overcomplete, meaning that there are more basis functions than effective dimensions in the image. Overcompleteness in the representation is important because it allows for the joint space of position, orientation, and spatial-frequency to be tiled smoothly without artifacts. More generally though, it allows for a greater degree of \ufb02exibility in the representation, as there is no reason to believe a priori that the number of causes for images is less than or equal to the number of pixels. &#8211; p.258<\/li>\n<li>Whitening removes second-order correlations due to the 1\/ f 2 power spectrum of natural images, and it approximates the type of \ufb01ltering performed by the retina. p.262<\/li>\n<li>When the model is adapted to time-varying natural images, the basis functions converge upon a set of space-time functions which are spatially Gabor-like and translate with time. Moreover, the sparsi\ufb01ed representation has a spike-like character, in that the coef\ufb01cient signals are mostly zero and tend to concentrate their non-zero activity into brief, punctuate events. These brief events represent longer spatiotemporal events in the image via the basis functions. The results suggest, then, that both the receptive \ufb01elds and spiking activity of V1 neurons may be explained in terms of a single principle, that of sparse coding in time. &#8211; p.268<\/li>\n<\/ul>\n<h2>Predictive coding, cortical feedback, and spike-timing dependent plasticity<\/h2>\n<ul>\n<li>Feedback connections from a higher to a lower cortical area carry predictions of expected neural activity in the lower area, while the feedforward connections carry the differences between that predictions and the actual neural activity. &#8211; p.297 <em>With strong differences, one could trigger a &#8220;move&#8221; to a different frame.<\/em><\/li>\n<li>Recurrent feedback connections between neurons within a cortical area are used to learn, store and predict temporal sequences of input neural activity. &#8211; p.297<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Rajesh Rao, Brnuo Olshausen and Michael Lewicki (Ed.) 29 mars 2010 Introduction Barlow \ufb01rst proposed a self-organizing strategy for sensory nervous systems based on the principle of redundancy reduction, the idea that neurons should encode information in such a way as to minimize statistical dependencies. The alluring aspect of this approach is that it does &#8230; <a title=\"Probabilistic Models of the Brain\" class=\"read-more\" href=\"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=3761\" aria-label=\"Read more about Probabilistic Models of the Brain\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[621],"tags":[],"class_list":["post-3761","post","type-post","status-publish","format-standard","hentry","category-books"],"_links":{"self":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/3761","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3761"}],"version-history":[{"count":7,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/3761\/revisions"}],"predecessor-version":[{"id":3871,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/3761\/revisions\/3871"}],"wp:attachment":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3761"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3761"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3761"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}