Natural images

Divisive normalization (DN), the division of a neuron’s activity by a pool of connected neurons, has been recognized as a general computational motif in the cortex. Arguments support that in the visual cortex, DN acts as a canonical computation in generative models, which delivers contextual effects and contributes to shaping response variability. Deep generative models (DGMs) have recently been adopted by neuroscience from machine vision to embrace nonlinear computations in the visual cortex. Yet, it remains unclear whether DGMs effectively integrate DN as a core computational element. We inv

We study the statistics of the 3D color space of natural images and develop a model for efficiently representing this structure. Analysis of a dataset of 503 calibrated natural images reveals striking non-Gaussian structure with heavy tails in distinct directions that are asymmetrically distributed in 3D color space. An overcomplete, non-negative sparse coding model adapted to this structure recovers a basis aligned with the directions of black, white, red, green, blue, and yellow - i.e., the 'unique hues' that are proposed to constitute a psychological basis for color appearance. Moreover, th

…oriented, and band-pass RFs from natural images, and exhibits signature V1 complex cell phase inv

Natural images contain statistical redundancies that produce long-range spatial correlations and regularities such as edges, contours and textures (Simoncelli, 2003; van der Schaaf and van Hateren,1996; Hermundstad et al., 2014). The efficient coding hypothesis suggests that the brain optimizes sensory representation by removing such redundancy in input stimuli (Barlow et al., 1961). However, critics argue that an efficient code that simply encodes the inputs may not necessarily be useful for the tasks performed by an organism (Simoncelli and Olshausen, 2001; Barlow, 2001), i.e., neural system

The brain computes internal representations by applying highly recurrent dynamics to feed-forward input. Such dynamics may be viewed as analogous to the inference step in latent variables models, where one usually follows the gradient of the latent posterior distribution until a stable state is reached. At the stable state, when this gradient has approximately zero norm, a parameter optimization step can be applied to slightly increase the likelihood of the current sample under the model. Here we suggest this procedure, which closely matches the expectation-maximization (EM) algorithm, can be

…(EEG) contain information about viewed natural images and their categories. However, it remains unclear which specific visual features contribute to this information. To address this, we applied spectral manipulations and different texturization algorithms to natural images of various categories and recorded MEG in human subjects presented with these stimuli. The employed stimulus space allowed us to attribute components of visual information to specific low-, mid- and high-level visual features. Furthermore, this

A prominent theory of sensory perception advocates that perception in the brain is implemented via probabilistic inference. The neural sampling hypothesis (NSH) posits that neuronal responses to a stimulus represent samples from the posterior distribution over latent world state variables (e.g., object identity) that underlie the stimulus. Existing work on NSH commonly evaluates qualitative agreement of experimental data with simple generative models of the stimulus and does not fit NSH models to experimentally observed sensory population responses. We propose a novel formulation for NSH that

Natural visual environments are full of complex and diverse patterns. Recent neurophysiological experiments based on artificial patterns have suggested that macaque V1 neurons’ receptive fields are more complex and diverse in feature selectivity (Tang et al 2017) rather than just oriented Gabor filters. However, that finding was based on extensive parametric artificial stimuli used and might be biased. Hence, the nature of the neural codes of macaque V1 neurons remains controversial. To resolve this question, we performed 2-photon calcium imaging on three macaque monkeys to obtain 1689 V1 neur

Working memory (WM) allows us to hold information temporarily and make complex decisions beyond reflexive response to stimuli. One prominent feature of WM is its capacity limit. Despite decades of study, the root of this limit is still not well-understood. Most previous accounts for this limit assume various forms of memory constraints and make strong, often oversimplified, assumptions about sensory representations of stimuli. In this work, we built visual-cognitive neural network models of WM that process raw sensory stimuli. In contrast to intuitions that capacity limit results from memory c

…to the statistical properties of natural images. Neurophysiology and fMRI studies have tested neural sensitivity to images altered to exhibit either typical or atypical statistics. A general finding is that visual cortical neurons respond more strongly to images with natural statistics, and that this can vary across the visual hierarchy. Convolutional neural networks (CNNs) have achieved human-level performance in vision tasks and have been used as visual cortex models. However, CNN responses to images with altered s

…a difference in fMRI responses between natural and urban landscapes in certain parts of the brain. Here we asked if this distinction in the representation of complex natural and man-made visual stimuli extends to the ventral visual stream and if st

An object’s colour, brightness and pattern are all influenced by its surroundings, and a number of visual phenomena and “illusions” have been discovered that highlight these often dramatic effects. Explanations for these phenomena range from low-level neural mechanisms to high-level processes that incorporate contextual information or prior knowledge. Importantly, few of these phenomena can currently be accounted for when measuring an object’s perceived colour. Here we ask to what extent colour appearance is predicted by a model based on the principle of coding efficiency. The model assumes th

It is well known that sparse coding models trained on natural images learn basis functions whose shapes resemble the receptive fields (RFs) of simple cells in the primary visual cortex (V1). However, few studies have considered how these basis functions develop during training. In particular, it is unclear whether certain types of basis functions emerge more quickly than others, or whether they develop simultaneously. In this work, we train an overcomplete sparse coding model (Sparsenet) on natural images and find that there is indeed order in the development of its basis functions, with lower

Exploiting invariances in the inputs is crucial for constructing efficient representations and accurate predictions in neural circuits. In neuroscience, translation invariance is at the heart of models of the visual system, while convolutional neural networks designed to exploit translation invariance triggered the first wave of deep learning successes. While the hallmarks of convolutions, namely localised receptive fields that tile the input space, can be implemented with fully-connected neural networks, learning convolutions directly from inputs in a fully-connected network has so far proven

…through unsupervised learning of natural statistics. Generative models have provided invaluable insights into the way neural response statistics of low level vision (including nonlinearities in response means, variability, oscillations) is shaped by probabilistic inference. However, progress has been hampered by the limited capabilities of generative models to learn both nonlinear and hierarchical representations of natural images. Here we harness the inspirations coming from neuroscience to

…providing efficient representations of natural images. Yet, we also know that many ganglion cells respond vigorously to fine spatial gratings that should not activate the linear filter of the receptive field. Thus, ganglion cells may integrate visual signals nonlinearly across space. In this talk, I will disc