Topic: efficient coding hypothesis

Biological visual systems learn complex representations of the world that support a wide range of cognitive behaviors without using a large number of labelled examples. The efficient coding hypothesis suggests that this is accomplished by adapting the sensory representation to the statistics of the input signal, i.e. in a way that facilitates redundancy reduction. Visual signals have several clear sources of redundancy. They evolve slowly in time, since temporally adjacent inputs typically correspond to different views of the same scene, which in turn are usually more similar than views of dis

Higher-order brain activity can be quite complex, switching between fast, stimulus- or motor-driven dynamics, and slower, rising and falling persistent activity. Many of these activity motifs can be successfully modelled by recurrent neural networks (RNNs) that are trained on specific tasks. However, the training of RNNs is usually an ill-posed problem so that, at least in principle, multiple solutions exist for any particular task. Accordingly, a specific match of a trained RNN to data can be serendipitous, providing only limited insight into the reasons underlying the similarity. Here we tak

Neural system identification aims at learning the response function of neurons to arbitrary stimuli by incorporating the right assumptions into the model that facilitate generalization beyond the particular stimuli used during training [1]. Here, we present normative network regularization as a novel regularization tool that allows to flexibly impose prior assumptions into the model training. In particular, we use this approach to incorporate the efficient coding hypothesis as a regularizer which states that neural response properties of sensory representations are strongly shaped by the need

The efficient coding hypothesis posits that sensory systems are adapted to the statistics of their inputs, capturing essential structure while minimizing the use of resources (neurons, spikes, etc). A variety of formulations have been developed and, differing primarily in their definition of efficiency. For example, Independent Components Analysis [Bell and Sejnowski, 1997] seeks a complete set of axes along which the data distribution is heavy-tailed. Sparse Coding [Olshausen and Field, 1996] learns a set of basis functions that can sparsely reconstruct natural image patches (

According to the efficient coding hypothesis, neural responses represent information so as to enable the most accurate readout possible, constrained by neuronal resources and neuronal noise. To date, much of the theoretical work on efficient neural coding has focused on relatively simple models of neural activity, characterized by smooth, often unimodal tuning curves. Real neurons, however, often exhibit more complex tuning curves. For instance, in the enthorinal cortex, the periodicity of grid cell tuning curves, as well as their functional organization in modules, imparts the population code

In line with the efficient coding hypothesis, the early visual system aims to minimize spectral and spatial redundancies arising from overlapping opsin sensitivities in retinal photoreceptors (PRs) and highly correlated structure in natural scenes. Encoding color information, or spectral information independent of intensity, requires comparing activities across different types of PRs. Mounting evidence shows that several species across the animal kingdom, such as the fruit fly Drosophila Melanogaster, have an uneven proportion of PR types in their retinas. However, it is unknown whether this u

A major goal of sensory neuroscience is to understand the system's response to individual stimuli. Although mutual information (MI) continues to be a useful tool when applying Information theory to neural systems, it does not suffice when trying to understand how informative individual stimuli are. In general such knowledge may be crucial for understanding the functional organization of sensory systems. To investigate this, we focus on ON-OFF Direction Selective Retinal Ganglion Cells (dsRGC). These cells respond robustly to motion in their preferred direction, and sparsely to motion in the op

Recent research in awake, behaving organisms revealed the strong modulatory effects of movement on sensory coding. Surprisingly, these effects are not consistent across species. For example, in rodents and insects locomotion increases the magnitude of visual responses [1-3], while in primates, locomotion has a weak suppressive influence [4]. These differences raise intriguing questions about the computational purpose of such modulations and the generality of the underlying principles of sensory processing. Here, we address these questions from a theoretical perspective. Our approach is groun

While sensory coding is traditionally thought of as a process independent of behavioral state, modern studies showcase potent modulatory effects of movement on sensory neurons. Intriguingly, these effects are not uniform and can differ between species. In insects and rodents, visual responses increase dramatically when the animal is moving, whereas in primates locomotion has a weak impact on responses in the primary visual cortex (V1). These differences raise questions about the existence and universality of computational principles underlying sensory coding during behavior. We take a normativ

Natural images contain statistical redundancies that produce long-range spatial correlations and regularities such as edges, contours and textures (Simoncelli, 2003; van der Schaaf and van Hateren,1996; Hermundstad et al., 2014). The efficient coding hypothesis suggests that the brain optimizes sensory representation by removing such redundancy in input stimuli (Barlow et al., 1961). However, critics argue that an efficient code that simply encodes the inputs may not necessarily be useful for the tasks performed by an organism (Simoncelli and Olshausen, 2001; Barlow, 2001), i.e., neural system