Topic: sparse coding

It is widely believed that storing and maintaining memories on long-time scales depends on modifying synapses in the brain in an activity-dependent way. Classical studies of learning and memory in neural networks model synaptic efficacy as a continuous or discrete scalar value [1–3]. Theoretical work has shown such models have a reasonably large capacity, especially in the biologically relevant sparse coding limit [4]. However, multiple recent results suggest an intermediate scenario in which synaptic efficacy can be described by a continuous variable, but whose distribution is peaked around a

The efficient coding hypothesis posits that sensory systems are adapted to the statistics of their inputs, capturing essential structure while minimizing the use of resources (neurons, spikes, etc). A variety of formulations have been developed and, differing primarily in their definition of efficiency. For example, Independent Components Analysis [Bell and Sejnowski, 1997] seeks a complete set of axes along which the data distribution is heavy-tailed. Sparse Coding [Olshausen and Field, 1996] learns a set of basis functions that can sparsely reconstruct natural image patches (

Recent work has shown that spontaneous movements are represented throughout the brain, even in sensory areas such as primary visual cortex (V1). Interference between visual stimuli and movement is minimized in V1 because they are encoded in orthogonal subspaces, but it is not known how these subspaces are formed. Here, we report that Neuropixels recordings in mouse V1 during slow-wave sleep (SWS) reveal internally generated low-dimensional manifold structure that is mostly preserved in the awake state. However, the awake state has more variance in “off-manifold” dimensions, defined as those ex

Since the theories of Marr, Ito, and Albus, the cerebellum has provided an attractive well-characterized model system to investigate biological mechanisms of learning. In recent years, theories have been developed that provide a normative account for many features of the anatomy and function of cerebellar cortex and cerebellum-like systems, including the distribution of parallel fiber-Purkinje cell synaptic weights, the expansion in neuron number of the granule cell layer and their synaptic in-degree, and sparse coding by granule cells. Typically, these theories focus on the learning of random