Sparse coding

Understanding how the brain encodes stimuli is a fundamental problem in computational neuroscience. This has inspired the development of artificial neural networks that learn representations by incorporating brain-like learning abilities. Recent work has focused on learning representations (latents) by capturing similarity between inputs and latents, but these methods have typically been used to learn downstream features from the input and have not been studied in the context of a generative paradigm, where one can map the representations back to the input space, incorporating not only bottom-

…The model is related to prior work on sparse coding and slow feature analysis, but goes further by positing multiplicative interactions between latent variables for invariant object representations and for equivariant transformations representi

Environmental enrichment improves hippocampus-dependent spatial learning and memory. However, how enrichment influences hippocampal network activity remains still largely unclear.Using cFos labeling, we showed that exploration of a novel context leads to increased cFos activity in hippocampal principal neurons in mice housed under both standard and enrichment conditions. Remarkably, the number of cFos-expressing cells was lower in enriched mice. Additionally, we recorded Ca2+ responses in CA1 pyramidal cells (PCs) using miniature microscope in mice performing exploration. We observed lower eve

Sparse coding is an effective computational strategy for ensuring robust pattern storage and memory recall of correlated patterns. In the hippocampus, many neurons fire in localised receptive fields that tile an animal's physical environment, forming a sparse spatial population code. But the underlying computations and their functional purpose remain unclear. Various ANNs have been proposed to learn localised, manifold-tiling codes, purportedly due to explicit constraints on network activity levels or to specific task objectives, such as predicting future observations. Here, we show that in si

The classical sparse coding (SC) model represents visual stimuli as a convex combination of a handful of learned basis functions that are Gabor-like when trained on natural image data. However, the Gabor-like filters learned by classical sparse coding far overpredict well-tuned simple cell receptive field profiles observed empirically. While neurons fire sparsely, neuronal populations are also organized in physical space by their sensitivity to certain features. In V1, this organization is a smooth progression of orientations along the cortical sheet. A number of subsequent models have either

It is well known that sparse coding models trained on natural images learn basis functions whose shapes resemble the receptive fields (RFs) of simple cells in the primary visual cortex (V1). However, few studies have considered how these basis functions develop during training. In particular, it is unclear whether certain types of basis functions emerge more quickly than others, or whether they develop simultaneously. In this work, we train an overcomplete sparse coding model (Sparsenet) on natural images and find that there is indeed order in the development of its basis functions, with lower

…in the biologically relevant sparse coding limit [4]. However, multiple recent results suggest an intermediate scenario in which synaptic efficacy can be described by a continuous variable, but whose distribution is peaked around a

The efficient coding hypothesis posits that sensory systems are adapted to the statistics of their inputs, capturing essential structure while minimizing the use of resources (neurons, spikes, etc). A variety of formulations have been developed and, differing primarily in their definition of efficiency. For example, Independent Components Analysis [Bell and Sejnowski, 1997] seeks a complete set of axes along which the data distribution is heavy-tailed. Sparse Coding [Olshausen and Field, 1996] learns a set of basis functions that can sparsely reconstruct natural image patches (

Recent work has shown that spontaneous movements are represented throughout the brain, even in sensory areas such as primary visual cortex (V1). Interference between visual stimuli and movement is minimized in V1 because they are encoded in orthogonal subspaces, but it is not known how these subspaces are formed. Here, we report that Neuropixels recordings in mouse V1 during slow-wave sleep (SWS) reveal internally generated low-dimensional manifold structure that is mostly preserved in the awake state. However, the awake state has more variance in “off-manifold” dimensions, defined as those ex

…and their synaptic in-degree, and sparse coding by granule cells. Typically, these theories focus on the learning of random