Topic: visual cortex

Arno Onken examines transformer models for predicting responses in the retina and visual cortex. These models learn complex neural-response patterns from large datasets and can transfer learned representations between settings, but fitting and interpreting them differs from linear–nonlinear and convolutional approaches. The talk discusses the opportunities and difficulties this creates for understanding sensory processing. It includes ViV1T, a transformer trained on natural movies that revealed previously uncharacterised response properties in mouse primary visual cortex. The work illustrates

Current machine learning systems consume vastly more energy than biological brains. Neuromorphic systems aim to overcome this difference by mimicking the brain’s information coding via discrete voltage spikes. However, it remains unclear how both artificial and natural networks of spiking neurons can learn energy-efficient information processing strategies. Here we propose Predictive Coding Light (PCL), a recurrent hierarchical spiking neural network for unsupervised representation learning. In contrast to previous predictive coding approaches, PCL does not transmit prediction errors to higher

Functional connectomics reveals general wiring rule in mouse visual cortex

Visual restoration after decades of blindness is now becoming possible by means of retinal and cortical prostheses, as well as emerging stem cell and gene therapeutic approaches. After restoring visual perception, however, a key question remains. Are there optimal means and methods for retraining the visual cortex to process visual inputs, and for learning or relearning to “see”? Up to this point, it has been largely assumed that if the sensory loss is visual, then the rehabilitation focus should also be primarily visual. However, the other senses play a key role in visual rehabilitation due t

While anatomical connectivity changes slowly through synaptic learning, the functional connectivity of neurons changes rapidly with ongoing activity of neurons and their functional interactions. Functional networks of neurons and neural populations reflect how their interactions change with behaviors, stimulus types, and internal states. Therefore, the information propagation across a network can be analyzed through the varying topological properties of the functional networks. Our study investigates the functional networks of the visual cortex at both the single-cell and population levels. Ou

Perhaps the most impressive aspect of the way the brain enables us to act on the sensory world is its flexibility. We can make a general inference about many sensory features (rating the ripeness of mangoes or avocados) and map a single stimulus onto many choices (slicing or blending mangoes). These can be thought of as flexibly mapping many (features) to one (inference) and one (feature) to many (choices) sensory inputs to actions. Both theoretical and experimental investigations of this sort of flexible sensorimotor mapping tend to treat sensory areas as relatively static. Models typically

Trends in NeuroAI is a reading group hosted by the MedARC Neuroimaging & AI lab (https://medarc.ai/fmri). Title: Brain-optimized inference improves reconstructions of fMRI brain activity Abstract: The release of large datasets and developments in AI have led to dramatic improvements in decoding methods that reconstruct seen images from human brain activity. We evaluate the prospect of further improving recent decoding methods by optimizing for consistency between reconstructions and brain activity during inference. We sample seed reconstructions from a base decoding method, then iteratively re

An appealing hypothesis states that visual neurons in the brain are sensitive to the statistical properties of natural images. Neurophysiology and fMRI studies have tested neural sensitivity to images altered to exhibit either typical or atypical statistics. A general finding is that visual cortical neurons respond more strongly to images with natural statistics, and that this can vary across the visual hierarchy. Convolutional neural networks (CNNs) have achieved human-level performance in vision tasks and have been used as visual cortex models. However, CNN responses to images with altered s

The human visual cortex processes visual stimuli hierarchically. Early visual areas (V1, V2) of the ventral visual stream feed crude visual features (like orientation and edges) into later visual areas (V4, lateral occipital (Lat Occ), inferior temporal (IT)) that then encode complex visual features (like object form). Previous studies have reported a difference in fMRI responses between natural and urban landscapes in certain parts of the brain. Here we asked if this distinction in the representation of complex natural and man-made visual stimuli extends to the ventral visual stream and if st

VIP inhibitory neurons in the cortex receive neuromodulatory and top-down feedback inputs, hence they are well-suited to process behavioral and predictive signals. A recent study found that VIP neurons respond to unexpected image omissions in an image sequence (Garrett et al., 2020). This omission-evoked activity may represent changes in the animal’s behavioral state (e.g., running or arousal), or it may represent a prediction-error signal that could either be stimulus specific, as suggested by the classical predictive coding models (Rao & Ballard, 1992), or stimulus non-specific. We investiga

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching – and sometimes even surpassing – human accuracy on a variety of visual recognition tasks. In this talk, however, I will show that these neural networks and their recent extensions exhibit a limited ability to solve seemingly simple visual reasoning problems involving incremental grouping, similarity, and spatial relation judgments. Our group has developed a recurrent network model of classical and

Microglia, traditionally classified as immune-responsive, adjust synaptic connections during development and disease. However, their role in the adult nervous system has been mostly diminished to an observer. In my research group, we are interested in how microglia are involved in establishing and maintaining accurate neuronal circuit function in the retina and in the visual cortex. In my talk, I will introduce our strategies how to decipher the microglia’s functional identity and how this information guided us to microglia enabled extracellular matrix remodeling and reinstatment of juvenile-l

It was long hypothesized that natural systems might take advantage of the extended temporal and spatial correlations close to the critical point to improve their computational capabilities. However, on the other side, different distances to criticality were inferred from the recordings of nervous systems. In my talk, I discuss how including additional constraints on the processing time can shift the optimal operating point of the recurrent networks. Moreover, the data from the visual cortex of the monkeys during the attentional task indicate that they flexibly change the closeness to the criti

Transcriptomics has revealed that cortical inhibitory neurons exhibit a great diversity of fine molecular subtypes, but it is not known whether these subtypes have correspondingly diverse activity patterns in the living brain. We show that inhibitory subtypes in primary visual cortex (V1) have diverse correlates with brain state, but that this diversity is organized by a single factor: position along their main axis of transcriptomic variation. We combined in vivo 2-photon calcium imaging of mouse V1 with a novel transcriptomic method to identify mRNAs for 72 selected genes in ex vivo slices.

Visual restoration is certainly the greatest challenge for brain-machine interfaces with the high pixel number and high refreshing rate. In the recent year, we brought retinal prostheses and optogenetic therapy up to successful clinical trials. Concerning visual restoration at the cortical level, prostheses have shown efficacy for limited periods of time and limited pixel numbers. We are investigating the potential of sonogenetics to develop a non-contact brain machine interface allowing long-lasting activation of the visual cortex. The presentation will introduce our genetic-based brain machi

A central goal of vision science is to understand the principles underlying the perception and neural coding of the complex visual environment of our everyday experience. In the visual cortex, foundational work with artificial stimuli, and more recent work combining natural images and deep convolutional neural networks, have revealed much about the tuning of cortical neurons to specific image features. However, a major limitation of this existing work is its focus on single-neuron response strength to isolated images. First, during natural vision, the inputs to cortical neurons are not isolate

How the brain separates information about multiple objects despite overlap in the neurons responsive to each item is not well understood. It has recently been proposed that when more than one stimulus is present, single neurons can fluctuate between coding one vs. the other(s) across some time period, i.e. a form of neural multiplexing in the time domain (Caruso et al., 2018). However, it is not known (a) whether such fluctuations occur specifically when two distinct objects are present and not in other cases, and (b) how fluctuations in individual neurons might be coordinated with one anot

Information processing is energetically expensive. In the mammalian brain, it is unclear how information coding and energy usage are regulated during food scarcity. I addressed this in the visual cortex of awake mice using whole-cell recordings and two-photon imaging to monitor layer 2/3 neuronal activity and ATP usage. I found that food restriction reduced synaptic ATP usage by 29% through a decrease in AMPA receptor conductance. Neuronal excitability was nonetheless preserved by a compensatory increase in input resistance and a depolarized resting membrane potential. Consequently, neurons sp

Neuroplasticity is a fundamental property of the nervous system that is maximal early in life, within a specific temporal window called critical period. However, it is still unclear to which extent the plastic potential of the visual cortex is retained in adulthood. We have surprisingly revealed residual ocular dominance plasticity in adult humans by showing that short-term monocular deprivation unexpectedly boosts the deprived eye (both at the perceptual and at the neural level), reflecting homeostatic plasticity. This effect is accompanied by a decrease of GABAergic inhibition in the primary

Historically, retinotopic organisation (the spatial mapping of the retina across the cortical surface) was considered the purview of early regions of visual cortex (V1-V4) only and that anterior, more cognitively involved regions abstracted this information away. The contemporary view is quite different. Here, with Advancing technologies and analysis methods, we see that retinotopic information is not simply thrown away by these regions but rather is maintained to the potential benefit of our broader cognition. This maintenance of visuospatial coding extends not only through visual cortex, but

Cortical state, defined by the synchrony of population-level neuronal activity, is a key determinant of sensory perception. While many arousal-associated neuromodulators—including norepinephrine (NE)—reduce cortical synchrony, how the cortex resynchronizes following NE signaling remains unknown. Using in vivo two-photon imaging and electrophysiology in mouse visual cortex, we describe a critical role for cortical astrocytes in circuit resynchronization. We characterize astrocytes’ sensitive calcium responses to changes in behavioral arousal and NE, identify that astrocyte signaling precedes in

During navigation, animals estimate their position using path integration and landmarks, engaging many brain areas. Whether these areas follow specialized or universal cue integration principles remains incompletely understood. We combine electrophysiology with virtual reality to quantify cue integration across thousands of neurons in three navigation-relevant areas: primary visual cortex (V1), retrosplenial cortex (RSC), and medial entorhinal cortex (MEC). Compared with V1 and RSC, path integration influences position estimates more in MEC, and conflicts between path integration and landmarks

The idea that perception in humans relies on some form of probabilistic computations has become very popular over the last decades. It has been extremely difficult however to characterize the extent and the nature of the probabilistic representations and operations that are manipulated by neural populations in the human cortex. Several theoretical works suggest that probabilistic representations are present from low-level sensory areas to high-level areas. According to this view, the neural dynamics implements some forms of probabilistic message passing (i.e. neural sampling, probabilistic pop

Multisensory responses are ubiquitous in so-called unisensory cortex. However, despite their prevalence, we have very little understanding of what – if anything - they contribute to perception. In this talk I will focus on audio-visual integration in auditory cortex. Anatomical tracing studies highlight visual cortex as one source of visual input to auditory cortex. Using cortical cooling we test the hypothesis that these inputs support audiovisual integration in ferret auditory cortex. Behavioural studies in humans support the idea that visual stimuli can help listeners to parse an auditory s