Topic: neural networks

Apply neural networks to biological sequence analysis, including sequence classification, functional-feature discovery, source tracking and sequence generation. Hands-on notebooks introduce sequence encoding and language-model concepts, progressing from convolutional networks and LSTMs to attention and Transformers. Exercises use TensorFlow and Keras to connect model training and performance evaluation with the biological interpretation of genomic data. 2027-01-25–2027-01-27, 14:00–20:00 daily, Europe/Berlin (CET, UTC+01:00). Online course; fee €480. Participants should be familiar with Pytho

The Max Planck School of Biomedical Artificial Intelligence is recruiting its first cohort for 25 fully funded, structured three-year PhD positions at the interface of AI and the life sciences. Doctoral researchers will develop computational and AI methods for molecular and cellular systems, protein and molecule design, complex biological traits and neural networks, with dual supervision by fellows from complementary backgrounds. The programme spans more than 30 research groups across Germany and includes annual interdisciplinary schools, conference funding and mentoring. Applications close on

Current machine learning systems consume vastly more energy than biological brains. Neuromorphic systems aim to overcome this difference by mimicking the brain’s information coding via discrete voltage spikes. However, it remains unclear how both artificial and natural networks of spiking neurons can learn energy-efficient information processing strategies. Here we propose Predictive Coding Light (PCL), a recurrent hierarchical spiking neural network for unsupervised representation learning. In contrast to previous predictive coding approaches, PCL does not transmit prediction errors to higher

Behavior and cognition depend on the integrated action of neural structures and populations distributed throughout the brain. We recently developed a set of molecular imaging tools that enable multiregional processing and plasticity in neural networks to be studied at a brain-wide scale in rodents and nonhuman primates. Here we will describe how a novel genetically encoded activity reporter enables information flow in virally labeled neural circuitry to be monitored by fMRI. Using the reporter to perform functional imaging of synaptically defined neural populations in the rat somatosensory sys

Critical phenomena abound in nature, from forest fires and earthquakes to avalanches in sand and neuronal activity. Since the 2003 publication by Beggs & Plenz on neuronal avalanches, a growing body of work suggests that the brain homeostatically regulates itself to operate near a critical point where information processing is optimal. At this critical point, incoming activity is neither amplified (supercritical) nor damped (subcritical), but approximately preserved as it passes through neural networks. Departures from the critical point have been associated with conditions of poor neurologica

Young children have sophisticated representations of their visual and linguistic environment. Where do these representations come from? How much knowledge arises through generic learning mechanisms applied to sensory data, and how much requires more substantive (possibly innate) inductive biases? We examine these questions by training neural networks solely on longitudinal data collected from a single child (Sullivan et al., 2020), consisting of egocentric video and audio streams. Our principal findings are as follows: 1) Based on visual only training, neural networks can acquire high-level vi

Learning in neural networks requires assigning the right values to thousands to trillions or more of individual connections, so that the network as a whole produces the desired behavior. Neuroscientists have gained insights into this “credit assignment” problem through decades of experimental, modeling, and theoretical studies. This has suggested key roles for synaptic eligibility traces and top-down feedback signals, among other factors. Here we study the potential contribution of another type of signaling that is being revealed in greater and greater fidelity by ongoing molecular and genomic

Neurons in the rodent hippocampus appear to encode the position of the animal in physical space during movement. Individual ``place cells'' fire in restricted sub-regions of an environment, a feature often taken as evidence that the hippocampus encodes a map of space that subserves navigation. But these same neurons exhibit complex responses to many other variables that defy explanation by position alone, and the hippocampus is known to be more broadly critical for memory formation. Here we elaborate and test a theory of hippocampal coding which produces place cells as a general consequence of

In this talk, we propose to decipher the activity of neural networks via a “multiply and conquer” approach. This approach considers limit networks made of infinitely many replicas with the same basic neural structure. The key point is that these so-called replica-mean-field networks are in fact simplified, tractable versions of neural networks that retain important features of the finite network structure of interest. The finite size of neuronal populations and synaptic interactions is a core determinant of neural dynamics, being responsible for non-zero correlation in the spiking activity and

Epilepsy is a common and disabling neurologic condition affecting adults and children that results from complex dysfunction of neural networks and is ineffectively treated with current therapies in up to one third of patients. This dysfunction can have especially severe consequences in pediatric age group, where neurodevelopment may be irreversibly affected. Furthermore, although seizures are the most obvious manifestation of epilepsy, the cognitive and psychiatric dysfunction that often coexists in patients with this disorder has the potential to be equally disabling. Given these challenges,

How seizures emerge from the abnormal dynamics of neural networks within the epileptogenic tissue remains an enigma. Are seizures random events, or do detectable changes in brain dynamics precede them? Are mechanisms of seizure emergence identical at the onset and later stages of epilepsy? Is the risk of seizure occurrence stable, or does it change over time? A myriad of questions about seizure genesis remains to be answered to understand the core principles governing seizure genesis. The last decade has brought unprecedented insights into the complex nature of seizure emergence. It is now bel

Animals are presented with noisy and incomplete information from which we learn how to react in novel situations. For example, a given training set could have many explanations, each implying a different behavioural response; acting appropriately requires inferring the most plausible explanation. The way animals choose to wield Occam's razor, selecting the most parsimonious explanation, is called their inductive bias, and it is implicitly built into the operation of animals' neural circuits. This relationship between an observed sensory circuit and its inductive bias is a useful explanatory wi

One of the many ocean swells in the history of computational approaches to mind and brain is the rise and fall of cognitive architectures. Although supplanted by neural networks as the dominant modelling modality, cognitive architectures possess many attributes sorely coveted by the computational modeller: large, architectural-level scales, unified explanations of disparate cognitive functions in single models, and rich, hierarchical-modular decompositional structure conducive to building and modifying large computational systems. In contrast, we find in the neural network literature a prolife

Flexible computation is a hallmark of intelligent behavior. Yet, little is known about how neural networks contextually reconfigure for different computations. Humans are able to perform a new task without extensive training, presumably through the composition of elementary processes that were previously learned. Cognitive scientists have long hypothesized the possibility of a compositional neural code, where complex neural computations are made up of constituent components; however, the neural substrate underlying this structure remains elusive in biological and artificial neural networks. He

Semantic cognition brings meaning to our world – it allows us to make sense of what we see and hear, and to produce adaptive thoughts and behaviour. Since we have a wealth of information about any given concept, our store of knowledge is not sufficient for successful semantic cognition; we also need mechanisms that can steer the information that we retrieve so it suits the context or our current goals. This talk traces the neural networks that underpin this flexibility in semantic cognition. It draws on evidence from multiple methods (neuropsychology, neuroimaging, neural stimulation) to show

It is widely believed that storing and maintaining memories on long-time scales depends on modifying synapses in the brain in an activity-dependent way. Classical studies of learning and memory in neural networks model synaptic efficacy as a continuous or discrete scalar value [1–3]. Theoretical work has shown such models have a reasonably large capacity, especially in the biologically relevant sparse coding limit [4]. However, multiple recent results suggest an intermediate scenario in which synaptic efficacy can be described by a continuous variable, but whose distribution is peaked around a

The success of neural networks derives from the fact that they can learn useful representations of observed inputs. However, the dynamics of when and how fast a network will discover good representations are not well understood, even when network training is governed by gradient descent. The potential complexity of learning dynamics is particularly apparent in so called insight moments, when useful task representations are discovered suddenly following a delay during which no learning is noticeable. Such insights are commonly observed in animals and humans, where they are often taken to refl

Introducing exciting methods that enable neuroscientists to look deep into the living brain, allowing us to study how the brain's neural networks learn and process sensory information.

Sensory systems reliably process incoming stimuli in spite of changes in context. Most recent models accredit this context invariance to an extraction of increasingly complex sensory features in hierarchical feedforward networks. Here, we study how context-invariant representations can be established by feedback rather than feedforward processing. We show that feedforward neural networks modulated by feedback can dynamically generate invariant sensory representations. The required feedback can be implemented as a slow and spatially diffuse gain modulation. The invariance is not present on the

Our ability to understand ongoing events depends critically on general knowledge about how different kinds of situations work (schemas), and also on recollection of specific instances of these situations that we have previously experienced (episodic memory). The consensus around this general view masks deep questions about how these two memory systems interact to support event understanding: How do we build our library of schemas? and how exactly do we use episodic memory in the service of event understanding? Given rich, continuous inputs, when do we store and retrieve episodic memory “snapsh

A myriad of pathological changes associated with epilepsy, including the loss of specific cell types, improper expression of individual ion channels, and synaptic sprouting, can be recast as decreases in cell and circuit heterogeneity. In recent experimental work, we demonstrated that biophysical diversity is a key characteristic of human cortical pyramidal cells, and past theoretical work has shown that neuronal heterogeneity improves a neural circuit’s ability to encode information. Viewed alongside the fact that seizure is an information-poor brain state, these findings motivate the hypothe

Deep neural networks (DNNs) with the flexibility to learn good top-layer representations have eclipsed shallow kernel methods without that flexibility. Here, we take inspiration from deep neural networks to develop a new family of deep kernel method. In a deep kernel method, there is a kernel at every layer, and the kernels are jointly optimized to improve performance (with strong regularisation). We establish the representational power of deep kernel methods, by showing that they perform exact inference in an infinitely wide Bayesian neural network or deep Gaussian process. Next, we conje

Ascending neuromodulatory projections from the locus coeruleus (LC) affect cortical neural networks via the release of norepinephrine (NE). However, the exact nature of these neuromodulatory effects on neural activity patterns in vivo is not well understood. Here we show that in awake monkeys, LC activation is associated with changes in coordinated activity patterns in the anterior cingulate cortex (ACC). These relationships, which are largely independent of changes in firing rates of individual ACC neurons, depend on the type of LC activation: ACC pairwise correlations tend to be reduced when

Since the theories of Marr, Ito, and Albus, the cerebellum has provided an attractive well-characterized model system to investigate biological mechanisms of learning. In recent years, theories have been developed that provide a normative account for many features of the anatomy and function of cerebellar cortex and cerebellum-like systems, including the distribution of parallel fiber-Purkinje cell synaptic weights, the expansion in neuron number of the granule cell layer and their synaptic in-degree, and sparse coding by granule cells. Typically, these theories focus on the learning of random