Predictive processing offers a unifying view of neural computation, proposing that brains continuously anticipate sensory input and update internal models based on prediction errors. In this talk, I will present converging evidence for the computational mechanisms underlying this framework across human neuroscience and deep neural networks. I will begin with recent work showing that large-scale distributed prediction-error encoding in the human brain directly predicts how sensory representations reorganize through predictive learning. I will then turn to PredNet, a popular predictive coding inspired deep network that has been widely used to model real-world biological vision systems. Using dynamic stimuli generated with our Spatiotemporal Style Transfer algorithm, we demonstrate that PredNet relies primarily on low-level spatiotemporal structure and remains insensitive to high-level content, revealing limits in its generalization capacity. Finally, I will discuss new recurrent vision models that integrate top-down feedback connections with intrinsic neural variability, uncovering a dual mechanism for robust sensory coding in which neural variability decorrelates unit responses, while top-down feedback stabilizes network dynamics. Together, these results outline how prediction error signaling and top-down feedback pathways shape adaptive sensory processing in biological and artificial systems.

This PhD is funded by the French ANR, under a 4 years' project on Sensorimotor integration of variability during birdsong learning. The applicant will develop an artificial neural model, developmental and brain-inspired, to learn the sound structure in real time and without explicit supervision. Until now, AI models for developmental learning of vocalizations have been solely validated by comparison against a human-annotated corpus and not yet via direct sensorimotor interactions with living animals. We expect to do so with an interactive robot under the framework of active inference and predictive coding.

For the NWO project ‘DBI2’ we are looking for a PhD candidate to study predictive error responses in the auditory cortex. The main goal is to design an experimental approach to distinguish between two alternative theories of predictive coding and processing. Predictive coding and predictive processing are compelling theories to explain brain function. The idea that the brain continually maintains and updates an internal model of the outside world, and compares the incoming input with the expectations generated by this model, can explain many phenomena including adaptive behaviour and sensory effects such as oddball responses. However, until now there is no consensus on how such predictive coding could be implemented in real neural tissue. Importantly, there are two alternative theories on how error signals in predictive processing could be coded in neural signals: either as (1) top down signals from ‘higher order’ brain areas (hierarchical predictive coding, [2]) or (2) local signals, resulting in membrane potentials reflecting error signals ([3], for a review, see [1]). The goal of the research presented here, under the primary supervision of Dr Zeldenrust, is to design an experimental approach to distinguish between these two theoretical approaches. Measuring error signals in neural tissue is experimentally challenging. Therefore, a direct exchange between theory and experiment is needed, so that hypotheses and specific predictions about which neurons to record from and stimulate and the results expected can be quickly updated for the design of optimal experiments. The student will work in close collaboration with the Englitz lab, so that there is a direct link between modelling, data analysis and experiment. As a PhD candidate you will use data on oddball paradigms [4,5], which provide the ability to directly observe predictions and distinguish them from prediction errors. The data are a combined approach of widefield imaging of the entire auditory cortex with local and layer-specific imaging using 2-photon recordings in the same animals. To directly test the top-down hypothesis, neurons in subareas of the prefrontal cortex will be transfected with an inhibitory opsin (eNpHR3.0) to modulate their top-down influence. You will develop a model of the hierarchical interaction between the auditory cortex and the prefrontal cortex, in which error signals are either coded as top-down (theory 1) or local (theory 2). You will use this model to formulate testable predictions, distinguishing theory 1 from theory 2. These predictions will be tested by both analysing existing data from the Englitz lab and formulating new experimental paradigms that are suitable to distinguish between the local and top-down hypothesis. [1] N’dri, A. W., Gebhardt, W., Teulière, C., Zeldenrust, F., Rao, R. P. N., Triesch, J., & Ororbia, A. (2024). Predictive Coding with Spiking Neural Networks: A Survey (arXiv:2409.05386). arXiv. https://doi.org/10.48550/arXiv.2409.05386 [2] Rao, R. P. N., & Ballard, D. H. (1999). Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1), 79–87. [3] Zeldenrust, F., Gutkin, B., & Denéve, S. (2021). Efficient and robust coding in heterogeneous recurrent networks. PLOS Computational Biology, 17(4), e1008673. https://doi.org/10.1371/journal.pcbi.1008673 [4] Nieto-Diego, J. & Malmierca, M. S. Topographic Distribution of Stimulus-Specific Adaptation across Auditory Cortical Fields in the Anesthetized Rat. (2016) PLOS Biol. 14, e1002397 [5] Lao-Rodríguez, A. B. ... Englitz B, (2023) Neuronal responses to omitted tones in the auditory brain: A neuronal correlate for predictive coding. Sci. Adv. 9, eabq8657 * We will give you a temporary employment contract (1.0 FTE) of 1.5 years, after which your performance will be evaluated. If the evaluation is positive, your contract will be extended by 2.5 years (4-year contract). * You will receive a starting salary of €2,901 gross per month based on a 38-hour working week, which will increase to €3,707 in the fourth year (salary scale P). * You will receive an 8% holiday allowance and an 8,3% end-of-year bonus. * We offer Dual Career Coaching. The Dual Career Coaching assists your partner via support, tools, and resources to improve their chances of independently finding employment in the Netherlands. * You will receive extra days off. With full-time employment, you can choose between 30 or 41 days of annual leave instead of the statutory 20.

My lab is looking for a PhD candidate in Modelling Predictive Error Responses: https://www.ru.nl/en/working-at/job-opportunities/phd-position-in-computational-neuroscience-modelling-predictive-error-responses For the NWO project ‘DBI2’ we are looking for a PhD candidate to study predictive error responses in the auditory cortex. The main goal is to design an experimental approach to distinguish between two alternative theories of predictive coding and processing. Predictive coding and predictive processing are compelling theories to explain brain function. The idea that the brain continually maintains and updates an internal model of the outside world, and compares the incoming input with the expectations generated by this model, can explain many phenomena including adaptive behaviour and sensory effects such as oddball responses. However, until now there is no consensus on how such predictive coding could be implemented in real neural tissue. Importantly, there are two alternative theories on how error signals in predictive processing could be coded in neural signals: either as (1) top down signals from ‘higher order’ brain areas (hierarchical predictive coding, [2]) or (2) local signals, resulting in membrane potentials reflecting error signals ([3], for a review, see [1]). The goal of the research presented here, under the primary supervision of Dr Zeldenrust, is to design an experimental approach to distinguish between these two theoretical approaches. Measuring error signals in neural tissue is experimentally challenging. Therefore, a direct exchange between theory and experiment is needed, so that hypotheses and specific predictions about which neurons to record from and stimulate and the results expected can be quickly updated for the design of optimal experiments. The student will work in close collaboration with the Englitz lab, so that there is a direct link between modelling, data analysis and experiment. As a PhD candidate you will use data on oddball paradigms [4,5], which provide the ability to directly observe predictions and distinguish them from prediction errors. The data are a combined approach of widefield imaging of the entire auditory cortex with local and layer-specific imaging using 2-photon recordings in the same animals. To directly test the top-down hypothesis, neurons in subareas of the prefrontal cortex will be transfected with an inhibitory opsin (eNpHR3.0) to modulate their top-down influence. You will develop a model of the hierarchical interaction between the auditory cortex and the prefrontal cortex, in which error signals are either coded as top-down (theory 1) or local (theory 2). You will use this model to formulate testable predictions, distinguishing theory 1 from theory 2. These predictions will be tested by both analysing existing data from the Englitz lab and formulating new experimental paradigms that are suitable to distinguish between the local and top-down hypothesis. [1] N’dri, A. W., Gebhardt, W., Teulière, C., Zeldenrust, F., Rao, R. P. N., Triesch, J., & Ororbia, A. (2024). Predictive Coding with Spiking Neural Networks: A Survey (arXiv:2409.05386). arXiv. https://doi.org/10.48550/arXiv.2409.05386 [2] Rao, R. P. N., & Ballard, D. H. (1999). Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1), 79–87. [3] Zeldenrust, F., Gutkin, B., & Denéve, S. (2021). Efficient and robust coding in heterogeneous recurrent networks. PLOS Computational Biology, 17(4), e1008673. https://doi.org/10.1371/journal.pcbi.1008673 [4] Nieto-Diego, J. & Malmierca, M. S. Topographic Distribution of Stimulus-Specific Adaptation across Auditory Cortical Fields in the Anesthetized Rat. (2016) PLOS Biol. 14, e1002397 [5] Lao-Rodríguez, A. B. ... Englitz B, (2023) Neuronal responses to omitted tones in the auditory brain: A neuronal correlate for predictive coding. Sci. Adv. 9, eabq8657 We offer * We will give you a temporary employment contract (1.0 FTE) of 1.5 years, after which your performance will be evaluated. If the evaluation is positive, your contract will be extended by 2.5 years (4-year contract). * You will receive a starting salary of €2,901 gross per month based on a 38-hour working week, which will increase to €3,707 in the fourth year (salary scale P). * You will receive an 8% holiday allowance and an 8,3% end-of-year bonus. * We offer Dual Career Coaching. The Dual Career Coaching assists your partner via support, tools, and resources to improve their chances of independently finding employment in the Netherlands. * You will receive extra days off. With full-time employment, you can choose between 30 or 41 days of annual leave instead of the statutory 20.

In a first part of the talk, I will present Predictive Coding Light (PCL), a novel unsupervised learning architecture for spiking neural networks. In contrast to conventional predictive coding approaches, which only transmit prediction errors to higher processing stages, PCL learns inhibitory lateral and top-down connectivity to suppress the most predictable spikes and passes a compressed representation of the input to higher processing stages. We show that PCL reproduces a range of biological findings and exhibits a favorable tradeoff between energy consumption and downstream classification performance on challenging benchmarks. A second part of the talk will feature our lab’s efforts to explain how infants and toddlers might learn abstract object representations without supervision. I will present deep learning models that exploit the temporal and multimodal structure of their sensory inputs to learn representations of individual objects, object categories, or abstract super-categories such as „kitchen object“ in a fully unsupervised fashion. These models offer a parsimonious account of how abstract semantic knowledge may be rooted in children's embodied first-person experiences.

The idea that perception in humans relies on some form of probabilistic computations has become very popular over the last decades. It has been extremely difficult however to characterize the extent and the nature of the probabilistic representations and operations that are manipulated by neural populations in the human cortex. Several theoretical works suggest that probabilistic representations are present from low-level sensory areas to high-level areas. According to this view, the neural dynamics implements some forms of probabilistic message passing (i.e. neural sampling, probabilistic population coding, etc.) which solves the problem of perceptual inference. Here I will present recent experimental evidence that human and non-human primate perception implements some form of message passing. I will first review findings showing probabilistic integration of sensory evidence across space and time in primate visual cortex. Second, I will show that the confidence reports in a hierarchical task reveal that uncertainty is represented both at lower and higher levels, in a way that is consistent with probabilistic message passing both from lower to higher and from higher to lower representations. Finally, I will present behavioral and neural evidence that human perception takes into account pairwise correlations in sequences of sensory samples in agreement with the message passing hypothesis, and against standard accounts such as accumulation of sensory evidence or predictive coding.

Predictive coding represents a promising framework for understanding brain function, postulating that the brain continuously inhibits predictable sensory input, ensuring a preferential processing of surprising elements. A central aspect of this view on cortical computation is its hierarchical connectivity, involving recurrent message passing between excitatory bottom-up signals and inhibitory top-down feedback. Here we use computational modelling to demonstrate that such architectural hard-wiring is not necessary. Rather, predictive coding is shown to emerge as a consequence of energy efficiency, a fundamental requirement of neural processing. When training recurrent neural networks to minimise their energy consumption while operating in predictive environments, the networks self-organise into prediction and error units with appropriate inhibitory and excitatory interconnections and learn to inhibit predictable sensory input. We demonstrate that prediction units can reliably be identified through biases in their median preactivation, pointing towards a fundamental property of prediction units in the predictive coding framework. Moving beyond the view of purely top-down driven predictions, we demonstrate via virtual lesioning experiments that networks perform predictions on two timescales: fast lateral predictions among sensory units and slower prediction cycles that integrate evidence over time. Our results, which replicate across two separate data sets, suggest that predictive coding can be interpreted as a natural consequence of energy efficiency. More generally, they raise the question which other computational principles of brain function can be understood as a result of physical constraints posed by the brain, opening up a new area of bio-inspired, machine learning-powered neuroscience research.

Deep learning has made tremendous progress in the last year but it's high computational and memory requirements impose challenges in using deep learning on edge devices. There has been some progress in lowering memory requirements of deep neural networks (for instance, use of half-precision) but there has been minimal effort in developing alternative efficient computational paradigms. Inspired by the brain, Spiking Neural Networks (SNN) provide an energy-efficient alternative to conventional rate-based neural networks. However, SNN architectures that employ the traditional feedforward and feedback pass do not fully exploit the asynchronous event-based processing paradigm of SNNs. In the first part of my talk, I will present my work on predictive coding which offers a fundamentally different approach to developing neural networks that are particularly suitable for event-based processing. In the second part of my talk, I will present our work on development of approaches for SNNs that target specific problems like low response latency and continual learning. References Dora, S., Bohte, S. M., & Pennartz, C. (2021). Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy. Frontiers in Computational Neuroscience, 65. Saranirad, V., McGinnity, T. M., Dora, S., & Coyle, D. (2021, July). DoB-SNN: A New Neuron Assembly-Inspired Spiking Neural Network for Pattern Classification. In 2021 International Joint Conference on Neural Networks (IJCNN) (pp. 1-6). IEEE. Machingal, P., Thousif, M., Dora, S., Sundaram, S., Meng, Q. (2021). A Cross Entropy Loss for Spiking Neural Networks. Expert Systems with Applications (under review).

Sensory experience often depends on one’s own actions, including self-motion. Theories of predictive coding postulate that actions are regulated by calculating prediction error, which is the difference between sensory experience and expectation based on self-generated actions. Signals consistent with prediction error have been reported in mouse visual cortex (V1) when visual flow coupled to running was unexpectedly stopped. Here, we show such signals can be elicited by visual stimuli uncoupled to animal’s running. We recorded V1 neurons while presenting drifting gratings that unexpectedly stopped. We found strong responses to visual perturbations, which were enhanced during running. Perturbation responses were strongest in the preferred orientation of individual neurons and perturbation responsive neurons were more likely to prefer slow visual speeds. Our results indicate that prediction error signals can be explained by the convergence of known motor and sensory signals, providing a purely sensory and motor explanation for purported mismatch signals.

A recent formulation of predictive coding theory proposes that a subset of neurons in each cortical area encodes sensory prediction errors, the difference between predictions relayed from higher cortex and the sensory input. Here, we test for evidence of prediction error responses in spiking responses and local field potentials (LFP) recorded in primary visual cortex and area V4 of macaque monkeys, and in complementary electroencephalographic (EEG) scalp recordings in human participants. We presented a fixed sequence of visual stimuli on most trials, and violated the expected ordering on a small subset of trials. Under predictive coding theory, pattern-violating stimuli should trigger robust prediction errors, but we found that spiking, LFP and EEG responses to expected and pattern-violating stimuli were nearly identical. Our results challenge the assertion that a fundamental computational motif in sensory cortex is to signal prediction errors, at least those based on predictions derived from temporal patterns of visual stimulation.

At any given moment the brain receives more sensory information than it can use to guide adaptive behaviour, creating the need for mechanisms that promote efficient processing of incoming sensory signals. One way in which the brain might reduce its sensory processing load is to encode successive presentations of the same stimulus in a more efficient form, a process known as neural adaptation. Conversely, when a stimulus violates an expected pattern, it should evoke an enhanced neural response. Such a scheme for sensory encoding has been formalised in predictive coding theories, which propose that recent experience establishes expectations in the brain that generate prediction errors when violated. In this webinar, Professor Jason Mattingley will discuss whether the encoding of elementary visual features is modulated when otherwise identical stimuli are expected or unexpected based upon the history of stimulus presentation. In humans, EEG was employed to measure neural activity evoked by gratings of different orientations, and multivariate forward modelling was used to determine how orientation selectivity is affected for expected versus unexpected stimuli. In mice, two-photon calcium imaging was used to quantify orientation tuning of individual neurons in the primary visual cortex to expected and unexpected gratings. Results revealed enhanced orientation tuning to unexpected visual stimuli, both at the level of whole-brain responses and for individual visual cortex neurons. Professor Mattingley will discuss the implications of these findings for predictive coding theories of sensory encoding. Professor Jason Mattingley is a Laureate Fellow and Foundation Chair in Cognitive Neuroscience at The University of Queensland. His research is directed toward understanding the brain processes that support perception, selective attention and decision-making, in health and disease.

Human vision flaunts a remarkable ability to recognize objects in the surrounding environment even in the absence of complete visual representation of these objects. This process is done almost intuitively and it was not until scientists had to tackle this problem in computer vision that they noticed its complexity. While current advances in artificial vision systems have made great strides exceeding human level in normal vision tasks, it has yet to achieve a similar robustness level. One cause of this robustness is the extensive connectivity that is not limited to a feedforward hierarchical pathway similar to the current state-of-the-art deep convolutional neural networks but also comprises recurrent and top-down connections. They allow the human brain to enhance the neural representations of degraded images in concordance with meaningful representations stored in memory. The mechanisms by which these different pathways interact are still not understood. In this seminar, studies concerning the effect of recurrent and top-down modulation on the neural representations resulting from viewing blurred images will be presented. Those studies attempted to uncover the role of recurrent and top-down connections in human vision. The results presented challenge the notion of predictive coding as a mechanism for top-down modulation of visual information during natural vision. They show that neural representation enhancement (sharpening) appears to be a more dominant process of different levels of visual hierarchy. They also show that inference in visual recognition is achieved through a Bayesian process between incoming visual information and priors from deeper processing regions in the brain.