Deep neural networks

Predictive processing offers a unifying view of neural computation, proposing that brains continuously anticipate sensory input and update internal models based on prediction errors. In this talk, I will present converging evidence for the computational mechanisms underlying this framework across human neuroscience and deep neural networks. I will begin with recent work showing that large-scale distributed prediction-error encoding in the human brain directly predicts how sensory representations reorganize through predictive learning. I will then turn to PredNet, a popular predictive coding in

Task-trained deep neural networks have emerged as leading models of neural responses in the primate visual system, especially for later stages of the ventral stream such as inferotemporal (IT) cortex. One major criticism of this approach is that the tasks used to train such networks rely on large numbers of labeled examples, and are thus not ecologically plausible. Recent approaches in representation learning circumvent the need for labeled examples, and match or surpass supervised learning methods on a variety of tasks. These approaches generally rely on supervision signals extracted from the

Deep neural networks (DNNs) are extremely vulnerable to adversarial attacks, where small changes to the input, unnoticeable to humans, can drastically alter the model’s predictions. One reason for their poor performance is that they process as much information as possible and make decisions based on the input received. Human vision performs differently. It tries be as efficient with information as possible while using predictions to fill in missing information based on existing knowledge. The brain may even suppress sensory input to save energy if it is familiar (i.e., predictable). As the kn

Understanding how neural representations align between biological and artificial systems has emerged as a central challenge in computational neuroscience. While deep neural networks now reliably predict neural responses across multiple brain areas, their utility for understanding biological computation remains limited by our ability to accurately measure representational similarities. This limitation becomes particularly acute when working with sparse neural recordings, where traditional similarity metrics may fail to capture true representational relationships. Recent spectral analyses of sim

…In recent years, deep neural networks (DNNs) have set new standards in predicting neuronal responses of the visual system to arbitrary stimuli, making in silico exploration of the stimulus-response space possible. As visual input is dynamic, there is a need for models that can faithf

Detecting a specific object in a visual scene is a task that the human visual system performs every day. To perform this target-specific object detection, it is necessary to filter out all unwanted objects during the processing, so that only the specific object can be detected. For this purpose, attention-related mechanisms along the visual pathway have been proposed to allow focusing on the desired object. Beuth and Hamker (2015) [1], using a biologically inspired recurrent model simulating lower and higher layers of the visual cortex, the frontal eye field, and the prefrontal cortex, showed

In modern deep neural networks, the intricate learning dynamics of individual neurons often remain obscured due to global optimization methods. This contrasts with biological neural networks, which achieve robustness and efficiency through self-organized, local learning processes using limited global information. To bridge this gap, a local learning framework inspired by the self-organizing principles of biological neurons has been proposed [1]. This framework allows for the investigation of how local information-processing goals can drive the emergence of complex network-level functions. The

Recent research has seen many behavioral comparisons between humans and deep neural networks (DNNs) in the domain of image classification. Often, comparison studies focus on the end-result of the learning process by measuring and comparing the similarities in the representations of object categories once they have been formed. However, the process of how these representations emerge—that is, the behavioral changes and intermediate stages observed during the acquisition—is less often directly and empirically compared. In this talk, I'm going to report a detailed investigation of the learning dy

The power of deep neural networks for pattern recognition tasks lies in their universal approximation abilities, and their theoretical expressivity is well-studied. Practical expressivity, however, is known to lag behind theoretical expressivity. Network performance achieved in practice is determined by a set of inductive biases such as architectural constraints (e.g. residual connections). In this work, we introduce weakly coupled residual recurrent networks (WCRNNs) in which the residual connections result in well-defined Lyapunov exponents. This allows for studying properties of their dynam

…Deep neural networks provide high performance at the cost of interpretability of learned features because of black box approaches. In this project, we evaluated how the interpretability of networks for electroencephalography (EEG) data is affected by the network architecture and the method for feature extraction and visualisation. To this aim, we used two convolutional neural network architectures: (1)ResNet, a residu

…can be predicted with high accuracy by deep neural networks trained on challenging visual classification problems. Similar results have been obtained for voxels of auditory cortex with fMRI [Kell 2018]. But the relatively poor time resolution of fMRI precludes capturing temporal structure below about 1 Hz, which is known to be an important characteristic of auditory processing [Downer 2020]. Here we investigate the relationship between sequences o

…Recent work has shown that deep neural networks are able to learn similar representations to those measured in visual areas in animals, suggesting they may serve as models for the brain. Varying network architectures and loss functions has been shown to modulate the biological similarity learned representations, however the extent to which this results from exposure to natural image statistics during training has not been fully characterized

…tracking is fundamental to unravel the neural basis of behavior. The recent growth of Deeplabcut, an open source toolbox for marker-less pose estimation based on transfer learning with deep neural networks, promises to greatly enhance our ability to track animal pose with high-accuracy and high-efficiency, because it can match human labelling accuracy with minimal training data. Here, we implement Deeplabcut routines and develop a pipeline to accurately track the pose of mice undergoing defensive behaviors. Defensive behaviors are elicited

Neural decoding uses brain activity to infer internal representations of the environmental states or predict behavior. Our study aimed to decode neural signal from the Posterior Parietal Cortex (V6A), to investigate intention coding and control of assistive devices. Data were recorded during delayed reaching in three different gaze/target conditions allowing to dissociate the retinotopic and the oculomotor reference frames. We used Deep Neural Networks to learn to decode the lateral displacement and the depth of the reached target from neural data averaged over four epochs, including pre-fixat

We examine how the saccade mechanism from biological vision can be used to make deep neural networks more efficient for classification and object detection problems. Our proposed approach is based on the ideas of attention-driven visual processing and saccades, miniature eye movements influenced by attention. We conduct experiments by analyzing: i) the robustness of different deep neural network (DNN) feature extractors to partially-sensed images for image classification and object detection, and ii) the utility of saccades in masking image patches for image classification and object tracking.

…meaningful internal representations in deep sensory networks with plastic synaptic connections. Experience-dependent plasticity presumably exploits temporal contingencies between sensory inputs to build these internal representations. However, the precise mechanisms underlying plasticity remain elusive. We derive a local synaptic plasticity model inspired by self-supervised machine learning techniques that shares a deep conceptual connection to Bienenstock-Coo

The internal activations of particular deep neural networks (DNNs) are remarkably similar to the neuronal population responses along the ventral visual cortex in primates. Nevertheless, the similarities between the two are often investigated through stimulus sets consisting of everyday objects under naturalistic settings. Recent work has revealed a gap in generalization ability of these models in predicting neuronal responses to out-of-distribution (OOD) samples (i.e. samples that are not regarded as natural photos) . Here, we investigated how the recent progress in improving DNNs’ object re

The ability to compare two magnitudes is observed in naïve animals in the absence of learning (Rugani 2016; McCrink 2007), but how this cognitive function emerges without training remains elusive. In monkeys, neurons in the prefrontal and parietal cortex were observed to respond selectively to the proportion between two magnitudes (Vallentin 2008, 2010), suggesting these units could be fundamental for innate magnitude comparisons. However, details of how these comparison units emerge without any learning also remain unclear. Here, we show that magnitude comparison units can arise spontaneously

Understanding synaptic plasticity is critical for elucidating mechanisms for neural development, learning and memory consolidation. Synaptic plasticity is modelled by simulating the activity of interacting neural populations, using hypothesized functions (plasticity rules) that modify the connections between neurons. Plasticity rules are typically hand-crafted from single-synapse experiment data, but have shown limited success in understanding network-level properties. Recent studies aim to discover rules using supervised learning rather than hand-crafting them. However, these approaches stil

The best current models of visual cortical neural responses rely on internal representations of large deep neural networks (DNNs) trained for object recognition. However, the inner workings of these DNN models are mostly uninterpretable due to their massive size—tens of millions of parameters. Do we need such large models in the first place? In this work, we focused on modeling visual cortical neurons in macaque mid-level visual area V4. We used ensemble learning and closed-loop experiments with active learning to train a DNN model to accurately predict V4 responses. However, our model had 90

Deep neural networks are commonly used as models of the ventral visual stream, but are less explored in audition. Prior work provided examples of audio-trained neural networks that produce good predictions of fMRI responses in auditory cortex, and exhibit correspondence between model stages and brain regions, but left it unclear the extent to which these results would generalize to other audio neural network models. We evaluated brain-model correspondence for a wide range of publicly available high-performing audio neural network models along with a set of models that we trained on four differ

Deep neural networks (DNNs) with the flexibility to learn good top-layer representations have eclipsed shallow kernel methods without that flexibility. Here, we take inspiration from deep neural networks to develop a new family of deep kernel method. In a deep kernel method, there is a kernel at every layer, and the kernels are jointly optimized to improve performance (with strong regularisation). We establish the representational power of deep kernel methods, by showing that they perform exact inference in an infinitely wide Bayesian neural network or deep Gaussian process. Next, we conje

Real-time rendering of closed-loop visual environments is important for next-generation understanding of brain function and behaviour, but is often prohibitively difficult for non-experts to implement and is limited to few laboratories worldwide. We developed BonVision as an easy-to-use open-source software for the display of virtual or augmented reality, as well as standard visual stimuli. BonVision has been tested on humans and mice, and is capable of supporting new experimental designs in other animal models of vision. As the architecture is based on the open-source Bonsai graphical program