Spiking neural networks

…engineering through research on spiking neural networks. It examines artificial and biologically plausible learning algorithms and how trained spiking circuits can reveal principles of neural processing. Two afternoons combine four invited talks, seven contributed-talk slots, flash presentations and a virtual poster session, with time for discussion. Invited speakers are Susanne Schreiber, Mihai Petrovici, Eugene Izhikevich and Giulia D’Angelo. The programme begins at 14:00 CET on both 4 and 5 November;

…NeuroAI, including theory of neural circuits and its experimental support, local learning and plasticity, and connectomics; neuroscience-ins

…unsupervised learning architecture for spiking neural networks. In contrast to conventional predictive coding approaches, which only transmit prediction errors to higher processing stages, PCL learns inhibitory lateral and top-down connectivity to suppress the most predictable spikes and passes a compressed representation of the input to higher processing stages. We show that PCL reproduces a range of biological findings and exhibits a favorable tradeoff between energy consumption and downstream classification p

Spiking neural networks (SNNs) promise energy efficiency and offer a more brain-like alternative to artificial neural networks. However, training them on challenging tasks requiring credit assignment across long time horizons remains a significant challenge due to their discrete spiking interactions, which make the error landscape non-differentiable. A recent approach replaces the spike interactions with a surrogate gradient in the backward pass of gradient descent.\footnote{Neftci et al., \textit{Surrogate Gradient Learning in Spiking Neural Networks}, IEEE Signal Processing Magazine, 2019}\f

Our ability to form and retrieve memories relies on a network of connected brain regions. In particular, neuronal ensembles across regions form distributed memory engrams that have structural and functional connectivity. However, the computational principles underlying the distributed nature of memory remain unclear. Here, we show that distributed engrams enable memory generalization and discrimination in parallel across brain regions. Orthogonal and complementary, generalization and discrimination are computations that must be balanced for adaptive, memory-guided behavior. For instance, while

…robustness and generalizability in neural networks. Previous work from our group has shown that a combinatorial neuronal code preserves the relational geometry of acoustic stimuli in the songbird brain. Here we show that combinatorial codes in autoencoding spiking neural networks (SNNs) can maintain relational invariance and display a hierarchical structure that optimizes the network’s internal representation. B

…The neural mechanisms underlying RL in spatial navigation has received limited attention and it remains unclear what neural mechanisms maintain behavioral flexibility. To address this issue, we extended an existing closed-loop simulator of spatial navigation and learning [1], based on spiking neural networks. The activity of place cells and boundary cells were fed as inputs to action selection neurons, which

Neural population dynamics are jointly shaped by numerous factors at the cellular, synaptic, and network levels. Mechanistic models, especially spiking neural networks, are crucial for understanding how these interacting variables influence neural activity. However, the process of adjusting model parameters to match experimental recordings, i.e., inverse modeling, is challenging, and often requires a trade-off between model complexity and tractability. Consequently, models often use fixed or manually tuned parameters, limiting their dynamic range and explanatory power while making cross-study

Spiking neural networks (SNNs) inherently rely on the timing of signals for representing and processing information. Transmission delays play an important role in shaping these temporal characteristics. Recent work [1, 2] has demonstrated the substantial advantages of learning these delays along with synaptic weights, both in terms of accuracy and memory efficiency. However, these approaches suffer from drawbacks in terms of precision and learning efficiency, as they operate in discrete time and with approximate gradients, while also requiring membrane potential recordings for calculating para

…To meet these processing requirements, spiking neural networks (SNNs) seem an attractive solution due to their minimal preprocessing requirements of cortical spike trains (CSTs) and their suitability for ultra-low-power, low-

…through the development of artificial neural networks (ANNs), has significantly benefited from insights gained from neuroscience. This progress has enhanced the replication of complex cognitive tasks such as vision and natural language processing (1,2). However, ANNs still struggle with continual learning, adaptable knowledge transfer, robustness, and resource efficiency - capabilities that biological systems handle seamlessly (3–8). Neuromodulators, the chemical messengers that target neurons and synapses in specific brain regions,

Spiking neural networks (SNNs) are characterized by low power consumption, highly parallel computation and event-driven information processing [1]. Many of the concepts that promise to improve the efficiency of SNNs compared to conventional deep neural networks, such as spikes, sparsity, and static suppression, are inspired by our understanding of how the brain processes information. Hence, we can further advance the development of SNNs by improving our current knowledge of how biological neural networks learn through spike-based local plasticity mechanisms. In recent years, many spike-based

…intricate functions of complex networks. In situations where direct measurement of structural connectivity is limited, like in cortical neuronal networks, causal inference tools such as transfer entropy are employed to reconstruct structural connectivity from network activity data. However, the causal connectivity inferred may not always align with the network's underlying structural connectivity, leaving the interpretation of inferred causal connectivity open for further clarification. Specifically, the relationship be

…CHL, developed for artificial networks, neurons learn in a two-step procedure where their free response to a given input is contrasted to their clamped response given the same input and an imposed target output. This two-step procedure, which is unlikely to be implementable in biological systems, has recently been made more plausible by exploiting the fundamental idea that neurons can learn by predicting their own future activity [1

…for efficient instantiations of neural network models in parallel computers [8]. While this approach neglects

…of heterogeneity of time constant in spiking neural networks (SNNs) [1, 2]. These findings suggest that heterogeneity, i.e. neuronal variability, should be embraced and leveraged to improve the performance of event-based computation. At the same time, Machine Learning teaches us that Neural Networks are surprisingly capable of representing abstract projections of input data, creating a hierarchy of representation in successive neural network layers. This is why Neural Networks have become much more complex, especi

The rise of spiking neural networks (SNNs) offers a promising alternative to traditional activation-based deep neural networks, advancing the pursuit of more sustainable and ethical computing architectures. Despite its potential, neuromorphic event-based computing, which promises to fully harness the benefits of spiking networks, remains largely confined to research settings, with commercialization hindered by the lack of user-friendly software tools for developers. While initiatives like Intel's Lava for Loihi and PySpiNNaker2 for deploying SNNs on the SpiNNaker 2 neuromorphic chip are promis

The ability to train spiking neural network models is essential for the modeling of biological neural networks as well as for neuromorphic computing. The standard approach when training non-spiking neural networks is gradient descent. In spiking neural networks, however, gradient descent learning is complicated by the all-or-none character of spikes (Fig. A). It can lead to unexpected (dis-)appearances of spikes during training, resulting in disruptive, discontinuous changes in the network dynamics [1]. Further, it seemingly prohibits exact gradients to systematically generate or remove spikes

Spiking Neural Networks (SNNs) promise energy efficiency and offer a more brain-like alternative to artificial neural networks. A major challenge to unleashing their potential, however, is to make them trainable on challenging tasks that require credit assignment across long time horizons. Gradient-based learning, which has driven much of the recent success in ANNs, is hampered in spiking networks by their discrete spiking interactions, which make the error landscape non-differentiable. A recent approach to this challenge is to replace the spike interaction with a surrogate gradient in the bac

Various models of spiking neural networks have been proposed in the literature. These models are governed by their parameters, such as time constants, input currents, and synaptic weights. While we have a good understanding of how these parameters affect the behavior of isolated neurons, a similar intuition for connected groups of neurons is lacking. Here, we simulated millions of neuronal network models using a range of parameters, building a library of dynamics that catalogs network parameters and their resulting dynamics. Towards this end, we collected spike-trains by simulating network mod

…information while enhancing the neural representation of relevant stimuli. This phenomenon is well-explained by the theory of biased competition, which posits that attention biases neural processing towards relevant stimuli when multiple objects are present. However, the mechanism of attention remains under debate. We show how biased competition can be solved in a bio

…In this talk, we examine nonlinear networks characterized by pulse signal outputs, such as spiking neural networks, using four p

Variations in cellular and network parameters shape neural dynamics and computation. Mechanistic models, such as spiking neural networks (SNNs), are instrumental for linking recordings of neural population dynamics to their underlying circuit parameters. However, SNNs lack analytical expressions and accessible gradients for parameter inference (i.e., model identification), making it difficult to systematically identify a single configuration that can reproduce experimental data, let alone the full space of data-consistent models and their uncertainty. As a result, SNNs are often reduced a prio

Biologically plausible spiking neural networks (SNNs) are an emerging architecture for deep learning tasks due to their energy efficiency when implemented on neuromorphic hardware. However, many of the biological features are at best irrelevant and at worst counterproductive when evaluated in the context of task performance and suitability for neuromorphic hardware. In this talk, I will present an alternative paradigm to design deep learning architectures with good task performance in real-world benchmarks while maintaining all the advantages of SNNs. We do this by focusing on two main feature