Temporal Coding
temporal coding
Dr. Robert Legenstein
The successful candidate will work on learning algorithms for spiking neural networks in the international consortium of the international project 'Scalable Learning Neuromorphics'. We will develop in this project learning algorithms for spiking neural networks for memristive hardware implementations. This project aims to develop scalable Spiking Neural Networks (SNNs) by leveraging the integration of 3D memristors, thereby overcoming limitations of conventional Artificial Neural Networks (ANNs). Positioned at the intersection of artificial intelligence and brain-inspired computing, the initiative focuses on innovative SNN training methods, optimizing recurrent connections, and designing dedicated hardware accelerators. These advancements will uniquely contribute to scalability and energy efficiency. The endeavor addresses key challenges in event-based processing and temporal coding, aiming for substantial performance gains in both software and hardware implementations of artificial intelligence systems. Expected research outputs include novel algorithms, optimization methods, and memristor-based hardware architectures, with broad applications and potential for technology transfer.
Perturbing the spatio-temporal organization of the grid cell network
The emergence and modulation of time in neural circuits and behavior
Spontaneous behavior in animals and humans shows a striking amount of variability both in the spatial domain (which actions to choose) and temporal domain (when to act). Concatenating actions into sequences and behavioral plans reveals the existence of a hierarchy of timescales ranging from hundreds of milliseconds to minutes. How do multiple timescales emerge from neural circuit dynamics? How do circuits modulate temporal responses to flexibly adapt to changing demands? In this talk, we will present recent results from experiments and theory suggesting a new computational mechanism generating the temporal variability underlying naturalistic behavior. We will show how neural activity from premotor areas unfolds through temporal sequences of attractors, which predict the intention to act. These sequences naturally emerge from recurrent cortical networks, where correlated neural variability plays a crucial role in explaining the observed variability in action timing. We will then discuss how reaction times in these recurrent circuits can be accelerated or slowed down via gain modulation, induced by neuromodulation or perturbations. Finally, we will present a general mechanism producing a reservoir of multiple timescales in recurrent networks.
On temporal coding in spiking neural networks with alpha synaptic function
The timing of individual neuronal spikes is essential for biological brains to make fast responses to sensory stimuli. However, conventional artificial neural networks lack the intrinsic temporal coding ability present in biological networks. We propose a spiking neural network model that encodes information in the relative timing of individual neuron spikes. In classification tasks, the output of the network is indicated by the first neuron to spike in the output layer. This temporal coding scheme allows the supervised training of the network with backpropagation, using locally exact derivatives of the postsynaptic spike times with respect to presynaptic spike times. The network operates using a biologically-plausible alpha synaptic transfer function. Additionally, we use trainable synchronisation pulses that provide bias, add flexibility during training and exploit the decay part of the alpha function. We show that such networks can be trained successfully on noisy Boolean logic tasks and on the MNIST dataset encoded in time. The results show that the spiking neural network outperforms comparable spiking models on MNIST and achieves similar quality to fully connected conventional networks with the same architecture. We also find that the spiking network spontaneously discovers two operating regimes, mirroring the accuracy-speed trade-off observed in human decision-making: a slow regime, where a decision is taken after all hidden neurons have spiked and the accuracy is very high, and a fast regime, where a decision is taken very fast but the accuracy is lower. These results demonstrate the computational power of spiking networks with biological characteristics that encode information in the timing of individual neurons. By studying temporal coding in spiking networks, we aim to create building blocks towards energy-efficient and more complex biologically-inspired neural architectures.
The role of temporal coding in everyday hearing: evidence from deep neural networks
COSYNE 2022
The role of temporal coding in everyday hearing: evidence from deep neural networks
COSYNE 2022
Implications of synaptic noise on rate coding and temporal coding in the lateral superior olive: A dynamic-clamp study
FENS Forum 2024