Reservoir computing

Despite wide recognition of the modular and hierarchical organization of neural circuits in the brain, our understanding of its influence on neural dynamics and information processing remains incomplete. To address this gap, we introduced a model of randomly connected neural populations (modules) and studied its dynamics by means of the mean-field theory and simulations. Our analysis uncovered a rich phase diagram: High-dimensional chaotic activity (microscopic chaos), common in homogeneous networks, is separated from the low-dimensional chaotic activity of strongly coherent modules (macroscop

The synaptic connection strengths in the brain largely follow a heavy-tailed distribution, whereas most theoretical frameworks assume a Gaussian distribution due to the mathematical complexity involved. Understanding how these heavy-tailed distributions influence neural dynamics is crucial for developing more biologically realistic models and theories that better capture the brain's properties. To bridge this gap, we provide both theoretical and computational insights into the dynamics shaped by heavy-tailed synaptic weights through mathematical derivations and case studies in recurrent neural

Reservoir computing is a machine learning framework in which only the output layer is trained to perform a task; the rest – composed of an input layer and recurrent neural network – has fixed connections. The benefit of restricting training to the output layer is that it does not require the computationally expensive and biologically-implausible backpropagation typically used to train deep-learning networks. However, while reservoirs can do tasks well, their performance is unreliable; depending on the random initialisation of fixed weights within the network, their capabilities vary wildly, si

…We used the reservoir computing paradigm to answer this question. Balanced networks with different levels of heterogeneity in their neuronal time constants were set up and benchmarked against various working-memory-related tasks, including stimulus recall, forecasting, and nonlinear transformation. Our results suggest that neuronal heterogeneity, even without any recurrent communication, si

With the recent success of Transformers in the field of automatic natural language processing [1], it has become clear that we are able to create complex, high-performance neural network architectures that can extract, combine and transform information to meet a specific objective, such as predicting the next word in a text. However, to function, Transformers need to receive a context of all N elements of the past sequence (N = 2k to 32k tokens) at each time step, causing the number of computations to grow quadratically (N^2) with the length of the sequence [1]. This "perfect" memory of a co

Reservoir computing facilitates the integration of high-dimensional spatiotemporal dynamics with information processing [1]. Here we investigated the correlation between neuronal dynamics and machine learning task performance, utilizing a cultured neuronal network with a modular structure. The results show that the modularity of the neuronal network enhances classification performance in tasks involving multiple speech signals [3].Figure 1 outlines the experimental setup, employing a modular cultured neuronal network of rat cortical neurons on a patterned substrate. Neural activity was measure

…designed a random network, based onthe reservoir computing paradi

The specific topology of biological neural networks is believed to be one of the most important ingredients of the nervous system's computational capabilities. Furthermore, multiple experimental studies have demonstrated that neural connectivity is highly inhomogeneous, following different patterns between different regions and neuron types. In particular, the presence of clusters of highly interconnected neurons is one of the most well-established connectivity patterns across multiple brain areas. Still, the functional implications of such brain connectivity features, especially for the perfo

Reservoir computing is a promising framework to study cortical computation, as it is based on continuous, online processing and the requirements and operating principles are compatible with cortical circuit dynamics. However, the framework has issues that limit its scope as a generic model for cortical processing. The most obvious of these is that, in traditional models, learning is restricted to the output projections and takes place in a fully supervised manner. If such an output layer is interpreted at face value as downstream computation, this is biologically questionable. If it is interpr