Topic: reservoir computing

Reservoir computing is a promising framework to study cortical computation, as it is based on continuous, online processing and the requirements and operating principles are compatible with cortical circuit dynamics. However, the framework has issues that limit its scope as a generic model for cortical processing. The most obvious of these is that, in traditional models, learning is restricted to the output projections and takes place in a fully supervised manner. If such an output layer is interpreted at face value as downstream computation, this is biologically questionable. If it is interpr

Neurons in the brain exhibit significant heterogeneity in their dynamics. Whether this diversity plays any computational role and if it is exploited to perform efficient computation is not well understood. We used the reservoir computing paradigm to answer this question. Balanced networks with different levels of heterogeneity in their neuronal time constants were set up and benchmarked against various working-memory-related tasks, including stimulus recall, forecasting, and nonlinear transformation. Our results suggest that neuronal heterogeneity, even without any recurrent communication, si

With the recent success of Transformers in the field of automatic natural language processing [1], it has become clear that we are able to create complex, high-performance neural network architectures that can extract, combine and transform information to meet a specific objective, such as predicting the next word in a text. However, to function, Transformers need to receive a context of all N elements of the past sequence (N = 2k to 32k tokens) at each time step, causing the number of computations to grow quadratically (N^2) with the length of the sequence [1]. This "perfect" memory of a co