Topic: dynamical mean-field theory

A high-dimensional nonlinear recurrent neural network is driven by periodic, incoherent inputs while its synapses undergo Hebbian-like plasticity. Depending on drive and plasticity strengths, the network can retain autonomous limit-cycle activity after both the inputs and learning are removed. Dynamical mean-field theory and finite-size simulations characterize the resulting rhythmic memories and transitions between chaotic, fixed-point and oscillatory regimes. Memory robustness depends on the interval between removing the input and stopping plasticity. These results suggest a mechanism by whi

Large recurrent networks are important models in several fields, including neuroscience, machine learning, physics, and applied mathematics. Yet their dynamics are difficult to study directly, because high-dimensional nonlinear systems can exhibit rich behavior that is hard to summarize in terms of individual trajectories. In this talk, I will discuss an approach that seeks to understand such dynamics through the structure of the network’s equilibria. I will focus on a random balanced network of threshold-linear units that undergoes a transition from a single stable equilibrium to extensive ch

Computations in neural circuits are often construed as being implemented through the coordinated dynamics of neurons. In this picture, the role of synaptic connectivity is to sculpt neuronal dynamics to implement computations of interest. Of course, synapses are not static but change on a variety of timescales, including fast timescales comparable to those of neurons. Thus, a more accurate view of computation in neural circuits may involve the coupled dynamics of neurons and synapses. This form of computation is closer to what is implemented by Transformers via an equivalence between ongoing s

Recurrent neural networks (RNNs) are powerful dynamical models, widely used in machine learning (ML) for processing sequential data, and also in neuroscience, to understand the emergent properties of networks of real neurons. Prior theoretical work in understanding the properties of RNNs has focused on models with additive interactions. However, real neurons can have gating i.e. multiplicative interactions, and gating is also a central feature of the best performing RNNs in machine learning. Here, we develop a dynamical mean-field theory (DMFT) to study the consequences of gating in RNNs. We u