Equilibrium Geometry and Chaotic Dynamics in Large Recurrent Neural Networks
Large recurrent networks are important models in several fields, including neuroscience, machine learning, physics, and applied mathematics. Yet their dynamics are difficult to study directly, because high-dimensional nonlinear systems can exhibit rich behavior that is hard to summarize in terms of individual trajectories. In this talk, I will discuss an approach that seeks to understand such dynamics through the structure of the network’s equilibria. I will focus on a random balanced network of threshold-linear units that undergoes a transition from a single stable equilibrium to extensive chaos as the disorder strength crosses a critical value. Using a combination of Kac–Rice theory, replica calculations, numerical root-finding, and dynamical mean-field theory, we show that the chaotic regime contains an exponentially large number of equilibria. These equilibria are all saddles, but with only a fractionally small number of unstable directions. Surprisingly, despite the completely random connectivity, the equilibria are not scattered randomly through phase space. Instead, they are strongly correlated and confined to a comparatively small region. The chaotic attractor lies within this same region, suggesting a direct geometric link between the organization of unstable equilibria and the collective structure of the dynamics. This picture helps explain why networks with extensive chaos can nevertheless display dynamics dominated by a relatively small number of collective modes. More broadly, the results suggest that the geometry of equilibria provides a useful complementary perspective to dynamical mean-field theory for understanding high-dimensional neural dynamics. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2026-05-27. Recording duration: 00:46:40.
Theory, reimagined
Physics offers countless examples for which theoretical predictions are astonishingly powerful. But it’s hard to imagine a similar precision in complex systems where the number and interdependencies between components simply prohibits a first-principles approach, look no further than the challenge of the billions of neurons and trillions of connections within our own brains. In such settings how do we even identify the important theoretical questions? We describe a systems-scale perspective in which we integrate information theory, dynamical systems and statistical physics to extract understanding directly from measurements. We demonstrate our approach with a reconstructed state space of the behavior of the nematode C. elegans, revealing a chaotic attractor with symmetric Lyapunov spectrum and a novel perspective of motor control. We then outline a maximally predictive coarse-graining in which nonlinear dynamics are subsumed into a linear, ensemble evolution to obtain a simple yet accurate model on multiple scales. With this coarse-graining we identify long timescales and collective states in the Langevin dynamics of a double-well potential, the Lorenz system and in worm behavior. We suggest that such an ``inverse’’ approach offers an emergent, quantitative framework in which to seek rather than impose effective organizing principles of complex systems.