Equilibrium Geometry and Chaotic Dynamics in Large Recurrent Neural Networks

Large recurrent networks are important models in several fields, including neuroscience, machine learning, physics, and applied mathematics. Yet their dynamics are difficult to study directly, because high-dimensional nonlinear systems can exhibit rich behavior that is hard to summarize in terms of individual trajectories. In this talk, I will discuss an approach that seeks to understand such dynamics through the structure of the network’s equilibria. I will focus on a random balanced network of threshold-linear units that undergoes a transition from a single stable equilibrium to extensive chaos as the disorder strength crosses a critical value. Using a combination of Kac–Rice theory, replica calculations, numerical root-finding, and dynamical mean-field theory, we show that the chaotic regime contains an exponentially large number of equilibria. These equilibria are all saddles, but with only a fractionally small number of unstable directions. Surprisingly, despite the completely random connectivity, the equilibria are not scattered randomly through phase space. Instead, they are strongly correlated and confined to a comparatively small region. The chaotic attractor lies within this same region, suggesting a direct geometric link between the organization of unstable equilibria and the collective structure of the dynamics. This picture helps explain why networks with extensive chaos can nevertheless display dynamics dominated by a relatively small number of collective modes. More broadly, the results suggest that the geometry of equilibria provides a useful complementary perspective to dynamical mean-field theory for understanding high-dimensional neural dynamics.