How Symmetry and Self-Coupling Shape Dynamics and Trainability of Recurrent Neural Networks
Matthew Ding & Rainer Engelken · COSYNE 2023
The connectivity between neurons plays a major role in the dynamics and function of all neural systems. Here, we investigate the joint effects of two previously reported connectivity motifs, partial symmetry and self-coupling, on the dynamics of recurrent neural networks (RNNs). Both motifs can be thought of as statistical deviations from i.i.d. connectivity. In particular, we define self-coupling as the values of diagonal entries and partial symmetry as the correlations across the diagonals of coupling matrices. We calculate the full Lyapunov spectrum to characterize the stability of perturba
Sat, Mar 11, 2023