Dynamical representations between biologically plausible and implausible task-trained neural networks

Recurrent neural networks (RNNs) are proving increasingly useful in systems neuroscience. Training RNNs on cognitive-like tasks has demonstrated their potential to infer dynamical processes and computations in the brain [1, 2]. They have additionally been shown to provide an interpretable link between connectivity structure and activity dynamics in low-rank networks [3]. However, a potentially serious shortcoming is that many (if not most) of these networks disobey biological constraints in terms of their connectivity structure and response properties, which raises questions as to their interpretational relevance.

We began exploring these questions by training networks which include specific biological constraints. Particularly, we trained networks with separate excitatory and inhibitory units, as well as unconstrained networks in which units may be simultaneously excitatory and inhibitory, on commonly used behavioral tasks [4]. We then compared the learned dynamics and connectivity structure between the two networks to determine the effects of these simple constraints [5]. First, we determined the extent to which dynamics learned in unconstrained networks have predictive power in the more realistic biological networks. Second, we explored whether and how one can map solutions from one network type to the other [6]. This work additionally lays the foundation to examine how network constraints affect learning in neural circuits.