Connectome-constrained cortical circuits optimized for visual function and working memory tasks

Recurrent neural networks (RNNs) trained on working memory (WM) tasks have been the subject of extensive study. These networks typically contain one or more aspects of biological-implausibility, such as the adoption of saturating output functions (tanh), violation of Dale's Law or introduction of artificial elements (LSTM). They also have over-simplistic input pathways such as binary cues, which are insufficient to represent the rich dynamics found in sensory brain areas. In particular, the role of inter-areal communication between sensory and association areas for WM has been critically underrepresented in these models. Here, we address these problems by constructing RNNs comprising of 15 sensory or association brain areas. We differentiate them by constraining network weights to match experimental inter-areal connectomic data while also constraining single-area recurrent weights to follow a biologically-motivated gradient of synaptic excitation. Stimulus is presented to the network through information-maximized Gabor receptive fields of V1 excitatory cells. We identified an approximate dorsal and ventral stream from these areas and trained them to determine the location and shape of stimuli respectively. Concurrently, each network is also trained on one of 11 WM tasks with a common set of readout weights shared across all networks. We find that sensory areas in trained networks exhibit higher-dimensional dynamics compared to association areas and have similar activities regardless of task. Based on behaviors that emerged from our networks during training, we propose generalizations to two theories that originated from analyzing handcrafted models: (1) counterstream inhibitory bias, where inter-areal excitatory-inhibitory feedback connections are stronger than excitatory-excitatory connections; and (2) bifurcation in space, where sensory and association areas produce functionally-distinct dynamics at all stages of WM tasks. Our approach represents a new direction for task-optimized RNNs, where regular functions are incorporated into multi-area networks under a single architecture in order to meaningfully analyze within-class inter-areal dynamics.