Resources
Authors & Affiliations
Julia Costacurta, Yu Duan, John Assad, Kanaka Rajan, Scott Linderman
Abstract
Neuromodulatory signals are powerful and prevalent influences on behavior. For example, dopaminergic inputs to the dorsolateral striatum are correlated with action selection and invigoration, and the loss of dopaminergic drive in Parkinson's disease manifests in deficits of movement timing and initiation. While the role of dopamine in the cortex-basal ganglia-thalamaus (C-BG-T) loop has been explored within the framework of reinforcement learning, computational models of how dopamine might influence fast-timescale movement generation have not been explored to the same extent. Here, we propose a multi-region neuromodulated RNN (NM-RNN) model, which consists of four linked subnetworks corresponding to the cortex, basal ganglia, thalamus, and substantia nigra pars compacta (SNc). Our model incorporates the connectivity pattern and excitatory/inhibitory constraints of biological C-BG-T circuits. We hypothesize that on fast timescales, the dopaminergic signal from SNc to BG acts by multiplicatively scaling the weights in the BG subnetwork. Since dopamine is implicated in the execution of timed movements, we train this network on a timing reproduction task where the network must measure and then reproduce a particular interval. We compare to RNNs with no neuromodulation and/or region-specificity. The multi-region NM-RNN outperforms vanilla RNNs on both trained and unseen interval reproduction. We next investigated how the SNc subnetwork contributes to computation. We found that the multi-region NM-RNN learns to use the SNc subnetwork to store timing information, analogous to the proposed role of dopamine in biological systems. Our results indicate how a multi-region RNN based on C-BG-T loop in the brain can yield novel solutions that generalize effectively and suggest new mechanistic hypotheses about the role of neuromodulation. This work contributes to the goal of understanding how dopamine controls rapid changes in network dynamics and eventual movement outputs.