ePoster, COSYNE 2023
A cortical microcircuit for reinforcement prediction error
Quentin Chevy, Rui Ponte Costa, Zoltan Szadai, Rozsa Balazs and Adam Kepecs
Washington University School of Medicine; Neuroscience; University of Bristol; Institute of Experimental Medecine, Budapest
- Montreal, Canada
- Neuroscience, Computational Neuroscience and
Abstract
Although distinct cortical regions specialize in different functions, they also benefit from receiving global reinforcement feedback to tune local processing. When and how those reinforcement signals shape cortical computations remains however largely unknown. Here, we present experimental evidence showing that acetylcholine inputs throughout the cortex form the backbone for a global signal, that presents the main characteristic of an unsigned reinforcement prediction error computation : modulation of the reinforcement response by expectation and coding of predictive cues. We show that this global signal is then integrated with local information by VIP interneurons, a specific GABAergic cell type that promotes plasticity through disinhibition, and that is known to respond to primary reinforcers across cortex. Thanks to a neural circuit involving local somatostatin (SST) interneurons, local sensory inputs indeed appear to null out the predictive cue coding transmitted by the cholinergic system to the VIP interneurons. To test the computational functions of this cortical circuit motif involving local VIP and SST inhibitory neurons and global acetylcholine, we designed a simple cortex-inspired reinforcement learning model. When we add disinhibitory VIP interneurons as a local relay for global reinforcement, our model performance improves due to an adaptable learning rate as well as protection against forgetting. Overall, our results provide new experimental and theoretical evidence for a microcircuit implementation of adaptive reinforcement learning in the cortex.
Topics
- acetylcholine, search World Wide
- reinforcement prediction error, search World Wide
- cortical computations, search World Wide
- VIP interneurons, search World Wide
- GABAergic cell type, search World Wide
- disinhibition, search World Wide
- somatostatin (SST) interneurons, search World Wide
- local sensory inputs, search World Wide
- reinforcement learning model, search World Wide
Details
- Session
- Poster Session III