A cortical microcircuit for reinforcement prediction error

Although distinct cortical regions specialize in different functions, they also benefit from receiving global reinforcement feedback to tune local processing. When and how those reinforcement signals shape cortical computations remains however largely unknown. Here, we present experimental evidence showing that acetylcholine inputs throughout the cortex form the backbone for a global signal, that presents the main characteristic of an unsigned reinforcement prediction error computation : modulation of the reinforcement response by expectation and coding of predictive cues. We show that this global signal is then integrated with local information by VIP interneurons, a specific GABAergic cell type that promotes plasticity through disinhibition, and that is known to respond to primary reinforcers across cortex. Thanks to a neural circuit involving local somatostatin (SST) interneurons, local sensory inputs indeed appear to null out the predictive cue coding transmitted by the cholinergic system to the VIP interneurons. To test the computational functions of this cortical circuit motif involving local VIP and SST inhibitory neurons and global acetylcholine, we designed a simple cortex-inspired reinforcement learning model. When we add disinhibitory VIP interneurons as a local relay for global reinforcement, our model performance improves due to an adaptable learning rate as well as protection against forgetting. Overall, our results provide new experimental and theoretical evidence for a microcircuit implementation of adaptive reinforcement learning in the cortex.