ePoster, COSYNE 2023
Signatures of belief representations in recurrent neural networks and prefrontal cortex
Jay Hennig, Sandra Romero Pinto, Scott Linderman, Naoshige Uchida and Samuel Gershman
Harvard University; Psychology; Harvard University; Stanford University
- Montreal, Canada
- Neuroscience, Computational Neuroscience and
Poster
Abstract
A popular model of the basal ganglia uses the theory of reinforcement learning (RL), where phasic dopamine activity encodes reward prediction errors to drive the learning of reward associations. In contrast to classical RL, however, where relevant state information is always provided to the agent, animals must find optimal behaviors using only incomplete state information. One theoretical observation is that animals (or agents) can solve these partially observable tasks by forming "beliefs," or optimal Bayesian posterior estimates of the state of the task. However, it is unknown whether animals could truly learn these theoretical constructs directly from experience, nor is it clear what signatures we might look for in neural data to know whether the brain truly uses a belief representation. To tackle this, we developed a two-pronged approach for linking the theory of beliefs with analyses. Using recurrent neural networks (RNNs), we can build brain-like model systems that must solve the task of estimating value given the same observations as animals. We then use these model systems to develop statistical analyses that determine whether or not the neural representations are consistent with beliefs. Using these analyses, we find that RNNs trained to estimate value do indeed discover belief-like representations. Tentative analysis of neural population activity from mouse cortex suggests cortex also exhibits similar dynamical structure as beliefs and RNNs.
Topics
- basal ganglia, search World Wide
- reinforcement learning, search World Wide
- dopamine, search World Wide
- reward prediction errors, search World Wide
- partially observable tasks, search World Wide
- Bayesian posterior estimates, search World Wide
- recurrent neural networks, search World Wide
- neural population activity, search World Wide
- mouse cortex, search World Wide
Details
- Session
- Poster Session I