Topic: Partially observed Markov Decision Process

Neural mechanisms of adaptive behavior

Animals and humans rapidly adapt their behavior to dynamic environmental changes, such as predator threats or fluctuating food resources, often without immediate rewards. Existing literature posits that animals rely on internal representations of the environment, termed “beliefs”, for their decision policy. However, previous work ties belief updates to external reward signals, which does not explain adaptation in scenarios where trial-and-error approaches are inefficient or potentially perilous. In this work, we propose that the brain utilize dynamic representations that continuously infer the state of the environment, allowing it to update behavior rapidly. I will present a Bayesian theory for state inference in a partially observed Markov Decision Process with multiple interacting latent variables. Optimal behavior requires knowledge of hidden interactions between latent states. I will show that recurrent neural networks trained through reinforcement solve the task by learning the hidden interaction between latent states, and their activity encodes the dynamics of the optimal Bayesian estimators. The behavior of rodents trained on an identical task aligns with our theoretical model and neural network simulations, suggesting that the brain utilizes dynamic internal state representation and inference. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2024-01-31. Recording duration: 00:51:21.