Topic: policy rollouts

When interacting with complex environments, humans can rapidly adapt their behavior to changes in task or context. This flexibility has been suggested to result from fast recurrent dynamics in the prefrontal network, acquired through a process of meta-reinforcement learning (Wang et al., 2018). However, these models fail to capture an important aspect of human behavior: when adapting to new information, we spend variable and often long periods of time contemplating possible futures before acting. For such planning to be rational, its benefits to future behavior must at least compensate for the