Hippocampal Replay
hippocampal replay
A recurrent network model of planning predicts hippocampal replay and human behavior
When interacting with complex environments, humans can rapidly adapt their behavior to changes in task or context. To facilitate this adaptation, we often spend substantial periods of time contemplating possible futures before acting. For such planning to be rational, the benefits of planning to future behavior must at least compensate for the time spent thinking. Here we capture these features of human behavior by developing a neural network model where not only actions, but also planning, are controlled by prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences drawn from its own policy, which we refer to as `rollouts'. Our results demonstrate that this agent learns to plan when planning is beneficial, explaining the empirical variability in human thinking times. Additionally, the patterns of policy rollouts employed by the artificial agent closely resemble patterns of rodent hippocampal replays recently recorded in a spatial navigation task, in terms of both their spatial statistics and their relationship to subsequent behavior. Our work provides a new theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by -- and in turn adaptively affect -- prefrontal dynamics.
A recurrent network model of planning explains hippocampal replay and human behavior
When interacting with complex environments, humans can rapidly adapt their behavior to changes in task or context. To facilitate this adaptation, we often spend substantial periods of time contemplating possible futures before acting. For such planning to be rational, the benefits of planning to future behavior must at least compensate for the time spent thinking. Here we capture these features of human behavior by developing a neural network model where not only actions, but also planning, are controlled by prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences drawn from its own policy, which we refer to as 'rollouts'. Our results demonstrate that this agent learns to plan when planning is beneficial, explaining the empirical variability in human thinking times. Additionally, the patterns of policy rollouts employed by the artificial agent closely resemble patterns of rodent hippocampal replays recently recorded in a spatial navigation task, in terms of both their spatial statistics and their relationship to subsequent behavior. Our work provides a new theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by - and in turn adaptively affect - prefrontal dynamics.
The medial prefrontal cortex replays generalized sequences
Whilst spatial navigation is a function ascribed to the hippocampus, flexibly adapting to a change in rule depends on the medial prefrontal cortex (mPFC). Single-units were recorded from the hippocampus and mPFC of rats shifting between a spatially- and cue-guided rule on a plus-maze. The mPFC population coded for the relative position between start and goal arm. During awake immobility periods, the mPFC replayed organized sequences of generalized positions which positively correlated with rule-switching performance. Conversely, hippocampal replay negatively correlated with performance and occurred independently of mPFC replay. Sequential replay in the hippocampus and mPFC may thus serve different functions.
Hippocampal replay reflects specific past experiences rather than a plan for subsequent choice
Executing memory-guided behavior requires storage of information about experience and later recall of that information to inform choices. Awake hippocampal replay, when hippocampal neural ensembles briefly reactivate a representation related to prior experience, has been proposed to critically contribute to these memory-related processes. However, it remains unclear whether awake replay contributes to memory function by promoting the storage of past experiences, facilitating planning based on evaluation of those experiences, or both. We designed a dynamic spatial task that promotes replay before a memory-based choice and assessed how the content of replay related to past and future behavior. We found that replay content was decoupled from subsequent choice and instead was enriched for representations of previously rewarded locations and places that had not been visited recently, indicating a role in memory storage rather than in directly guiding subsequent behavior.
The Role of Hippocampal Replay in Memory Consolidation
The hippocampus lies at the centre of a network of brain regions thought to support spatial and episodic memory. Place cells - the principal cell of the hippocampus, represent information about an animal’s spatial location. Yet, during rest and awake quiescence place cells spontaneously recapitulate past trajectories (‘replay’). Replay has been hypothesised to support systems consolidation – the stabilisation of new memories via maturation of complementary cortical memory traces. Indeed, in recent work we found place and grid cells, from the deep medial entorhinal cortex (dMEC, the principal cortical output region of the hippocampus), replayed coherently during rest periods. Importantly, dMEC grid cells lagged place cells by ~11ms; suggesting the coordination may reflect consolidation. Moreover, preliminary data shows that the dMEC-hippocampal coordination strengthens as an animal becomes familiar with a task and that it may be led by directionally modulated cells. Finally, on-going work, in my recently established lab, shows replay may represent the mechanism underlying the maturation of episodic/spatial memory in pre-weanling pups. Together, these results indicate replay may play a central role in ensuring the permanency of memories.
Hippocampal replays appear after a single experience and slow down with subsequent experience as greater detail is incorporated
The hippocampus is implicated in memory formation, and neurons in the hippocampus take part in replay sequences, time-compressed reactivations of trajectories through space the animal has previously explored. These replay sequences have been proposed to be a form of memory for previously experienced places. I will present work exploring how these replays appear and change with experience. By recording from large ensembles of hippocampal neurons as rats explored novel and familiar linear tracks in various experiments, we found that hippocampal replays appear after a single experience and slow down with subsequent experience as greater detail is incorporated. We also investigated hover-and-jump dynamics within replays that are associated with the slow gamma (25-50Hz) oscillation in the LFP and found that replays slow down by adding more hover locations, corresponding to depiction of the behavioral trajectory with increased resolution. Thus, replays can reflect single experiences, and be rapidly modified by subsequent experience to incorporate more detail, consistent with their proposed role as a basic mechanism of hippocampally dependent memory.
Distributed replay in the human brain, and how to find it
I will present work on a novel fMRI analysis method that allows us to investigate sequential reactivation in the hippocampus. Our method focuses on analysing the time courses of probabilistic multivariate classifiers and allows us to infer the presence and frequency of fast sequential reactivation events. Using a paradigm in which we controlled the speed of sequential visually elicited activations, we validated the method in visual cortex for event sequences with only 32 ms between items. We show that detectability remains possible if low signal-to-noise ratio and when sequence events occur at unknown times. In a preliminary analysis, we show that even the exposure to our visual paradigm elicits reactivations in visual cortex at rest following the task. I then present work in which we tested how representations influence replay by asking whether transitions between task-state representations are reactivated at rest during hippocampal replay events. Participants learned to make decisions about ambiguous stimuli that depended on past events and attentionally filtered stimulus processing. FMRI signals during rest periods following this task indicated sequential reactivation of task states. These results indicate that adaptive task state representations are computed and replayed at different cortical sites. In combination with other methods, fMRI may allow us to unravel this coordinated nature of replay.
The anterior thalamus drives hippocampal replay following spatial learning
COSYNE 2022
Experience-Driven Rate Modulation is Reinstated During Hippocampal Replay
COSYNE 2022
An RNN model of planning explains hippocampal replay and human behavior
COSYNE 2023
Hippocampal replay events are impaired in a rat model of Fragile X Syndrome
FENS Forum 2024