Successor representation

…that requires a precise relational representation of the spatial environment and its associated rewards. This process enables agents to choose optimal paths that maximize long-term rewards, which is essential for the survival of both humans and animals. While hippocampal place cells are traditionally linked to representing spatial location information, rodent studies have shown that they also exhibit predictive coding and reward-dependent place fields, suggesting their role extends beyond spatial representation. The Successor Representati

When rodents learn to navigate in a new environment, high place field density emerge at reward locations, fields elongate against the trajectory, and individual fields change spatial selectivity or drift while demonstrating stable behavior. Why place fields demonstrate these characteristic phenomena during learning remains elusive. We develop a model that optimizes for a normative goal, which is to maximize cumulative rewards. Place fields are modelled using Gaussian basis functions to represent spatial information of an environment, and directly synapse to an actor-critic for policy learning

…abstract variables are encoded by the representational geometry of single-neuron population activity in a format that supports generalization. However, these observations typically are made after learning has converged, leaving open the question of how these representations form. We designed the present study to tackle this question. We developed a factorized model of temporal abstraction that builds on the successor representation and is motivated by r

The successor representation has emerged as a powerful model for understanding the spatial coding properties of hippocampal place cells and entorhinal grid cells. However, the diverse, spatially-tuned cells of the primary output of hippocampal neurons - the subiculum - have eluded a unified account. Here, we demonstrate that incorporating rodent behavioural biases into the successor representation reproduces the heterogeneous activity patterns of subicular neurons. Notably, this framework accounts for boundary vector and corner cells; neuronal types that are absent from upstream hippocampal re

Although hippocampal place cells prominently "replay" nonlocal trajectories, the computational function of these events remains elusive. One hypothesis holds that replay plans routes to current goals by propagating information about rewards over space. An alternative view contends that replay learns and maintains routes more generally, separate from current goals or plans, to update a "cognitive map". The former view has been formalized in a reinforcement learning framework (Mattar and Daw, 2018), which predicts which trajectories should be replayed preferentially ("prioritized") to support pl

Navigating rodents show “forward sweeps” - sequential reactivation of non-local neural representations of future states starting from their current position. While direct evidence remains scant, it is widely thought that such forward sweeps support model-based (MB) evaluation of actions. However, a key challenge in testing this hypothesis is the absence of experimental paradigms that allow for precise measurement of MB action evaluation. Notably, typical assessments of MB evaluation, i.e. flexibility following changes to rewards (reward revaluation), conflate reliance on full MB evaluation wit

The predictive map hypothesis provides a promising framework to model representations in the hippocampal formation. I will introduce a tractable implementation of a predictive map called the successor representation (SR), before presenting data showing that rats and humans display SR-like navigational choices on a novel open-field maze. Next, I will show how such a predictive map could be implemented using spatial representations found in the hippocampal formation, before finally presenting how such learning might be well approximated by phenomena that exist in the spatial memory system - name

Mammals instinctively explore and form mental maps of their spatial environments. Models of cognitive mapping in neuroscience tend to depict map-learning as a process of random or biased diffusion; however, in practice, animals explore spaces using structured, purposeful, sensory-guided actions. A promising model system for probing the relationship between spontaneous exploration and spatial cognition is threat-evoked escape behavior in mice. Notably, Shamash et al. 2021 examined how mice learn routes to a shelter when the direct path is blocked by an obstacle. They found that during a 20-min

The successor representation (SR) is a promising candidate principle for hippocampal function. Theory proposes that each place cell encodes the expected state occupancy of its target location in the near future. This framework has desirable consequences on the generalisability and efficiency of reinforcement learning algorithms operating over these representations and is supported by behavioural and electrophysiological evidence. However, it is unclear how the SR might be learnt in the brain. Temporal difference learning, commonly used to learn SRs in artificial agents, is not known to be impl

…model based on the eigenvectors of the successor representation, human positional memory was degraded in a trapezoid compared to a square environment; an effect particularly pronounced in