Rapid approximation of successor representations with STDP and theta phase precession

The successor representation (SR) is a promising candidate principle for hippocampal function. Theory proposes that each place cell encodes the expected state occupancy of its target location in the near future. This framework has desirable consequences on the generalisability and efficiency of reinforcement learning algorithms operating over these representations and is supported by behavioural and electrophysiological evidence. However, it is unclear how the SR might be learnt in the brain. Temporal difference learning, commonly used to learn SRs in artificial agents, is not known to be implemented in hippocampal networks. Instead, we demonstrate that spike-timing dependent plasticity (STDP), a modified form of Hebbian learning, acting on temporally compressed trajectories known as “theta sweeps”, is sufficient to rapidly learn a useful approximation to the SR. Our model is biologically plausible: it maps onto validated aspects of hippocampal circuitry, it uses spiking neurons modulated by theta-band oscillations, diffuse and overlapping place cell-like states and experimentally matched parameters. It explains substantial variance in the true successor matrix and gives rise to place cells demonstrating key experimentally observed phenomena associated with the SR including policy-dependent backwards expansion on a 1D track and elongation near walls in 2D. In our model, larger place cells encode longer timescale SRs. We shed insight on the observed topographical ordering of place cell size down the dorsal-ventral axis by showing this is necessary to prevent the detrimental mixing of these timescales.