The ability to continually adapt to the environment is key for survival. How the brain learns continuously while retaining previous knowledge is not known. Using a combination of neural and behavioural data analysis together with deep learning modelling, we studied the role of the hippocampus in continual spatial reinforcement learning. First, we introduce a deep recurrent Q-learning agent consistent with the hippocampal architecture (hcDRQN) along with three control models: a non-recurrent model (hcDQN) and two standard machine continual learning algorithms. We trained all these reinforcement