What circuit mechanisms underlie the encoding and retrieval of spatial memory? In standard models, neurons have discrete fields that map onto locations, which are associated with actions [1]. Given that spatial-memory regions (such as hippocampus and entorhinal cortex) have recurrent architectures, it is unclear if this feedforward association mechanism is the only way in which spatial memories can be stored. Here, we propose a new mechanism for spatial memory that is derived from reverse-engineering recurrent neural networks (RNNs). We train RNNs on a spatial navigation task posed as a reinforcement learning problem. The agent receives noisy velocity inputs and can choose to stop or move forwards. The agent navigates across a linear track and its goal is to run to a hidden reward location and then move to the end of the track to initiate a new trial. This task is adapted from a path integration task that we previously trained mice to solve [2]. Reverse engineering trained RNNs reveals that their single neuron activity is similar to that recorded from the entorhinal cortex of mice performing the same task [3], and during vector navigation in primates [4]. We find that the activity of the RNN is low-dimensional, and that a circular manifold representing track position emerges with learning. Using interpretability tools from the machine learning world, we discover that a small population of neurons is important for conveying velocity information through the RNN. Interestingly we find that traditional neuroscience methods did not identify these mechanistically important neurons. To understand the computations underlying memory of multiple reward locations, we train the agent to navigate to a context-dependent reward location. We find that the same manifold was reused across contexts, with memory implemented by changing the gain at which velocity pushed the neural trajectory to the manifold’s reward zone. We identified a structure in the input weights that underlies this mechanism, and were able to transplant spatial memories between contexts. Thus, rather than storing context-specific information by associating locations with outcomes, neural circuits can instead learn context-specific spatial dynamics. Applied more generally, our results suggest a novel framework for understanding semantic memory, for example learning of concepts, in which specific instances of a concept are associated with specific dynamics of a general low dimensional ‘concept model’