There has been a long-lasting debate on whether the function of the hippocampal formation is to store and retrieve episodic memory, supported by experiments in humans, or to code for space, suggested by spatially tuned cells (place cells, grid cells, etc.) recorded in nonhuman animals. In an attempt to reconcile the two views, we use a memory-augmented neural network in a reinforcement learning setup to solve a navigation task in a simulated maze. The key feature of the model is that it learns autonomously what and when to store into, and retrieve from, an external memory buffer. Once the model successfully solves the spatial task, we find that, although the model receives only camera images as external inputs, the information encoded in memory at the population level reflects the 2D structure of the maze. Even more surprising is that the process of memory retrieval performs a coordinate transformation of the goal location. We find that the computations and memory representations in the model can be explained why a simple geometric theory. In addition, we also find that individual units in the network are spatially tuned, and their firing fields are consistent with the spatial representations of the population encoding. It appears that it is the spatial nature of the simulated task that moulds the emergence of the spatial representations found in the model. When learning to solve tasks in a different domain, the model would eventually develop different kinds of representations and computations, tailored for the specific task and the domain. In other words, coding for space might just be one use case of the hippocampal formation, which performs a much broader cognitive function, storing and retrieving memories.