Neural circuit computations are modulated by combinations of internal and external factors. For example, neurons in the medial entorhinal cortex (MEC) change their firing patterns or “remap” during a reward-seeking task, responding to changing task conditions and environmental cues. Why does this remapping occur? What unifying principles (if any) underlie remapping? Recent work proposes that remapping reflects hidden state inference—changes to an animal’s internal beliefs about the world. Recent experimental evidence suggests that remapping can indeed arise from latent behavioral or cognitive state changes without changes to the environmental or task cues. There remains, however, a critical gap between our theoretical and biological perspectives on neural remapping. Sanders et al. frame remapping as hidden state inference, but do not propose a circuit mechanism to implement this inference. Low et al. used Neuropixels recordings of MEC neural ensembles to characterize remapping as transitions between geometrically aligned neural activity manifolds, but they do not attribute remapping to any functional purpose, such as hidden state inference. Here we leverage recurrent neural network (RNN) models to bridge this gap. We find that, when RNNs are trained to simultaneously track a hidden state (“context cue”) and perform a path integration task, the RNN dynamics converge to a similar solution as MEC neurons. Namely, the number of hidden states reflects the number of neural manifolds (spatial maps of the environment) in the RNN and, critically, these manifolds are geometrically aligned in the precise manner described by Low et al. This geometric alignment allows for remapping along an orthogonal dimension to ongoing circuit computations, supporting flexible hidden state inference alongside reliable spatial coding in a single circuit. We thus propose a normative model for neural remapping with a biologically plausible implementation, reconciling prior theoretical and experimental work and generating hypotheses for future studies in both fields.