Inverse Rational Control (IRC) is a new theoretical framework to infer subjects' beliefs about the world from their actions. Here we apply IRC for the first time to experimental data and use it to interpret monkey behavior in a spatial navigation task. In this task, monkeys use a joystick to navigate to a transiently visible target (a 'firefly') using optic flow cues. In general, we define beliefs to be summary statistics for posteriors over partially observable latent world states. For our task, these beliefs represent the subject's estimated current position and uncertainty, assumed observation noise level, relative utility of rewards and actions, and several other task-relevant properties. Previous methods of inferring beliefs are often built upon the assumption that subjects are optimal, so their predictions will be falsified if the subject is sub-optimal. Instead of assuming the subject is optimal, we hypothesize that there could be a mismatch between the subjects' assumed world model and the true world, but that subjects behave optimally under these mistaken assumptions, which we call sub-optimal but `rational'. We modeled rational behavior by a policy obtained by reinforcement learning based on a cognitive model of latent dynamics. We parameterize this model family by its hidden assumptions and subjective preferences, and infer these parameters from the animal's observable responses to its sensory observations. Our fitted model successfully captures the animal behavior on our task. Analysis of the best fitting model reveals how the monkey alters its weight on sensory observations when controls behave unexpectedly, and shows that the monkey values straight-ahead targets more than the targets on the side. These inferred beliefs serve as novel targets for studying neural representations. Our results demonstrate the utility of our approach for rich characterization of latent dynamics underlying behavior, and will thereby help advance the neuroscience community's understanding of neural computation involved in perception, prediction, and planning.