Decision-making occurring​ during navigation and learning is widely studied in choice behaviors, but less understood in natural and more continuous settings, especially under stress and uncertainty. This process could be investigated in rodent spatial navigation, which has been modeled with place-cell-based models. However, traditional models usually ignore detailed trajectories or kinematics. Here we extended a place cell-based reinforcement learning model to include detailed kinematics and used it to investigate the role of motivational stress. We performed experiments with two strains of mice learning two versions of the Morris Water Maze task under different water temperatures: the task with a fixed platform and the task where platform location varied randomly between two positions. Using computational modeling and parameter estimation, we were able to reproduce detailed mouse behaviors and reveal computational correlates of temperature-based behavioral differences. We then extended the model to include a wall-distance-based component, where learning is guided not just by place information but also by distance to the wall (cue-like signal), which reproduced mouse behavior in tasks with uncertain platform locations better than place-cell-based strategies alone. We further implemented a more biologically plausible model, combining border (boundary) cells and place cells. We finally compared model performance with place cells only, place and distance cells, and place and border cells under different experimental conditions and animal strains and performed parameter estimation to find the best-fitting model settings and parameters for each animal. Our findings provide insights into computational mechanisms underlying spatial navigation in mice and how various modulators influence it.