A principle of modern decision theories is reference dependence, in which the subjective value of a reward is computed relative to an internal reference point that is thought to reflect expectations. Previous work often assumes that reference points are retrospective, deriving directly from past experiences. We used behavioral modeling to show that, instead, rats infer underlying task states in order to switch between multiple reference points. We designed a behavioral paradigm with partially observable states – uncued blocks of trials offering only small rewards (low blocks) or large rewards (high blocks). Rats (n=165) must determine how long to wait for rewards, providing an explicit behavioral readout of a rat’s subjective value of the reward. Rats’ wait times are sensitive to the block structure - rats wait longer for the same volume of water in low blocks compared to high blocks. We developed computational models that instantiate retrospective or inferential strategies for computing the reference point, which corresponds to the opportunity cost of time in our task. Model comparison showed that an inferential reference point better describes the rats’ behavior, and this model strongly outperformed the retrospective model around block transitions. However, we found that rats exhibited considerable heterogeneity in the quality of their prior over the blocks, where the prior has knowledge of the task structure and transition probabilities. Rats with poorer quality priors were slower to infer switches into low blocks, but not high blocks - a form of “optimism” wherein the rats’ priors are biased toward high blocks. These data show that rats perform inference to switch between multiple reference points but vary in their ability to incorporate optimistic versus realistic beliefs about task structure.