When faced with a constantly changing, uncertain environment it is necessary to infer its underlying structure to guide behaviour. This has often been formalised as a value updating problem – where actions are chosen based on an incrementally updated weighted average of past reward. Activity of midbrain dopamine neurons has been commonly associated with the error in such value prediction, and it has been proposed to be crucial for learning and updating of values that inform decision making. However, it has become increasingly apparent that both humans and animals often use an alternative strategy – based on the use of inferred hidden states to make predictions and to guide their choices. In this study we hypothesised that, as is commonly seen in humans, mice might use hidden state strategies even in simple tasks such as reversal learning, and that midbrain dopamine activity would reflect the use of such strategies. To investigate this, we used a probabilistic reversal learning task in mice. In this paradigm, for optimal performance it is necessary to continuously integrate past trial outcomes to predict reward contingencies associated with different actions across reversals. Probing animals’ behaviour with computational modelling, we found that mouse behaviour was consistently best fit by models that utilised hidden states, as opposed to value updating or alternate strategies. Furthermore, by recording dopamine release in the nucleus accumbens during the task using the dopamine sensor dLight, we found phasic dopamine was most strongly predicted by error associated with hidden state inference, rather than error in reward prediction. Overall, we find that mouse behaviour and midbrain dopamine activity during probabilistic reversal learning task is best described by a belief state rather than value updating strategy. Ongoing work is investigating the sources of the belief state prediction that influence dopamine signalling during decision making.