Making optimal decisions in a noisy and ever-changing world is difficult. To do this, we must continuously track the outcomes of our actions and the uncertainty associated with them over both long and short timescales. This allows identification of the underlying regularities in the world that can be used to guide behavior. But how neural circuits support the integration of experience is poorly understood. Neuromodulators are ideal candidates for tracking such variables as they can signal over multiple timescales from milliseconds to seconds. Specifically, acetylcholine has frequently been proposed to be key for the calculation of uncertainty, but the mechanism of its action is unknown.To test this, we trained mice in a probabilistic reversal learning task where mice had to integrate past outcomes over tens of seconds across different experimentally controlled uncertainties. We found that performance in this task was dramatically impaired by pharmacological inhibition of cholinergic signaling. Regression analysis showed this was due to reducing the influence of outcome history on upcoming choice. To compliment this finding, we used a genetically encoded acetylcholine sensor to record acetylcholine release into a key area for flexible learning; the ventral hippocampus. Consistent with our pharmacological findings, hippocampal acetylcholine levels around choice were scaled by outcome history. Together this suggests a key role for hippocampal acetylcholine in shaping flexible behaviour. Ongoing work is utilising reinforcement learning agents with different strategies to track uncertainty, to more quantitatively investigate the role of acetylcholine signalling in reward integration and uncertainty estimation.