A key component of cognition is set-shifting, which requires animals to update their knowledge of current rules or context, allowing flexible behaviour in a changing environment. This updating of rules or context relies on constantly evaluating and detecting mismatches between expectation and observed outcomes of events. The anterior cingulate cortex (ACC) has been implicated in set-shifting behaviour, consistent with its well-established role in processing conflicts during cognitive tasks in primates and humans. However, the neural circuit mechanisms underlying set- shifting are largely unknown. Here we trained mice to perform an attentional set-shifting task in which they alternated between blocks of distinct task rules. Mice typically required a single experience of an expectation violation to accurately adapt their behaviour to respond to the same stimuli using different rules. The behaviour was well-fit to a reinforcement learning (RL) model incorporating context belief states, but not to a basic RL model. Optogenetic inhibition of the anterior cingulate cortex significantly impaired behavioural switching but had no effect on performance of the task once a behavioural switch was achieved. Inhibition of prelimbic cortex (PL) had no such effect. Chronic in vivo two-photon calcium imaging during the task identified prediction-mismatch tuned cells in the ACC but not in V1. These cells maximally fired when a stimulus was expected but not received, and these responses were inhibited when the expected stimulus was received. Crucially, the magnitude of the mismatch responses in the ACC could predict successful behavioural transitions in the subsequent trial. These results suggest an essential role for the ACC in driving rapid behavioural changes in response to changing context using prediction mismatch signals.