Performing goal-directed behaviors in an uncertain environment requires organisms to continuously evaluate the sensory consequences of their motor actions and fine-tune them to ultimately reach the goal. This evaluation involves a comparison between the predicted and actual sensory change following a movement and implementing corrective movements if there is a mismatch between the two. However, details about the neural implementation of this comparison and correction are unclear. We study this paradigm in larval zebrafish using a virtual reality setup. We first “train” the fish such that they establish a behavioral policy and then introduce a reversal in the visual feedback, such that the fish needs to reverse its policy in order to achieve the goal. We show that larvae can indeed perform this policy reversal. Taking advantage of the comprehensive imaging capabilities offered by this model organism, we perform whole brain calcium imaging while larval zebrafish undergo this assay. Our results reveal that several regions show differential activations in response to this altered feedback., including the cerebellum and inferior olive. Overall, this study shows how altered visual feedback can modulate behavior and the neural correlates associated with these dynamic changes provide valuable insights into the underlying processes of sensorimotor integration in the brain.