Goal-directed behavior in dynamic environments requires animals to predict relevant changes, such as the occurrence of reinforcement, from the presence of sensory cues. This has been formalized in the prediction error theory, where the difference between the predicted and the received reinforcement serves as a driving force in the learning process to optimize future reinforcement prediction and to enable anticipatory approach or avoidance behavior. In the fruit fly brain, dopaminergic neurons in the mushroom body, a central structure for learning and memory, have been suggested to compute such prediction errors. Using mechanistic computational models, we explore the underlying circuit motifs that can dynamically alter dopaminergic neuron activity through various feed-forward and feedback pathways within the mushroom body. We demonstrate that this directly affects learning and explore the scope of learning phenomena it can explain, such as saturating learning curves, extinction, or higher-order learning. Using simulations of behavior, we show that the learned reinforcement prediction directly reflects the temporal dynamics observed in animal learning experiments.