The remarkable ability of neural networks to make predictions about the immediate future and recognize unexpected events is a ubiquitous hallmark of intelligent life. In recent years, researchers have begun to unravel the neural substrates underlying this predictive information processing [1]. An integral part of predictive processing is a subset of excitatory neurons that encode prediction errors: while negative prediction-error (nPE) neurons are only activated when sensory signals are weaker than predicted, positive prediction-error (pPE) neurons respond only when sensory signals exceed the internal predictions [1-3]. How these different types of prediction-error neurons can co-exist and simultaneously form in the same recurrent neural circuit they are embedded in, and how they are shaped by the rich diversity of cell types [4] is still largely unresolved. To unravel the circuit-level mechanisms that underlie the parallel formation and refinement of nPE and pPE neurons, we make use of a computational model of a cortical circuit with excitatory pyramidal cells and three types of inhibitory interneurons: parvalbumin-expressing, somatostatin-expressing, and vasoactive intestinal peptide-expressing interneurons [4]. By means of a mathematically tractable model and network simulations, we show that the presence of nPE and pPE neurons requires balanced pathways that can be learned simultaneously with homeostatic inhibitory plasticity with low baseline firing rates. The resulting robust PE circuits generalize to sensory stimuli not seen during learning. Furthermore, we show that the responses to unexpected events are mainly determined by the network’s initial connectivity and the distribution of actual and predicted sensory inputs onto the interneurons. Finally, we demonstrate that PE neurons can support biased perception and may underly faster learning as well as generalization across stimuli statistics. In summary, our results shed light on the formation, refinement, robustness, and computational role of PE circuits.