Head direction (HD) cells track an animal’s head direction in darkness by integrating angular velocity signals, a phenomenon called path integration (PI). Ring attractor models for angular PI have received strong experimental support. To function as integrators, HD circuits require precisely tuned connectivity, which is costly to pass down genetically. This suggests that synaptic plasticity is crucial in setting up these circuits. We propose a network model in which a local, biologically plausible learning rule adjusts synaptic efficacies during development, guided by supervisory allothetic cues. The learning rule is inspired by layer-5 pyramidal neurons assumed to be the fundamental associative unit in the cortex, where backpropagating action potentials implement coincidence detection. The learning rule contains an anti-Hebbian component which performs predictive coding, whereby idiothetic inputs get associated with allothetic inputs that arrive at another compartment, so that the former can predict the latter. Applied to the Drosophila HD system, where such a segregation of inputs exists, the model learns to path-integrate accurately for the full range of angular velocities that the fly displays, and develops a connectivity strikingly similar to the one reported in experiments. The mature network is a quasi-continuous attractor (CAN), and reproduces key experiments in which optogenetic stimulation controls the internal representation of heading, and where the network quickly remaps to integrate with different gains akin to experiments conducted in augmented reality in rodents. Our model proposes a general framework to learn gain-1 PI, even in architectures that lack the physical topography of a ring, like the HD system in mammals. Finally, we develop a tractable reduced model that exploits circular symmetries present in the full network, explains how the latter solves credit assignment, and offers a rigorous mathematical framework to study the self-organization of CANs for angular PI in general.