The ease with which two-legged dogs can run illustrates the versatility and agility of mammals in learning arbitrary motor skills, an ability currently unrivaled by artificial systems. How the brain supports these capacities is unclear, as control mechanisms are distributed and interdependent between many brain regions. We make progress on this question by developing a neural network model comprising three major brain systems that are known to play central roles in learned motor control: the cerebellum (CB), primary motor cortex (M1), and associative sensorimotor cortex (ASC). We both leverage and modify advanced machine learning techniques to equip our model with empirically grounded computational hypotheses about how the different regions learn. The model incorporates challenging constraints that characterize real-world motor tasks: continuous space and time, no supervised motor trajectory information, and noisy, delayed, and occluded sensory input. We posit that M1 learns through a process of exploring and reinforcing control strategies that accomplish a final goal (implemented via policy gradient), while the ASC-CB system learns through a self-supervised process by making moment-by-moment sensory predictions. We find that the network learns realistic and robust motor strategies such as active-sensing through overreaching – observed in humans during occluded reaches – as well as rapid adaptive compensation to curl-field perturbation when only delayed and noisy feedback is available. The three brain systems synergize, especially in environments with very sparse observations: M1 helps ASC and CB learn by driving active sensing, M1 inhibits CB control outputs while sensory-prediction-based control is immature, and – once a good internal predictive model is learned – CB and ASC complement M1 control. Finally, we show that the model’s activity and behavior are consistent with a broad range of electrophysiological and lesion data and make several novel predictions.