In a recent experiment, Shamash et al. demonstrated that mice spontaneously adopt a subgoal strategy when escaping to a shelter hidden behind a wall. Their optimized trajectory, learned in 1-3 trials, involves running straight to a subgoal located at the edge of the wall, and then to the shelter. Strikingly, even if the wall is removed, animals consistently show curved escape trajectories, revealing that they continue to use the subgoal even though this involves a detour. This behavior can disappear depending on the details of the environment configuration during initial exploration, and can switch to straight trajectories towards the shelter in the absence of the wall. We show here that a general compositional model of learning captures all these behavioral variations, in contrast to traditional reinforcement learning algorithms [3]. The model relies on a classic pair of a motor planner (MP) which learns to generate motor primitives with a predictive representation (PR) which predicts the outcome of motor primitives. The novelty and power of our approach reside in its ability to learn how to achieve a goal, or minimize a cost function, with a sequence of motor primitives, connected via subgoals, as opposed to learning single-primitive plans. Zero-shot learning is made possible through rapid gradient descent, using a combination of off-line (i.e. internal simulations with the help of the PR) and on-line learning. When applied to the escape behavior, our model achieves zero-shot behavior through off-line learning converging on a two-primitive solution with a subgoal by the edge of the wall. As a result, the model exhibits curved trajectories even after the wall is removed. Crucially, the model can revert to a single-primitive solution depending on initial exploration, accounting for the full range of experimental manipulations. This work provides one of the first models of compositional learning in animals.