Abstraction and generalization are essential for flexible decision-making in novel situations. Recent work in humans and monkeys has shown how abstract variables are encoded by the representational geometry of single-neuron population activity in a format that supports generalization. However, these observations typically are made after learning has converged, leaving open the question of how these representations form. We designed the present study to tackle this question. We developed a factorized model of temporal abstraction that builds on the successor representation and is motivated by recent proposals for how hippocampus interacts with surrounding cortical regions to learn structure. The model learns and represents the relationships between behavioral states in a format analogous to the neural geometry observed previously. Critically, it disentangles the contributions of different levels of abstract learning---from specific stimulus-response associations to generalizable task structure---in the form of a factorized prediction error. These trial-wise factors predict the change in representational geometry on each trial, which can then be used to probe the dynamics of the neural geometry. We fit the model to the behavior of human participants performing a context-dependent decision task during fMRI. The model captured the learning dynamics at multiple timescales, including the increasing contribution of generalization as participants transferred structural knowledge between novel instances of the task. In fMRI, the generalization component correlated selectively with BOLD activity in orbitofrontal cortex and hippocampus. This result aligned with previous single-neuron work showing hippocampus represented latent context in a generalizable format after humans and monkeys had learned a similar task. Moreover, as the behavioral evidence for generalization increased, so did its neural correlate in parahippocampal gyrus and hippocampus---a proposed source and recipient, respectively, of generalized structural knowledge. The present study offers a computational framework for discovering how representational geometry forms and changes under continual abstract learning.