Just as humans often follow the instructions of a tutor to learn new skills, juvenile male zebra finches learn to sing by copying the song of an adult tutor, providing a powerful system for analyzing tutor-driven learning from both behavioral and neurobiological perspectives. Decades of work have characterized the general trajectory of this learning process [1,2], but recent work using generative modeling methods like Variational Autoencoders (VAEs) has opened entirely new avenues for analyzing song learning [3,4]. While it is generally agreed that juveniles learn to sing by using auditory feedback to evaluate their own songs in reference to a memorized model of the tutor song [5,6], the actual dynamics of juvenile song copying remain unexplained. Specifically, it is unclear how juvenile zebra finches direct their vocal exploration on a rendition-by-rendition basis toward the tutor song. We characterized changes in vocal exploration across development, examining differences between renditions. We used VAEs to create low-dimensional representations of birdsong over the course of vocal learning—from just after tutor exposure to song crystallization. We modeled vocalizations as trajectories within these low-dimensional representations using Langevin dynamics with a learned drift term and temporally autocorrelated noise. The model recapitulated prior findings in juvenile song development [1] and demonstrated that the balance between stochastic vocal exploration and exploitation of learned vocalizations changes based on both developmental age and millisecond-level variation in vocalization. We found that pupil song becomes more deterministic as syllables consolidate closer to the tutor song (as measured by distance in latent space). Moreover, even at a young age, vocalizations that more closely resemble the final, crystallized song are less variable than those farther away. This model of directed exploration thus makes possible future experiments in which neural activity can be related to local measures of tutor-referenced maturity in developing birdsong.