In recurrent circuits, neurons and synapses are coupled in an intricate dance: neurons influence synapses through activity-dependent plasticity, and synapses influence neurons by shaping network dynamics. While previous studies have analyzed recurrent circuits with static synapses, or synapses displaying short-term facilitation and depression, the impact of ongoing Hebbian plasticity on network behavior is not well elucidated. Such an understanding is required to map the full landscape of neural-circuit dynamics and to probe computational roles of dynamic synapses. To address this knowledge gap, we developed a dynamical mean-field theory for a model of coupled neuronal-synaptic dynamics. In this model, neuronal rates follow recurrent dynamics shaped by time-dependent synaptic weights that are modulated, in turn, by pre- and postsynaptic neuronal rates. We assumed that plasticity modulates each synapse about a random baseline strength. We show that neuronal-synaptic dynamics are much richer than neuronal dynamics alone. Hebbian plasticity generates slow chaos, while anti-Hebbian plasticity generates fast chaos with an oscillatory component. Studying the spectrum of the joint neuronal-synaptic Jacobian revealed that these behaviors manifest as differential effects of eigenvalue repulsion. Hebbian plasticity can generate chaos in a circuit that, without plasticity, would be quiescent. When Hebbian plasticity is sufficiently strong, a chaotic state coexists with stable nonzero fixed points. Finally, in the chaotic regime, halting plasticity can leave a stable fixed point of the neuronal dynamics, freezing the chaotic state. This phase of freezable chaos provides a natural mechanism for keeping a running copy of the instantaneous neuronal state and could shed light on features of general anesthesia. Overall, our work presents a theoretical framework for studying circuits in which synapses are dynamical variables on equal footing with neurons and elucidates several surprising dynamical characteristics of such circuits.