Neural replay, a phenomenon where activation sequences experienced during wakefulness are replayed during rest or sleep, is thought to play a crucial role in memory consolidation and planning [1]. It is commonly modeled as a chain of activity, where links are formed based on temporal associations in the stimulus. Existing work [2,3] focuses on simple patterns, yet we know that the brain is capable of learning complex spatiotemporal dynamics. Specifically chaotic dynamics are even challenging for conventional deep learning algorithms due to exploding gradients. We demonstrate that by introducing fixed transmission delays, a recurrent neural circuit with differential Hebbian learning can learn and replay the Lorenz system. The heterogeneous delays provide an embedding of incoming inputs and effectively expand the receptive fields in the temporal dimension. Each rate-based node of the network is assigned a position on the attractor manifold and stimulated during learning if the Lorenz trajectory passes it. Using firing rate adaptation and lateral inhibition, the model can indefinitely replay the dynamics without external input. This study highlights the potential functional role of inherent delays in neural systems, and offers directions for local on-chip learning in neuromorphic systems.