Phase remembers: trained RNNs develop phase-locked limit cycles in a working memory task

Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or ‘frame of reference’ relative to which information can be encoded. In line with this hypothesis, there have been many empirical observations of this phase code in the brain. What are the latent dynamics and circuits that support phase coding with neural oscillations? Here, we propose a new computational hypothesis which is derived from analyzing trained recurrent neural networks (RNNs). We train RNNs on a working memory task, while also giving them access to a reference oscillation (either a pure sine wave or rat CA1 local field potentials). The task is to produce an oscillation such that its phase maintains the identity of transient stimuli. Although this task could be solved with static attractors, we find networks converging to oscillatory dynamics that persist in the absence of oscillatory input. Reverse engineering these trained network reveals bistability: in particular, each phase-coded memory corresponds to a separate limit cycle attractor in a toroidal manifold. We characterise the nonlinear dependence of the stability of the attractor dynamics on reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the computations underlying stable phase-coding, we show that trained networks converge to dynamics that can be described as two phase coupled oscillators. Using this insight, we condense our trained networks to a reduced model consisting of two functional modules: one that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. We show how incoming stimuli transiently modify this coupling function. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a novel mechanism by which neural networks can harness reference oscillations for working memory.