Invertible readouts to improve the dynamical accuracy of neural population models

An emerging perspective in neuroscience connects neural dynamics, or the rules that govern how neural population activity develops over time, to functional computations performed by the brain [1]. Unfortunately, neural dynamics cannot be observed directly and must instead be inferred from neural recordings. One promising method to infer these dynamics is to use a latent dynamics model, commonly a Sequential Auto-Encoder (SAE), to reconstruct neural activity [2] with the assumption that better reconstruction performance corresponds to better dynamical accuracy. Unfortunately, most SAE readouts, the component of an SAE that maps from the latent activity to predicted neural activity, have critical shortcomings. First, they are usually simple linear models and therefore can’t capture potential nonlinearities between the latent and neural spaces. Second, they have a null-space, or a set of latent dimensions that don’t affect neural reconstruction. This null-space permits models to learn “superfluous” dynamics that improve reconstruction performance at the expense of dynamical accuracy. Our model, called Ordinary Differential equations auto-encoder with INvertible readout (ODIN), combines Neural Ordinary Differential Equation (NODE) [3] and Invertible Neural Networks (INNs) [4] into a new SAE variant with a trainable non-linear readout without a null-space. To validate ODIN’s dynamical accuracy, we simulated neuronal population activity from a low-D dynamical system embedded non-linearly into high-D neural space. We then trained ODIN and alternative models on these simulated data and quantified their dynamical accuracy. We found that ODIN’s dynamics were more accurate than comparable models with MLP or linear readouts (mean Hidden R2 of 0.93 vs. 0.80, 0.84, respectively), and that ODIN’s dynamical accuracy was more robust to changes in relevant hyperparameters. In summary, ODIN is a promising extension to existing SAE models that will improve our understanding of neural computation by enabling more accurate estimation of neural dynamics.