Uncovering relationships between neural network activation changes and parameter dynamics during learning

Learning in neural circuits manifests via slow changes in activity driving behaviour changes. Uncovering how the coordination of plasticity mechanisms found in the brain leads to such changes is a difficult problem, chiefly because we do not have access to the synaptic parameters underlying neural computation. Nevertheless, as neural population recording techniques improve and experiments track changes in neural dynamics over long durations, finer details about plastic changes could be inferred by observing and characterizing changes in neural activity. In this work, we present a step towards this goal and describe an effort to extract a data-driven mapping between neural activity space and neural network parameter space during learning. We present in silico experiments with recurrent neural networks (RNNs) trained for single and multi-objective tasks and explore the relationship between the dynamics of network parameters and activity over the course of training. We consider two tasks: (i) classification of one or both digits from an image containing two digits; and (ii) a center-out cursor control task with cursor position and size targets. Using simple classifiers and linear dimensionality reduction tools, we show that: (1) changes in the neural activations are representative of the corresponding changes in parameters over the course of training; and (2) it is possible to reliably distinguish single and multi-objective task settings based on activation representations during learning. Our experiments pave the way for more latent space design that could reveal key features of connectivity plasticity dynamics from those of neural activities recorded during learning. We posit that similar techniques can be used in longitudinal systems neuroscience experiments and in experiments using brain-computer interfaces which promote and require targeted learning.