NOISE CORRELATION ON A LOW-DIMENSIONAL NEURAL MANIFOLD ENABLES EFFECTIVE LEARNING AT SCALE

Learning in neural systems requires effective credit assignment: determining how changes in individual neurons or synapses influence behavioral or network outputs. Noise correlation–based learning rules provide a biologically plausible approach by estimating gradients through correlations between neural perturbations and output changes. However, standard implementations scale poorly, as accurate gradient estimation requires that the number of perturbations grow with network size. In addition, the use of isotropic noise is inconsistent with experimental evidence that population activity in biological neural circuits is constrained to a low-dimensional manifold.
Here we introduce neural manifold noise correlation (NMNC), a credit-assignment framework that restricts perturbations to the intrinsic neural manifold. We show, both theoretically and empirically, that in trained networks the row space of the network Jacobian aligns closely with this manifold, and that manifold dimensionality increases only slowly with network size. These properties allow NMNC to estimate gradients efficiently without requiring high-dimensional perturbations.
Across a range of architectures and tasks, including convolutional networks trained on CIFAR-10 and ImageNet as well as recurrent neural networks, NMNC substantially improves learning performance and sample efficiency compared to conventional noise correlation-based methods. Furthermore, networks trained with NMNC develop internal representations that more closely resemble those observed in the primate visual system than those trained with conventional noise correlation-based methods.
Together, these results suggest a mechanistic account of how biological circuits could perform scalable credit assignment using structured neural variability, and demonstrate that biologically motivated constraints on neural activity may facilitate, rather than hinder, efficient learning at scale.