Local Deep Learning without Gradients in Asymmetric Recurrent Networks
We introduce a statistical physics framework for learning in neural architectures composed of single or interconnected asymmetric attractor networks. These systems can exhibit a manifold of global fixed points capable of implementing sophisticated input-output mappings, which we characterize analytically. Learning from extensive datasets is achieved through the stabilization of fixed points via a fully distributed and local learning process, implemented at the single-neuron level. This simple mechanism yields performance comparable to that of conventional feedforward deep neural networks trained using gradient-based methods. The effectiveness of the model stems from the dense and accessible manifolds of stable fixed points, which encode the internal representations of data. Unlike other approaches to deep learning without backpropagation, our method does not attempt to estimate gradients. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2025-06-18. Recording duration: 00:45:35.
Generalizing theories of cerebellum-like learning
Since the theories of Marr, Ito, and Albus, the cerebellum has provided an attractive well-characterized model system to investigate biological mechanisms of learning. In recent years, theories have been developed that provide a normative account for many features of the anatomy and function of cerebellar cortex and cerebellum-like systems, including the distribution of parallel fiber-Purkinje cell synaptic weights, the expansion in neuron number of the granule cell layer and their synaptic in-degree, and sparse coding by granule cells. Typically, these theories focus on the learning of random mappings between uncorrelated inputs and binary outputs, an assumption that may be reasonable for certain forms of associative conditioning but is also quite far from accounting for the important role the cerebellum plays in the control of smooth movements. I will discuss in-progress work with Marjorie Xie, Samuel Muscinelli, and Kameron Decker Harris generalizing these learning theories to correlated inputs and general classes of smooth input-output mappings. Our studies build on earlier work in theoretical neuroscience as well as recent advances in the kernel theory of wide neural networks. They illuminate the role of pre-expansion structures in processing input stimuli and the significance of sparse granule cell activity. If there is time, I will also discuss preliminary work with Jack Lindsey extending these theories beyond cerebellum-like structures to recurrent networks.