The Neural Race Reduction: Dynamics of nonlinear representation learning in deep architectures
Prof.
UCL
Event Information
Host
NYU Swartz
Duration
70 minutes
Abstract
What is the relationship between task, network architecture, and population activity in nonlinear deep networks? I will describe the Gated Deep Linear Network framework, which schematizes how pathways of information flow impact learning dynamics within an architecture. Because of the gating, these networks can compute nonlinear functions of their input. We derive an exact reduction and, for certain cases, exact solutions to the dynamics of learning. The reduction takes the form of a neural race with an implicit bias towards shared representations, which then govern the model’s ability to systematically generalize, multi-task, and transfer. We show how appropriate network architectures can help factorize and abstract knowledge. Together, these results begin to shed light on the links between architecture, learning dynamics and network performance.
Topics
Related Job Opportunities
PhD Studentship: Mitochondrial Metabolism and Novel Therapeutic Strategies for Metabolic Dysfunction-Associated Steatotic Liver Disease (MASLD) (Fixed Term)
Supervisors: Professor Andrew Murray, Department of Physiology, Development and Neuroscience, University of Cambridge Dr Ross Lindsay, Novo Nordisk Funding: Fully funded PhD studentship (Home/UK…
Research Associate (Fixed Term)
We seek a highly motivated Postdoctoral Research Associate to join the laboratory of Professor Kathy Niakan. We are based in the Loke Centre for Trophoblast Research (LCTR), in the Department of…
Research Assistant/Associate (Fixed Term)
Applications are invited for a postdoctoral research associate position to study the neural mechanisms of visual learning in mice, in the laboratories of Professor Ole Paulsen and Dr Jasper Poort…