47 min
Seminar, Hosted by Simons Institute for the Theory of Computing
Capacity of Neural Networks for Lifelong Learning of Composable Tasks
Harvard University
Recording
Abstract
Les Valiant asks how many associations a neural network can acquire in a single exposure each, retain over time, and compose later into chains. The model requires each association to activate its target while limiting unintended activity elsewhere. Bounds for random networks compare direct connections with a scheme that uses strong intermediate relays, which can substantially increase capacity and make it superlinear in network size.