Topic: deep networks

Deep networks display recurring phenomena such as grokking, scaling behaviour, generalization despite overparameterization and failures outside training distributions. This talk surveys controlled settings where those behaviours can be explained, suggesting that language models may combine many tractable, largely interpolative mechanisms. It then distinguishes that scientific account from a more speculative argument about the role of industrial computation and human labour, and what these imply for claims of broadly extrapolative intelligence.

This talk explores how combining neural data from multiple brains with deep networks could benefit both neuroscience and AI. It presents approaches to mapping information across brains without shared observations and considers how collective neural information might guide new discoveries. The same perspective motivates alternative ways to train and interpret artificial networks.

Tomaso Poggio discusses principles that could explain learning and intelligence in brains and machines. Topics include the relationship between engineering and theory, levels of explanation, the role of learning, and compositional sparsity and genericity as potential explanations for the effectiveness of deep networks.