Big Boxes, Not Black Boxes: What we can compute about LLMs, and what it may say about AGI

Deep networks display recurring phenomena such as grokking, scaling behaviour, generalization despite overparameterization and failures outside training distributions. This talk surveys controlled settings where those behaviours can be explained, suggesting that language models may combine many tractable, largely interpolative mechanisms. It then distinguishes that scientific account from a more speculative argument about the role of industrial computation and human labour, and what these imply for claims of broadly extrapolative intelligence.