Big Boxes, Not Black Boxes: What we can compute about LLMs, and what it may say about AGI

Deep networks are often thought of as black boxes. Their ability to encompass vast swathes of knowledge indeed makes them hard to explain. Yet many of their behaviours — generalization under overparametrization, grokking, OOD failures, neural scaling laws — recur across architectures and scales, and each, however surprising, can be reproduced and explained in controlled settings. I will review these efforts to identify and explain the universal phenomena of deep learning, and suggest that an LLM may amount to a sum of such tractable sub-phenomena, interpolative in nature. Finally, leaving scientific rigor aside, I'll argue that what separates this prosaic picture from the apparent magic of LLMs may well be the industrial
scale of compute and human labour behind it, and that AGI in its deeper extrapolative sense may be much further away than claimed.