Why All Models Learn the Same Thing with Phillip Isola (MIT)

MIT professor Phillip Isola discusses what makes learned representations useful and why independently trained models can develop similar internal structure. He explores the platonic representation hypothesis, local clustering versus global geometry, and the neural thickets account of why pretrained networks can adapt readily to downstream tasks. Other topics include language models as world models, recurrent architectures, biological comparisons, and studying autonomous language-model agents as artificial life. Hosted by Ravid Shwartz Ziv and Allen Roush; watch the full research conversation on YouTube.