Language, Cognition, and the Limits of LLMs - with Tal Linzen (NYU/Google)

Tal Linzen of NYU and Google discusses the relationship between cognitive science and large language models. Children learn language from about 100 million words while models need trillions, and as models improve at next-word prediction they become worse models of human language processing. He explains how his lab compares models with human eye-tracking and reading-time data to study prediction and working memory, and how inductive biases can be instilled by pretraining on synthetic languages.