Alignment of ANN Language Models with Humans After a Developmentally Realistic Amount of Training
Eghbal Hosseini, Martin Schrimpf, Yian Zhang, Samuel Bowman, Noga Zaslavsky, Evelina Fedorenko · COSYNE 2023
Artificial neural networks (ANN) have emerged as computationally plausible models of human language processing. A major criticism of these models is that the amount of training data they receive far exceeds that of humans. Here, we use two complementary approaches to ask how the models’ ability to capture human neural and behavioral responses to language is affected by the amount of training data. First, we evaluate GPT-2 models trained on 1 million, 10 million, 100 million, or 1 billion tokens against a neural (fMRI) and a behavioral (reading times) benchmark. Because children are exposed to
Sun, Mar 12, 2023