1 h 4 min
Seminar, Hosted by Simons Institute for the Theory of Computing
In-Context Learning: A Case Study of Simple Function Classes
Stanford University
Recording
Abstract
Gregory Valiant compares how transformers and recurrent architectures learn functions from input-output examples embedded in the prompt. The talk uses linear and sparse linear functions and small decision trees to examine training efficiency, evaluation methods and differences between architectures. It also discusses whether regression and similar primitives help explain in-context learning in language tasks.