1 h
Seminar, Hosted by Simons Institute for the Theory of Computing
Weak-to-Strong Generalization Even in Random Feature Networks, Provably
University of Chicago
Recording
Abstract
Can a student model outperform the weaker teacher that supplied all of its labels? Using random-feature networks with fixed first layers and trainable output layers, the speaker shows how a larger student can do so. The analysis explains the role of early stopping and establishes quantitative limits on the improvement.
Topics
- Weak-to-Strong Generalization, broader topic: generalization
- Random Feature Networks, broader topic: networks
- student model, broader topic: model
- weaker teacher, search World Wide
- labels, search World Wide
- fixed first layers, search World Wide
- trainable output layers, search World Wide
- Early Stopping
- quantitative limits, search World Wide