Skip to contentSkip to filters

We use cookies for analytics.

Weak-to-Strong Generalization Even in Random Feature Networks, Provably

Marko Medvedev

University of Chicago

Recording

1 h

Abstract

Can a student model outperform the weaker teacher that supplied all of its labels? Using random-feature networks with fixed first layers and trainable output layers, the speaker shows how a larger student can do so. The analysis explains the role of early stopping and establishes quantitative limits on the improvement.

Topics