47 min
Seminar, Hosted by Simons Institute for the Theory of Computing
Meta-learning of Optimizers and Update Rules
Google Brain
Recording
Abstract
Jascha Sohl-Dickstein examines when a learned parameter update rule or loss can better match the desired behavior than a hand-designed one. He presents update rules for unsupervised representations, semi-supervised learning, data augmentation and classification, explores constraints that make some learned rules biologically plausible, and discusses failure modes and remedies when optimizing the rules themselves.