Seminar, Hosted by Fields Institute for Research in Mathematical Sciences
Influencing generalization through the geometry of the loss
Resolution
Abstract
Neural networks can reach different regions of the same set of loss minima and generalize differently. Elliott examines patterning, a technique that changes the optimization problem, for example by adjusting the mixture of training distributions, to favor particular generalization behavior. Singular learning theory connects those changes in the data distribution to the Bayesian posterior.