1 h
Seminar, Hosted by Simons Institute for the Theory of Computing
Adam and friends via duality
Recording
Abstract
How do adaptive gradient methods such as Adam look in the dual formulation of an optimization problem? The talk derives and analyzes several methods through duality, showing relationships between them that are less apparent in the primal formulation. Examples emphasize linearly separable problems, with neural-network training as motivation.
Topics
- Adam, search World Wide
- adaptive gradient methods, broader topic: gradient methods
- dual formulation, search World Wide
- optimization problem, search World Wide
- duality
- primal formulation, search World Wide
- linearly separable problems, search World Wide
- neural-network training, broader topic: training
- gradient methods