1 h 6 min
Seminar, Hosted by Simons Institute for the Theory of Computing
A Statistical Theory of Contrastive Pre-training and Multimodal Generative AI
UC Berkeley
Recording
Abstract
The talk asks why contrastive pretraining helps with tasks as different as zero-shot classification and conditional image generation. Near-optimal contrastive representations are shown to retain approximately sufficient statistics. A hierarchical model of paired images and text connects transformer computation to belief propagation and yields sample-complexity bounds tested in simulations.
Topics
- Contrastive Pre-training, broader topic: pre-training
- multimodal generative AI, broader topic: generative AI
- zero-shot classification
- conditional image generation, broader topic: image generation
- contrastive representations, broader topic: representations
- sufficient statistics, broader topic: statistics
- hierarchical model
- transformer computation, search World Wide
- belief propagation
- sample-complexity bounds