Skip to contentSkip to filters

We use cookies for analytics.

A Statistical Theory of Contrastive Pre-training and Multimodal Generative AI

Song Mei

UC Berkeley

Recording

1 h 6 min

Abstract

The talk asks why contrastive pretraining helps with tasks as different as zero-shot classification and conditional image generation. Near-optimal contrastive representations are shown to retain approximately sufficient statistics. A hierarchical model of paired images and text connects transformer computation to belief propagation and yields sample-complexity bounds tested in simulations.

Topics