47 min
Seminar, Hosted by Simons Institute for the Theory of Computing
Efficient Reinforcement Learning for Diffusion Models
Georgia Institute of Technology
Recording
Abstract
Applying language-model reinforcement-learning recipes directly to diffusion trajectories is costly in both computation and memory. This talk examines objectives, likelihood estimation and rollout design for diffusion post-training. An estimator based on a forward-process variational bound separates training from rollout and permits fast ODE sampling, producing substantial speed improvements. A related first-order stochastic-control approach is also considered.
Topics
- language-model reinforcement-learning, broader topic: reinforcement learning
- diffusion trajectories, search World Wide
- diffusion post-training, broader topic: post-training
- objectives, search World Wide
- likelihood estimation, search World Wide
- rollout design, broader topic: design
- forward-process variational bound, search World Wide
- ODE sampling, broader topic: sampling
- first-order stochastic-control approach, search World Wide