Deep Learning seminars

Deep networks display recurring phenomena such as grokking, scaling behaviour, generalization despite overparameterization and failures outside training distributions. This talk surveys controlled settings where those behaviours can be explained, suggesting that language models may combine many tractable, largely interpolative mechanisms. It then distinguishes that scientific account from a more speculative argument about the role of industrial computation and human labour, and what these imply for claims of broadly extrapolative intelligence.

Diffusion and energy-based models can support reasoning by searching during inference and combining separate learned components. This talk develops that compositional perspective and examines how smaller models can jointly solve more complex questions. Empirical examples cover symbolic reasoning, decisions and planning.

Diffusion and flow models provide expressive distributions over continuous actions in robotic control. This talk explains how modelling extended action sequences improved imitation learning and subsequently offline and offline-to-online reinforcement learning. It reviews algorithms built around these representations and large-scale control models that use them, connecting policy expressiveness with practical learning performance.

Masked diffusion provides a common lens for comparing autoregressive and diffusion-based generation. This talk argues that observed differences in parallelism and perceptual quality can follow from training and specification choices rather than a fundamental divide. It then studies adaptive generation orders, insertion-based planning and scalable maximum-likelihood learning beyond fixed or random ordering.

Inference-time algorithms place pretrained models inside larger computational loops, trading extra computation for accuracy or control. This talk studies efficiency and error propagation with imperfect learned oracles. Examples include stochastic backtracking in generator–verifier systems and the conditions under which diffusion steering can favour rewards while remaining close to the original model.

This talk surveys the components of modern diffusion language models, beginning with masked prediction and extending to iterative refinement, controlled generation and variable-length inference. It examines how parallel generation and repeated revision change inference-time computation. Examples from natural language, proteins and DNA illustrate both practical model designs and uses of fine-grained control in scientific sequence generation.