Reinforcement Learning seminars

Diffusion and flow models provide expressive distributions over continuous actions in robotic control. This talk explains how modelling extended action sequences improved imitation learning and subsequently offline and offline-to-online reinforcement learning. It reviews algorithms built around these representations and large-scale control models that use them, connecting policy expressiveness with practical learning performance.

Rare molecular and physical transitions are difficult to observe in ordinary simulations. This talk casts estimation of the committor function as stochastic optimal control, using feedback to sample reactive trajectories. It develops backpropagation and off-policy Value Matching objectives, addresses trapping in intermediate metastable states, and compares committor, rate and equilibrium estimates on benchmark systems.

Applying language-model reinforcement-learning recipes directly to diffusion trajectories is costly in both computation and memory. This talk examines objectives, likelihood estimation and rollout design for diffusion post-training. An estimator based on a forward-process variational bound separates training from rollout and permits fast ODE sampling, producing substantial speed improvements. A related first-order stochastic-control approach is also considered.

Traditional work in the study of human reward-based learning involves designing an experimental task---often inspired by Reinforcement Learning (RL) theory---and fits a small set of computational models---often inspired by RL algorithms---to that dataset. For example, researchers often model human behavior on bandit tasks using variants of Q-learning. While this approach has been highly productive, leading to landmark discoveries such as the dopamine reward prediction error hypothesis, it also has limitations. This talk focuses on the lack of generalizability of such models: Even if they closely fit behavior on the original task, models derived from the one-task-one-model paradigm usually predict behavior on other tasks quite poorly. I argue that this lack of generalizability is a fundamental problem for the cognitive sciences: we intuitively expect our models to be robust to superficial task differences, such as variations in the number of choice options, reward probabilities, or the exact kind of non-stationarity. I will propose potential solutions to this problem along two dimensions: the behavioral dataset and the computational model. Regarding computational models, I will introduce work in which we moved beyond the limitations of hand-crafted one-off models by employing flexible, data-driven methods. These methods allowed us to compare classes of models instead of individual model instances, allowing us to cover the space of possible models more exhaustively, and innovate cognitive mechanisms very efficiently. For the behavioral dataset, we move from using single learning tasks to a comprehensive task space that encompasses most existing paradigms in the literature, while closing the gaps between them in a near-continuous fashion. Our results suggest that more general models in conjunction with broader datasets can pave the road toward increasingly general models of human reward-based learning and decision making, and a persistent departure from many aspects of RL theory. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2026-05-20. Recording duration: 00:51:19.

Edmond Chow examines how asynchronous updates improve parallel iterative computation. The first part covers asynchronous versions of classical first- and second-order linear iterations, Chebyshev methods, and multigrid, with attention to efficiency and fault tolerance. The second introduces reinforcement learning and asynchronous state-value estimation for finding optimal policies. When the state space is too large to enumerate, these updates focus computational effort on frequently visited regions.

This talk aims to answer these questions along four dimensions. First is history. RL was the basis of AI long before the term AI was introduced in 1956. The first machine learning (ML) systems were based on RL even before digital computers existed. Despite notable early successes of ML based on RL, RL essentially disappeared from ML until relatively recently. A second reason for renewed interest in RL is the clarification of some misunderstandings that have been prevalent in the ML community. A third, and most important, reason for this resurgence is that new, or rediscovered, algorithms and connections to well developed mathematical and engineering methods have been worked out. Finally, a fourth reason for the renewed interest in RL is its strong links to animal reward systems, in particular, to the role that dopamine plays in motivation and learning. VVTNS Sixth Season Opening Lecture. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2025-10-29. Recording duration: 00:55:21.