Cognitive Psychology

Traditional work in the study of human reward-based learning involves designing an experimental task---often inspired by Reinforcement Learning (RL) theory---and fits a small set of computational models---often inspired by RL algorithms---to that dataset. For example, researchers often model human behavior on bandit tasks using variants of Q-learning. While this approach has been highly productive, leading to landmark discoveries such as the dopamine reward prediction error hypothesis, it also has limitations. This talk focuses on the lack of generalizability of such models: Even if they closely fit behavior on the original task, models derived from the one-task-one-model paradigm usually predict behavior on other tasks quite poorly. I argue that this lack of generalizability is a fundamental problem for the cognitive sciences: we intuitively expect our models to be robust to superficial task differences, such as variations in the number of choice options, reward probabilities, or the exact kind of non-stationarity. I will propose potential solutions to this problem along two dimensions: the behavioral dataset and the computational model. Regarding computational models, I will introduce work in which we moved beyond the limitations of hand-crafted one-off models by employing flexible, data-driven methods. These methods allowed us to compare classes of models instead of individual model instances, allowing us to cover the space of possible models more exhaustively, and innovate cognitive mechanisms very efficiently. For the behavioral dataset, we move from using single learning tasks to a comprehensive task space that encompasses most existing paradigms in the literature, while closing the gaps between them in a near-continuous fashion. Our results suggest that more general models in conjunction with broader datasets can pave the road toward increasingly general models of human reward-based learning and decision making, and a persistent departure from many aspects of RL theory. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2026-05-20. Recording duration: 00:51:19.

Reward expectations – one’s prediction about the likelihood of future outcomes - play a central role in shaping the satisfaction derived from those outcomes. Most existing research treats expectations as static, assuming they remain fixed in time. However, real-life expectations are often dynamic, fluctuating as new information becomes available. For example, during a soccer game, your expectations of seeing your team winning will likely rise and fall as the game unfolds. In the main part of this talk, I will present a series of studies demonstrating that human expectations can be tracked at sub-second timescales. Using slot machines as a case study, we leverage the continuous deceleration of the reels to elicit moment-by-moment fluctuations in rewardexpectations. To capture these dynamics, we take complementary approaches: we use the high temporal resolution of electroencephalography (EEG) to track neural signatures of evolving expectations, and we develop a novel behavioral paradigm (“Slot or Not”) designed to measure changes in expectations via betting behavior. Across four studies, we show that expectations fluctuate continuously and can be tracked both behaviorally and neurally. Extending these findings, a subsequent intracranial study shows that the human orbitofrontal cortex (OFC) encodes the moment-by-moment changes of reward expectations. In the second part of this talk, I will return to the relationship between expectations andsatisfaction. If expectations shape satisfaction, and if they are best conceptualized as dynamic trajectories rather than static quantities, a key question arises: does the trajectory leading up to an outcome influence how that outcome is evaluated? I will outline a new research direction aimed at formalizing this relationship using computational modeling. This is ongoing work, and I welcome feedback on how best to formalize these ideas. Finally, I will discuss potential extensions of this framework to psychopathology, asking whether alterations in dynamic expectations may characterize conditions such as Major Depressive Disorder and Gambling disorder. Together, this work introduces a new framework for studying expectations as dynamic processes, offering a richer understanding Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2026-04-29. Recording duration: 00:42:25.