ePoster

Dopamine ramps encode discounted future value on a moment-by-moment basis

Johannes de Jong, Yilan Liang, Stephan Lammel
COSYNE 2025(2025)
Montreal, Canada

Conference

COSYNE 2025

Montreal, Canada

Resources

Authors & Affiliations

Johannes de Jong, Yilan Liang, Stephan Lammel

Abstract

Dopamine (DA) ramps are gradual increases in DA concentration observed during motivated behavior, contrasting with phasic DA release typically associated with signaling reward prediction errors (RPE). While RPE-related DA transients align with traditional models of reinforcement learning, where DA encodes the difference between expected and received rewards, dopamine ramps are less understood. Previous research demonstrated that DA ramps occur both at the level of DA cell bodies in the ventral tegmental area (VTA) and at the axon terminals in the nucleus accumbens (NAc), specifically in the medial VTA and medial NAc (NAcMed), respectively. Conversely, lateral VTA DA neurons projecting to the lateral NAc (NAcLat) encode the temporal derivative of these ramps. However, the precise information that is encoded in these ramps remains subject to intense debate. To address this, we developed a novel virtual reality task that allows for flexible, within-trial manipulations of key aspects of reward-seeking behavior. Using in vivo fiber photometry to record DA release in different NAc subregions, we found that DA ramps in the NAcMed reflect the temporally discounted expected trial outcome. In contrast, DA release in the NAcLat showed transient peaks at cue onset and reward delivery, corresponding to the temporal derivative of the discounted value, a continuous assessment of the temporal difference RPE that does not depend on discrete state evaluations. This parallel encoding of future expected value and its derivative challenges our understanding of how the mesolimbic DA system supports reinforcement learning. This is significant for two reasons. First, it provides a better understanding of the neural mechanisms underlying motivated behavior, offering potential therapeutic targets for conditions like drug addiction and eating disorders. Second, understanding dopamine ramps could enhance computational models of reinforcement learning and decision-making, leading to the development of more sophisticated algorithms that better mimic the complexities of animal behavior.

Unique ID: cosyne-25/dopamine-ramps-encode-discounted-9582cc2d