ePoster

Linking tonic dopamine and biased value predictions in a biologically inspired reinforcement learning model

Sandra Romero Pinto,Naoshige Uchida
COSYNE 2022(2022)
Lisbon, Portugal

Conference

COSYNE 2022

Lisbon, Portugal

Resources

Authors & Affiliations

Sandra Romero Pinto,Naoshige Uchida

Abstract

Some psychiatric disorders are characterized by excessively optimistic or pessimistic predictions of future events, as well as changes in dopamine levels. However, how changes in dopamine could lead to biased value predictions is unknown. Here, we draw this link by examining the role of baseline dopamine levels in value learning. Value learning is thought to depend, in part, on synaptic plasticity driven by dopamine reward prediction errors (RPEs) acting upon D1 and D2 receptors in medium spiny neurons (MSNs) of the striatum. At reported striatal dopamine levels, D1 and D2 receptors are mostly unoccupied and occupied by dopamine, making them sensitive to increases and decreases of dopamine, respectively. Accordingly, studies have reported that potentiation in MSNs expressing D1 or D2 receptors is triggered by phasic increases or decreases of dopamine [1,2]. Moreover, given the sigmoidal dose-occupancy relationship of these receptors, shifts in dopamine baseline should change their sensitivity to dopamine transients (i.e., take the baseline to a “steep” or “shallow” region of the dose-occupancy curve). Here, we show that a reinforcement learning (RL) model incorporating these plasticity rules develops positive or negative biases in predictions of probabilistic rewards when baseline dopamine is increased or decreased, respectively. We validate this model using data from a previous study [3]. This study showed that lesions of the habenula resulted in positive biases both in reward-seeking behavior (anticipatory licking) and dopamine neurons responses to cues predictive of probabilistic rewards. In our model, an increase in baseline firing of dopamine neurons, which was observed in the data, is sufficient to lead to optimistic biases. Taken together, our biologically inspired RL model highlights a causal impact of baseline dopamine on biasing value predictions, which may underlie abnormalities in psychiatric patients, including altered risk preferences.

Unique ID: cosyne-22/linking-tonic-dopamine-biased-value-predictions-6cdd61cd