ePoster

Preference dynamics in economic decision-making explained by dopaminergic distributional codes

Mehrdad Salmasi, Raymond Dolan
Bernstein Conference 2024(2024)
Goethe University, Frankfurt, Germany

Conference

Bernstein Conference 2024

Goethe University, Frankfurt, Germany

Resources

Authors & Affiliations

Mehrdad Salmasi, Raymond Dolan

Abstract

The brain's ability to make economically efficient decisions plays a crucial role in processes like foraging and risk management. Given the inherent uncertainty about reward outcomes and world's latent states, the brain needs to construct distributional beliefs to enhance its performance in the decision-making process. It is not yet fully understood how the brain makes economic decisions in the presence of uncertainty, for instance, in a two-armed bandit task where arms have different reward distributions. Different frameworks have been suggested for representing probability distributions in the brain [1-5]. We hypothesize that the brain represents the reward distributions with the expected values of dopaminergic encoding functions. This encoding scheme, known as distributed distributional coding (DDC), has been effectively used in various settings [6-10]. We propose a DDC network that receives the recursively updated DDC values as input and outputs a mapping of the weighted sums of the DDC values. We show that with a simple learning rule, the preference/risk measure of the brain can be inferred; moreover, the DDC network suggests a biologically plausible implementation of the inferred measure in the dopaminergic system. Studies indicate that the agent's risk attitude and preference measure vary over time and across different contexts; for example, subjects may transition from consistently avoiding risk to adopting a risk-seeking attitude. Yet, the mechanism by which the reward system alters the preference/risk measure remains unclear. We study the dynamics of preference measures and show that modifications in the read-out weights of the dopaminergic neurons in the DDC network can explain a wide range of alterations in the agent's risk attitude. The model captures the temporal changes of utility function and explains variations of coherent risk measure. For instance, we demonstrate how an agent who is using the expected tail gain, ETG, with ten percent quantile of reward distribution, transitions to an ETG with fifty percent quantile, or even switches to an expected utility measure. Our framework provides an inference mechanism to derive the preference measure, proposes a biologically plausible implementation of the measure in the dopaminergic system, and explains how modifications of synapses, which correspond to readout weights, alter agent's preference measure and risk attitude.

Unique ID: bernstein-24/preference-dynamics-economic-decision-making-e5313575