Q-learning

…on bandit tasks using variants of Q-learning. While this approach has been highly productive, leading to landmark discoveries such as the dopamine reward prediction error hypothesis, it also has limitations. This talk focuses on the lack of generalizability of such models: Even if they close

How does the activity of midbrain dopamine (DA) neurons influence behavior? A prominent, but incompletely tested, hypothesis is that the activity of ventral tegmental area (VTA) DA neurons generates downstream representations of state values in ventral striatum (NAc), while substantia nigra pars compacta DA activity generates action values in the dorsal striatum. To test this model, we performed Neuropixels recordings in a trial-and-error probabilistic learning task where mice continuously adapted their choices to obtain a reward of optogenetic stimulation of VTA DA neurons (paired with an aud

Maintaining physiological homeostasis is essential for survival. To achieve this, brains integrate various inputs, such as internal physiological and external environmental signals, to guide behavior. Inspired by brain-body literature, we consider three key axes along which the dynamics of these inputs can differ: their timescales, dependence on actions, and contribution to error signals that drive behavior. To understand how these axes impact physiological homeostasis, we consider an agent that can influence its internal physiology through interactions with the environment. We assume that env

Animals can learn to repeat actions that lead to rewards (model-free learning). However, animals can also learn about environmental structure without receiving immediate and explicit rewards (latent learning), and then use this knowledge to guide future reward-seeking behavior efficiently. How the brain accomplishes this is not clear, in part due to limitations in existing behavioral tasks to study latent learning in rodents. In particular, traditional latent learning tasks suffer from their “one-shot” characteristics, such that learning is typically performed and tested in one environment, pr