Reinforcement Learning ePosters

Task acquisition in the brain relies on both reward-based and error-based learning (RBL, EBL), yet how the brain brings these two learning signals together remains elusive. Here, we build on the traditional view that dopamine (DA) projections within the basal ganglia (BG) drive RBL via reinforcement learning, while…

The successor representation has emerged as a powerful model for understanding the spatial coding properties of hippocampal place cells and entorhinal grid cells. However, the diverse, spatially-tuned cells of the primary output of hippocampal neurons - the subiculum - have eluded a unified account. Here, we…

Animals rely on sensory inputs to navigate complex, dynamic environments. Understanding how the brain integrates these inputs to guide behavior remains a central question in neuroscience. Training AI and animals on the same task offers a unique opportunity to uncover both similarities and critical differences in the…

The brain navigates mazes using the entorhinal grid and hippocampal place codes. While specific grid codes can represent maze structures independent of behavioral policy, little is known about how these task structure-specific grid codes generalize to novel tasks requiring different structures and policies.…

The acquisition of novel motor skills requires new patterns of neural activity and changes in synaptic weights, yet the precise rules governing synaptic plasticity in the motor cortex and how they give rise to the observed changes in activity are still unknown. Brain computer interfaces (BCIs) allow for task…

Associative learning relies on the contingency between stimuli and outcomes, yet the neural mechanisms instantiating contingency remain unclear. We investigated dopamine activity in the ventral striatum of mice performing a Pavlovian contingency degradation task. Both anticipatory licking behavior and dopamine…

Animals use knowledge of the environment to guide efficient foraging decisions. It is unclear how this knowledge is accumulated or maintained by recurrently connected circuits in the brain. In order to guide the study of how structures of knowledge are accumulated and maintained to support optimal foraging decisions,…

Neuromodulatory systems play a crucial role in the brain, driving rapid, global changes in neuronal computation and learning. While the dopaminergic system is thought to broadcast a scalar reward prediction error, the serotonin system’s function has been difficult to elucidate, partly due to the heterogeneity of its…

The basal ganglia are a network of subcortical nuclei that play a central role in regulating motor and cognitive functions. A critical hub in these structures is the external globus pallidus (GPe) as it links all nuclei in this network. This nucleus was long considered a homogenous relay in the motor-suppressing…

Animals quickly learn to navigate to rewarding or salient landmarks in their environments. However, existing models often require thousands of trials to learn contingencies that animals learn within tens of trials, and they do so via unstructured sequences of actions that do not mimic real behavior. In this work, we…

The in vivo responses of dorsal raphe nucleus (DRN) serotonin neurons to emotionally-salient stimuli are a puzzle. Existing theories centred on reward, surprise, salience, and uncertainty individually account for some aspects of serotonergic activity but not others. Combining ideas from reinforcement learning (RL)…

Abstraction and generalization are essential for flexible decision-making in novel situations. Recent work in humans and monkeys has shown how abstract variables are encoded by the representational geometry of single-neuron population activity in a format that supports generalization. However, these observations…

One of key hallmarks of cognitive flexibility is the capability of quickly performing neural computation to accommodate dynamic, context-switching environments and deal with task uncertainty [1-3]. Accumulating evidence from human and animal experiments have suggested that the mediodorsal (MD) thalamus and prefrontal…

Maintaining physiological homeostasis is essential for survival. To achieve this, brains integrate various inputs, such as internal physiological and external environmental signals, to guide behavior. Inspired by brain-body literature, we consider three key axes along which the dynamics of these inputs can differ:…

Emotions significantly shape behavior, with mood---a prolonged emotional state---affecting decision-making processes by biasing perceptions of future outcomes. Recent studies suggest that mood functions as an extrapolation mechanism, tracking deviations between recent experiences and expectations to predict and inform…

When rodents learn to navigate in a new environment, high place field density emerge at reward locations, fields elongate against the trajectory, and individual fields change spatial selectivity or drift while demonstrating stable behavior. Why place fields demonstrate these characteristic phenomena during learning…

Phasic dopamine (DA) release plays a major role in learning by assigning incentive value to associated stimuli. A leading theory proposes that this process is analogous to a reinforcement learning algorithm called temporal difference (TD) learning, and that DA acts analogously to the reward prediction error (RPE) term…

How does the activity of midbrain dopamine (DA) neurons influence behavior? A prominent, but incompletely tested, hypothesis is that the activity of ventral tegmental area (VTA) DA neurons generates downstream representations of state values in ventral striatum (NAc), while substantia nigra pars compacta DA activity…

Midbrain dopamine (DA) neurons send teaching signals, reward prediction errors (RPEs), to all regions of the striatum [1]. However, these signals converge with regionally segregated inputs from all levels of the cortical hierarchy [2] that vary in degree of abstraction and temporal extendedness [3]. Recent work…

Traditional approaches to studying decision-making in neuroscience focus on simplified behavioral tasks where animals perform repetitive, stereotyped actions to receive explicit rewards. While informative, these methods constrain our understanding of decision-making to short timescale behaviors driven by explicit…

Every day, we make decisions between multidimensional options. Learning each option’s value by simultaneously attending to all its dimensions is challenging. However, humans navigate this complexity by selectively focusing on relevant information, making real-world, multidimensional reinforcement learning (RL)…

Although reinforcement learning has been highly successful in training artificial agents to perform a wide variety of complex tasks, the principles governing how animals actually learn to perform new tasks remain poorly understood. While normative approaches have provided profound insights into the design of optimal…

Because the world is dynamic and only imperfectly observable, many of the decisions we make are necessarily uncertain. How do we navigate such uncertainty? From the perspective of cognitive neuroscience, the classic answer would be that we evaluate the benefits of each potential choice and then lean towards the one…

Goal-directed navigation is a complex cognitive process that requires a precise relational representation of the spatial environment and its associated rewards. This process enables agents to choose optimal paths that maximize long-term rewards, which is essential for the survival of both humans and animals. While…