Exploration

…there are multiple mechanisms for exploration and we

…This exploration is modulated by dopamine release in the striatum, a prediction-error teaching signal that allows animals to learn from both rewards and threats depending on the striatal subregion in which dopamine is released. Exploration and dopamine signaling are modulated by internal state, including caloric depletion or “hunger”, such that when animals are hungry they explore their environment more, presumably in an attempt to find food. While this adaptation has been known for decades, the n

Rodents and primates exhibit trade-offs between exploitation and exploration. Understanding how different behavioral states modulate neural activities remains a critical question. Previous research has demonstrated that high and low head pitch in rats correlate with distinct head-movement frequencies, which are coupled with breathing frequency, suggesting a potential connection between pitch states and exploration/exploitation behaviors. Additionally, place cells exhibit remapping across low/high-pitch states. In our study using a path-integration paradigm with rats, we found that parameters

…potentially better alternatives (exploration). Despite its significance, the intricate computational and neural mechanisms that underpin these processes remain unclear. To address this question, we designed a task in which head-fixed mice acquire and reverse self-initiated lever-pressing preferences in the absence of instructive external cues. We combined

…alternation between long-lasting exploration or exploitation states (Flavell et al., 2013; Martin, Ernst, & Heisenberg, 1999, Marques et al., 2020). We aim to understand how these i

Sex-based modulation of cognitive processes could set the stage for individual differences in vulnerability to neuropsychiatric disorders. While value-based decision making processes in particular have been proposed to be influenced by sex differences, the overall correct performance in decision making tasks often show variable or minimal differences across sexes. Computational tools allow us to uncover latent variables that define different decision making approaches, even in animals with similar correct performance. Here, we quantify sex differences in mice in the latent variables underlying

…show substantial variability in their exploration, and theoretical but only limited empirical work has suggested that excessive exploration is a critical mechanism underlying the psychiatric dimension of impulsivity. We put these theories to test using a large online sample (N=580 healthy adults), dimensional analyses, and computational modelling in a pre-registered study. Capitalising

In uncertain environments, we balance exploitation and exploration: we generally exploit rewarding opportunities, but sometimes explore uncertain alternatives that could be even better. Exploration is associated with a sudden disorganization of neuronal activity patterns in the prefrontal cortex, which could be a powerful way to promote discovery and learning. Although the mechanisms behind this disorganization remain unknown, one possibility is pupil-linked neuromodulatory systems. However, it is not clear whether pupil size predicts the neural signatures of exploration, much less the sudden

Identifying situations in which exploration could be beneficial, and learning from such choices, is critical for survival in complex environments. Nonetheless, it is still unclear how animals adapt their choice strategies depending on the potential benefits of exploration and what the neural underpinning of this ability is. We thus adapted a 2-option forced-choice task in which 3 rhesus macaques made choices while the information and the feedback they received was being manipulated: (i) feedback was given about either just the chosen option or both the chosen and unchosen counterfactual option

Exploration is an essential part of learning, and the question of how to achieve efficient exploration has been extensively studied in the field of Reinforcement Learning (RL). Recent works have demonstrated -- both in theory and in practice -- the importance and effectiveness of sophisticated methods of exploration which are sensitive to the long-term exploratory consequences of actions, and to the global structures of the environment. This is analogous to the standard concept of value in RL. How "good" an action is depends not only on the immediate reward gained by taking it, but also on the

…valuation is the key variable to guide exploration, little is known about the role of another variable - the uncertainty in state-space representation. One reason is that a simple task design consisting of only a few states and actions cannot accommodate the uncertainty of the environmental structure. Here, we hypothesize that metacognition, the human's unique ability to introspect and estimate one's level of uncertainty, gui

Machine learning researchers frequently focus on human-level performance, in particular in games. However, in these applications human (or human-level) behavior is commonly reduced to a simple dot on a performance graph. Cognitive science, in particular theories of learning and decision making, could hold the key to unlock what is behind this dot, thereby gaining further insights into human cognition and the design principles of intelligent algorithms. However, cognitive experiments commonly focus on relatively simple paradigms such as restricted multi-armed bandit tasks. In this talk, I will