Action selection

The concept that discrete actions act as an atomic basis of behavior has pervaded ethology from Lorenz and Tinbergen’s foundational studies to the current computational era. Recent research corroborates these atomic units, showing that laboratory mice choose actions that appear syllabic: i.e. both discontinuous in time and syntactically structured. Yet, little is known about how these syllables vary across phylogeny, where species adapt both behavior and physiology to form ecological niches. Here, we address this question by analyzing the syllabic repertoire and distribution of 18 species and

…[2] that vary in degree of abstraction and temporal extendedness [3]. Recent work highlights how regionally diverse DA RPEs might update parallel estimates of future reward as informed by region-specific inputs[4, 5], yet the behavioral implications of such parallel processing are poorly understood. We used photometry to measure DA release across the striatum in mice

Animals compose low-level actions in the service of high-level goals. Prefrontal cortex is essential for this organization when the goals are externally imposed, as in tasks where reward contingencies switch over time. Here we asked if it plays a similar role during spontaneous exploratory behaviors in the mouse. Using a hierarchical model, we identify stable patterns of action selection that capture emergent ‘task-states’ like wall following, object investigation or social interaction. These states align with distinct epochs of activity in medial prefrontal cortex (mPFC) – a coincidence that

…mechanism controlling the speeds of action selection and execution. To test this hypothesis, we developed a self-paced foraging task in which rats run back-and-forth across a motorized treadmill to collect drops of water, and separately manipulated reward and effort. We used simple models to 1) quantify changes in running speed and in the pause between runs (run and inter-run durations) and 2) explore algorithmic and neural principles controlling these

Humans tend to repeat past actions, with habits as extreme case. With each execution of an action this repetition bias increases the probability to repeat this action. However, there is a lack of experiments investigating repetition bias.To test for the presence of a repetition bias, we developed a new sequential decision-making task (see Figure). For this task, participants had to collect points with four moves and meet a trial-specific goal as closely as possible. To ensure frequent repetition of at least one sequence of actions, we informed participants about a default action sequence that

Individuals regularly have to decide upon motor actions to be performed. Motor decision-making involves the selection between and specification of potential action plans (Cisek, 2007) and necessitates the integration of perceptual, cognitive, and motor processes (Cisek & Kalaska, 2010; van Ede & Nobre, 2023). The influence of cognitive load during decision-making on subsequent motor action execution remains largely unexplored (Gordon et al., 2021). This study investigates the influence of working memory (WM) load on goal-directed pointing movements, hypothesizing that increased WM load

…with superior performance in action selection (AS). However, the causal relationship has not been thoroughly examined. Therefore, this study aimed to determine if M1-PMd connectivity could be modulated through covert fMRI-NF during rest, subsequently affecting cognitive-motor connectivity and behaviour. 20 adults took part in this counterbalanced within-subject double-blind s

Striatal dopamine (DA) is critical to the regulation of action selection and motivation. DA neurons have extensively branched axons which afford axonal regulation of DA release by diverse striatal neuromodulators including acetylcholine (ACh), GABA and adenosine, which regulate the amplitude and activity-sensitivity of DA release. Adrenoceptors are expressed throughout the striatum but their role, and that of norepinephrine (NE) as a potential striatal neuromodulator, is poorly understood. Given only a sparse norepinephrinergic input of the striatum but the dense innervation by DA, the endogen

…(storage of procedural memories and/or action selection/suppression) and therefore drive the execution of learned actions. However, other studies have shown that the dStr might also be implicated in the moment-to-moment kinematics specification of learned movements, or the vigor of reward-oriented actions, which reflect m

I will discuss work with Jack Lindsey modeling reinforcement learning for action selection in the basal ganglia. I will argue that the presence of multiple brain regions, in addition to the basal ganglia, that contribute to motor control motivates the need for an off-policy basal ganglia learning algorithm. I will then describe a biological implementation of such an algorithm that predicts tuning of dopamine neurons to a quantity we call "action surprise," in addition to reward prediction error. In the same model, an implementation of learning from a motor efference copy also predicts a novel

…choices to arrive at an appropriate action to report the choice. When sensory evidence and potential actions to report choice are simultaneously available, neural activity in dorsal premotor (PMd) and lateral intraparietal (LIP) cortices covary with sensory evidence and choice. However, when sensory evidence and possible actions are separated in time, PMd and LIP show action selection related responses and only when the stimulus-response relationship is apparent. Thus, these areas are largely involved in action selec

During visually guided behavior, the prefrontal cortex plays a pivotal role in mapping sensory inputs onto appropriate motor plans. When the sensory input is ambiguous, this involves deliberation. It is not known whether the deliberation is implemented as a competition between possible stimulus interpretations or between possible motor plans. Here we study neural population activity in prefrontal cortex of macaque monkeys trained to flexibly report categorical judgments of ambiguous visual stimuli. The subjects judged whether a visual stimulus presented near the central visual field was orient

…value) to choose an appropriate action plan. Neural correlates of perceptual decisions are more distributed than once thought, having been found within the cortex, striatum, and superior colliculus (SC). Recent work has suggested a potentially novel node in this network - the midbrain reticular nucleus (MRN) - which was one of the few regions that encoded visual and action selection signals in a brain-wide survey of

…possible role for the basal ganglia in action selection, guided by reinforcement learning. In this talk, I will discuss a framework for how this function might be performed and computational results using an upward mapping to identify putative low-dimensional control ensembles that may be involved in tuning decision policy. I will also present some recent experimental results and theory – related to ef

The basal ganglia (BG) play a crucial role in action-selection and reinforcement learning (RL), but how multiple nuclei, transmitters and receptors realize computations for reward-based learning is still unclear. We built a topologically organized spiking BG model. Striatal medium spiny neurons (MSN) were classified based on the expression of dopamine D1 and D2 receptors. We implemented spike-timing dependent plasticity and structural parameters: i) the asymmetry of connections between MSN’s; and ii) the overlap between direct and indirect pathways. In action-selection simulations, we assume

Multiple brain regions are involved in integrating reward to drive action selection, with rich representations of reward history in the striatum, retrosplenial cortex, and frontal areas. A major challenge in dissecting the circuits that govern reward-guided behavior is the existence of multiple strategies for reward maximization. For instance, in dynamic environments, mice can engage in both model-free behavior, where they update action values from trial to trial, and inference-based learning, where they use an internal model to infer the current world state. These two modes are challenging to

…possible role for the basal ganglia in action selection, guided by reinforcement learning. In this talk, I will discuss a framework for how this function might be performed. I will also present some recent experimental results and theory that call for a re-evaluation of certain aspects of this framework. Next, I will turn to the changes in basal ganglia activity observed to occur with the dopamine deple

In addition to its role in the learning and expression of conditioned behavior, the amygdala has long been implicated in the regulation of persistent states, such as anxiety and drive. Yet, it is not evident what projections of the neuronal activity capture the functional role of the network across such different timescales, specifically when behavior and neuronal space are complex and high-dimensional. We applied a data-driven dynamical approach for the analysis of calcium imaging data from the basolateral amygdala, collected while mice performed complex, self-paced behaviors, including spati

…context-dependent parameters of action selection, affect,