Temporal Hierarchies in Reward and Behavioral Control
The basal ganglia and addiction
Developmental and evolutionary perspectives on thalamic function
Brain organization and function is a complex topic. We are good at establishing correlates of perception and behavior across forebrain circuits, as well as manipulating activity in these circuits to affect behavior. However, we still lack good models for the large-scale organization and function of the forebrain. What are the contributions of the cortex, basal ganglia, and thalamus to behavior? In addressing these questions, we often ascribe function to each area as if it were an independent processing unit. However, we know from the anatomy that the cortex, basal ganglia, and thalamus, are massively interconnected in a large network. One way to generate insight into these questions is to consider the evolution and development of forebrain systems. In this talk, I will discuss the developmental and evolutionary (comparative anatomy) data on the thalamus, and how it fits within forebrain networks. I will address questions including, when did the thalamus appear in evolution, how is the thalamus organized across the vertebrate lineage, and how can the change in the organization of forebrain networks affect behavioral repertoires.
Dopaminergic Network Dynamics
Cholinergic Interneurons
Honorary Lectures 2025
Schizophrenia and BG
SWEBAGS conference 2024: The basal ganglia in action
SWEBAGS conference 2024: Shared network mechanisms of dopamine and deep brain stimulation for the treatment of Parkinson’s disease: From modulation of oscillatory cortex – basal ganglia communication to intelligent clinical brain computer interfaces
SWEBAGS conference 2024: The involvement of the striatum in autism spectrum disorder
Basal Ganglia in Songbirds
Top-down models of learning and decision-making in BG
Cerebellum-Basal Ganglia Interactions
Dopamine Acetylcholine interactions
Subthalamic nucleus
Honorary Lecture 2024
Imaging the subcortex; Microstructural and connectivity correlates of outcome variability in functional neurosurgery for movement disorders
We are very much looking forward to host Francisca Ferreira and Birte Forstmann on December 14th, 2023, at noon ET / 6PM CET. Francisca Ferreira is a PhD student and Neurosurgery trainee at the University College of London Queen Square Institute of Neurology and a Royal College of Surgeons “Emerging Leaders” program laureate. Her presentation title will be: “Microstructural and connectivity correlates of outcome variability in functional neurosurgery for movement disorders”. Birte Forstmann, PhD, is the Director of the Amsterdam Brain and Cognition Center, a Professor of Cognitive Neuroscience at the University of Amsterdam, and a Professor by Special Appointment of Neuroscientific Testing of Psychological Models at the University of Leiden. Besides her scientific presentation (“Imaging the human subcortex”), she will give us a glimpse at the “Person behind the science”. You can register via talks.stimulatingbrains.org to receive the (free) Zoom link!
September webinar
Dopamine and Acetylcholine waves in the striatum
May Webinar
Auditory input to the basal ganglia; Deep brain stimulation and action-stopping: A cognitive neuroscience perspective on the contributions of fronto-basal ganglia circuits to inhibitory control
On Thursday, May 25th we will host Darcy Diesburg and Mark Richardson. Darcy Diesburg, PhD, is a post-doctoral research fellow at Brown University. She will tell us about “Deep brain stimulation and action-stopping: A cognitive neuroscience perspective on the contributions of fronto-basal ganglia circuits to inhibitory control”. Mark Richardson, MD, PhD, is the Director of Functional Neurosurgery at the Massachusetts General Hospital, Charles Pappas Associate Professor of Neurosciences at Harvard Medical School and Visiting Associate Professor of Brain and Cognitive Sciences at MIT. Beside his scientific presentation on “Auditory input to the basal ganglia”, he will give us a glimpse at the “Person behind the science”. The talks will be followed by a shared discussion. You can register via talks.stimulatingbrains.org to receive the (free) Zoom link!
Richly structured reward predictions in dopaminergic learning circuits
Theories from reinforcement learning have been highly influential for interpreting neural activity in the biological circuits critical for animal and human learning. Central among these is the identification of phasic activity in dopamine neurons as a reward prediction error signal that drives learning in basal ganglia and prefrontal circuits. However, recent findings suggest that dopaminergic prediction error signals have access to complex, structured reward predictions and are sensitive to more properties of outcomes than learning theories with simple scalar value predictions might suggest. Here, I will present recent work in which we probed the identity-specific structure of reward prediction errors in an odor-guided choice task and found evidence for multiple predictive “threads” that segregate reward predictions, and reward prediction errors, according to the specific sensory features of anticipated outcomes. Our results point to an expanded class of neural reinforcement learning algorithms in which biological agents learn rich associative structure from their environment and leverage it to build reward predictions that include information about the specific, and perhaps idiosyncratic, features of available outcomes, using these to guide behavior in even quite simple reward learning tasks.
Off-policy learning in the basal ganglia
I will discuss work with Jack Lindsey modeling reinforcement learning for action selection in the basal ganglia. I will argue that the presence of multiple brain regions, in addition to the basal ganglia, that contribute to motor control motivates the need for an off-policy basal ganglia learning algorithm. I will then describe a biological implementation of such an algorithm that predicts tuning of dopamine neurons to a quantity we call "action surprise," in addition to reward prediction error. In the same model, an implementation of learning from a motor efference copy also predicts a novel solution to the problem of multiplexing feedforward and efference-related striatal activity. The solution exploits the difference between D1 and D2-expressing medium spiny neurons and leads to predictions about striatal dynamics.
Basal Ganglia in addiction
My evolution in invasive human neurophysiology: From basal ganglia single units to chronic electrocorticography; Therapies orchestrated by patients' own rhythms
On Thursday, April 27th, we will host Hayriye Cagnan and Philip A. Starr. Hayriye Cagnan, PhD, is an associate professor at the MRC Brain Network Dynamics Unit and University of Oxford. She will tell us about “Therapies orchestrated by patients’ own rhythms”. Philip A. Starr, MD, PhD, is a neurosurgeon and professor of Neurological Surgery at the University of California San Francisco. Besides his scientific presentation on “My evolution in invasive human neurophysiology: from basal ganglia single units to chronic electrocorticography”, he will give us a glimpse at the person behind the science. The talks will be followed by a shared discussion. You can register via talks.stimulatingbrains.org to receive the (free) Zoom link!
Computations performed in the basal ganglia
Classification of Dopamine Cells
Beta oscillations in the basal ganglia: Past, Present and Future; Oscillatory signatures of motor symptoms across movement disorders
On Wednesday, January 25th, at noon ET / 6PM CET, we will host Roxanne Lofredi and Hagai Bergman. Roxanne Lofredi, MD, is a research fellow in the Movement Disorders and Neuromodulation Unit at Charité Universitätsmedizin Berlin. Hagai Bergman, MD, PhD, is a Professor of Physiology in the Edmond and Lily Safra Center for Brain Research and Faculty of Medicine at the Hebrew University of Jerusalem, and is Simone and Bernard Guttman Chair in Brain Research. Beside his scientific presentation on “Beta oscillations in the basal ganglia: Past, Present and Future”, he will also give us a glimpse at the “Person behind the science”. The talks will be followed by a shared discussion. You can register via talks.stimulatingbrains.org to receive the (free) Zoom link!
Sampling the environment with body-brain rhythms
Since Darwin, comparative research has shown that most animals share basic timing capacities, such as the ability to process temporal regularities and produce rhythmic behaviors. What seems to be more exclusive, however, are the capacities to generate temporal predictions and to display anticipatory behavior at salient time points. These abilities are associated with subcortical structures like basal ganglia (BG) and cerebellum (CE), which are more developed in humans as compared to nonhuman animals. In the first research line, we investigated the basic capacities to extract temporal regularities from the acoustic environment and produce temporal predictions. We did so by adopting a comparative and translational approach, thus making use of a unique EEG dataset including 2 macaque monkeys, 20 healthy young, 11 healthy old participants and 22 stroke patients, 11 with focal lesions in the BG and 11 in the CE. In the second research line, we holistically explore the functional relevance of body-brain physiological interactions in human behavior. Thus, a series of planned studies investigate the functional mechanisms by which body signals (e.g., respiratory and cardiac rhythms) interact with and modulate neurocognitive functions from rest and sleep states to action and perception. This project supports the effort towards individual profiling: are individuals’ timing capacities (e.g., rhythm perception and production), and general behavior (e.g., individual walking and speaking rates) influenced / shaped by body-brain interactions?
SWEBAGS conference 2022
SWEBAGS conference 2022
SWEBAGS conference 2022
November Webinar
Dopamine receptors dysregulation in BG disease
September webinar
PPN inputs to striatum
Dyskinesia: the failure of dopamine-dependent motor control
Learning in/about/from the basal ganglia
The basal ganglia are a collection of brain areas that are connected by a variety of synaptic pathways and are a site of significant reward-related dopamine release. These properties suggest a possible role for the basal ganglia in action selection, guided by reinforcement learning. In this talk, I will discuss a framework for how this function might be performed and computational results using an upward mapping to identify putative low-dimensional control ensembles that may be involved in tuning decision policy. I will also present some recent experimental results and theory – related to effects of extracellular ion dynamics -- that run counter to the classical view of basal ganglia pathways and suggest a new interpretation of certain aspects of this framework. For those not so interested in the basal ganglia, I hope that the upward mapping approach and impact of extracellular ion dynamics will nonetheless be of interest!
Visualising time in the human brain
We all have a sense of time. Yet it is a particularly intangible sensation. So how is our “sense” of time represented in the brain? Functional neuroimaging studies have consistently identified a network of regions, including Supplementary Motor Area and basal ganglia, that are activated when participants make judgements about the duration of currently unfolding events. In parallel, left parietal cortex and cerebellum are activated when participants predict when future events are likely to occur. These structures are activated by temporal processing even when task goals are purely perceptual. So why should the perception of time be represented in regions of the brain that have more traditionally been implicated in motor function? One possibility is that we learn about time through action. In other words, action could provide the functional scaffolding for learning about time in childhood, explaining why it has come to be represented in motor circuits of the adult brain.
GP and STN dynamics
March webinar
Basal ganglia activity dynamics in health and disease
Honorary Lecture 2022
Neurocognitive mechanisms of proactive temporal attention: challenging oscillatory and cortico-centered models
To survive in a rapidly dynamic world, the brain predicts the future state of the world and proactively adjusts perception, attention and action. A key to efficient interaction is to predict and prepare to not only “where” and “what” things will happen, but also to “when”. I will present studies in healthy and neurological populations that investigated the cognitive architecture and neural basis of temporal anticipation. First, influential ‘entrainment’ models suggest that anticipation in rhythmic contexts, e.g. music or biological motion, uniquely relies on alignment of attentional oscillations to external rhythms. Using computational modeling and EEG, I will show that cortical neural patterns previously associated with entrainment in fact overlap with interval timing mechanisms that are used in aperiodic contexts. Second, temporal prediction and attention have commonly been associated with cortical circuits. Studying neurological populations with subcortical degeneration, I will present data that point to a double dissociation between rhythm- and interval-based prediction in the cerebellum and basal ganglia, respectively, and will demonstrate a role for the cerebellum in attentional control of perceptual sensitivity in time. Finally, using EEG in neurodegenerative patients, I will demonstrate that the cerebellum controls temporal adjustment of cortico-striatal neural dynamics, and use computational modeling to identify cerebellar-controlled neural parameters. Altogether, these findings reveal functionally and neural context-specificity and subcortical contributions to temporal anticipation, revising our understanding of dynamic cognition.
Reinforcement Learning
Pathogenesis of Parkison's Disease
Network dynamics in the basal ganglia and possible implications for Parkinson’s disease
The basal ganglia are a collection of brain areas that are connected by a variety of synaptic pathways and are a site of significant reward-related dopamine release. These properties suggest a possible role for the basal ganglia in action selection, guided by reinforcement learning. In this talk, I will discuss a framework for how this function might be performed. I will also present some recent experimental results and theory that call for a re-evaluation of certain aspects of this framework. Next, I will turn to the changes in basal ganglia activity observed to occur with the dopamine depletion associated with Parkinson’s disease. I will discuss some of the potential functional implications of some of these changes and, if time permits, will conclude with some new results that focus on delta oscillations under dopamine depletion.
Computational Neuroscience
A role for dopamine in value-free learning
Recent success in training artificial agents and robots derives from a combination of direct learning of behavioral policies and indirect learning via value functions. Policy learning and value learning employ distinct algorithms that depend upon evaluation of errors in performance and reward prediction errors, respectively. In mammals, behavioral learning and the role of mesolimbic dopamine signaling have been extensively evaluated with respect to reward prediction errors; but there has been little consideration of how direct policy learning might inform our understanding. I’ll discuss our recent work on classical conditioning in naïve mice (https://www.biorxiv.org/content/10.1101/2021.05.31.446464v1) that provides multiple lines of evidence that phasic dopamine signaling regulates policy learning from performance errors in addition to its well-known roles in value learning. This work points towards new opportunities for unraveling the mechanisms of basal ganglia control over behavior under both adaptive and maladaptive learning conditions.
Striatal Circuitry
Dopamine
Basal ganglia anatomy
Computations in the basal ganglia
Dynamic reward signaling in ventral basal ganglia circuits
The Corticostriatal Pathway
The shared predictive roots of motor control and beat-based timing
fMRI results have shown that the supplementary motor area (SMA) and the basal ganglia, most often discussed in their roles in generating action, are engaged by beat-based timing even in the absence of movement. Some have argued that the motor system is “recruited” by beat-based timing tasks due to the presence of motor-like timescales, but a deeper understanding of the roles of these motor structures is lacking. Reviewing a body of motor neurophysiology literature and drawing on the “active inference” framework, I argue that we can see the motor and timing functions of these brain areas as examples of dynamic sub-second prediction informed by sensory event timing. I hypothesize that in both cases, sub-second dynamics in SMA predict the progress of a temporal process outside the brain, and direct pathway activation in basal ganglia selects temporal and sensory predictions for the upcoming interval -- the only difference is that in motor processes, these predictions are made manifest through motor effectors. If we can unify our understanding of beat-based timing and motor control, we can draw on the substantial motor neuroscience literature to make conceptual leaps forward in the study of predictive timing and musical rhythm.
What about antibiotics for the treatment of the dyskinesia induced by L-DOPA?
L-DOPA-induced dyskinesia is a debilitating adverse effect of treating Parkinson’s disease with this drug. New therapeutic approaches that prevent or attenuate this side effect is clearly needed. Wistar adult male rats submitted to 6-hydroxydopamine-induced unilateral medial forebrain bundle lesions were treated with L-DOPA (oral or subcutaneous, 20 mg kg-1) once a day for 14 days. After this period, we tested if doxycycline (40 mg kg-1, intraperitoneal, a subantimicrobial dose) and COL-3 (50 and 100 nmol, intracerebroventricular) could reverse LID. In an additional experiment, doxycycline was also administered repeatedly with L-DOPA to verify if it would prevent LID development. A single injection of doxycycline or COL-3 together with L-DOPA attenuated the dyskinesia. Co-treatment with doxycycline from the first day of L-DOPA suppressed the onset of dyskinesia. The improved motor responses to L-DOPA remained intact in the presence of doxycycline or COL-3, indicating the preservation of L-DOPA-produced benefits. Doxycycline treatment was associated with decreased immunoreactivity of FosB, cyclooxygenase-2, the astroglial protein GFAP and the microglial protein OX-42 which are elevated in the basal ganglia of rats exhibiting dyskinesia. Doxycycline also decreased metalloproteinase-2/-9 activity, metalloproteinase-3 expression and reactive oxygen species production. Metalloproteinase-2/-9 activity and production of reactive oxygen species in the basal ganglia of dyskinetic rats showed a significant correlation with the intensity of dyskinesia. The present study demonstrates the anti-dyskinetic potential of doxycycline and its analog compound COL-3 in hemiparkinsonian rats. Given the long-established and safe clinical use of doxycycline, this study suggests that these drugs might be tested to reduce or to prevent L-DOPA-induced dyskinesia in Parkinson’s patients.
Basal ganglia circuits underlying sensorimotor function
A journey through connectomics: from manual tracing to the first fully automated basal ganglia connectomes
The "mind of the worm", the first electron microscopy-based connectome of C. elegans, was an early sign of where connectomics is headed, followed by a long time of little progress in a field held back by the immense manual effort required for data acquisition and analysis. This changed over the last few years with several technological breakthroughs, which allowed increases in data set sizes by several orders of magnitude. Brain tissue can now be imaged in 3D up to a millimeter in size at nanometer resolution, revealing tissue features from synapses to the mitochondria of all contained cells. These breakthroughs in acquisition technology were paralleled by a revolution in deep-learning segmentation techniques, that equally reduced manual analysis times by several orders of magnitude, to the point where fully automated reconstructions are becoming useful. Taken together, this gives neuroscientists now access to the first wiring diagrams of thousands of automatically reconstructed neurons connected by millions of synapses, just one line of program code away. In this talk, I will cover these developments by describing the past few years' technological breakthroughs and discuss remaining challenges. Finally, I will show the potential of automated connectomics for neuroscience by demonstrating how hypotheses in reinforcement learning can now be tackled through virtual experiments in synaptic wiring diagrams of the songbird basal ganglia.
Human voluntary action: from thought to movement
The ability to decide and act autonomously is a distinctive feature of human cognition. From a motor neurophysiology viewpoint, these 'voluntary' actions can be distinguished by the lack of an obvious triggering sensory stimulus: the action is considered to be a product of thought, rather than a reflex result of a specific input. A reverse engineering approach shows that such actions are caused by neurons of the primary cortex, which in turn depend on medial frontal areas, and finally a combination of prefrontal cortical connections and subcortical drive from basal ganglia loops. One traditional marker of voluntary action is the EEG readiness potential (RP), recorded over the frontal cortex prior to voluntary actions. However, the interpretation of this signal remains controversial, and very few experimental studies have attempted to link the RP to the thought process that lead to voluntary action. In this talk, I will report new studies that show learning an internal model about the optimum delay at which to act influences the amplitude of the RP. More generally, a scientific understanding of voluntariness and autonomy will require new neurocognitive paradigms connecting thought and action.