The National Institute on Drug Abuse (NIDA) is seeking a GS-15 Data Scientist for the Office of the Director. The incumbent will serve as a Data Scientist and Senior Advisor to the NIDA Director and other leadership positions, on matters related to health informatics and data science, data management, emerging technologies, opportunities for collaboration, data standards and policy within the scope of the NIDA mission. Responsibilities include planning and carrying out quality control programs, selecting statistical methods for quality control analysis, ensuring the reliability and consistency of data, providing technical expertise and project leadership to advance strategic initiatives in data- and knowledge-driven methods, and developing collaborations with researchers and administrators in neuroscience.

This webinar addressed computational perspectives on how animals and humans make decisions, spanning normative, descriptive, and mechanistic models. Sam Gershman (Harvard) presented a capacity-limited reinforcement learning framework in which policies are compressed under an information bottleneck constraint. This approach predicts pervasive perseveration, stimulus‐independent “default” actions, and trade-offs between complexity and reward. Such policy compression reconciles observed action stochasticity and response time patterns with an optimal balance between learning capacity and performance. Jonathan Pillow (Princeton) discussed flexible descriptive models for tracking time-varying policies in animals. He introduced dynamic Generalized Linear Models (Sidetrack) and hidden Markov models (GLM-HMMs) that capture day-to-day and trial-to-trial fluctuations in choice behavior, including abrupt switches between “engaged” and “disengaged” states. These models provide new insights into how animals’ strategies evolve under learning. Finally, Kenji Doya (OIST) highlighted the importance of unifying reinforcement learning with Bayesian inference, exploring how cortical-basal ganglia networks might implement model-based and model-free strategies. He also described Japan’s Brain/MINDS 2.0 and Digital Brain initiatives, aiming to integrate multimodal data and computational principles into cohesive “digital brains.”

This seminar, hosted by the LIBRE hub project, will provide an in-depth introduction to the Global Open Science Hardware (GOSH) movement. Since its inception, GOSH has been instrumental in advancing open-source hardware within scientific research, fostering a diverse and active community. The seminar will cover the history of GOSH, its current initiatives, and future opportunities, with a particular focus on the contributions and activities of the Latin American branch. This session aims to inform researchers, educators, and policy-makers about the significance and impact of GOSH in promoting accessibility and collaboration in science instrumentation.

When interacting with complex environments, humans can rapidly adapt their behavior to changes in task or context. To facilitate this adaptation, we often spend substantial periods of time contemplating possible futures before acting. For such planning to be rational, the benefits of planning to future behavior must at least compensate for the time spent thinking. Here we capture these features of human behavior by developing a neural network model where not only actions, but also planning, are controlled by prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences drawn from its own policy, which we refer to as `rollouts'. Our results demonstrate that this agent learns to plan when planning is beneficial, explaining the empirical variability in human thinking times. Additionally, the patterns of policy rollouts employed by the artificial agent closely resemble patterns of rodent hippocampal replays recently recorded in a spatial navigation task, in terms of both their spatial statistics and their relationship to subsequent behavior. Our work provides a new theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by -- and in turn adaptively affect -- prefrontal dynamics.

When interacting with complex environments, humans can rapidly adapt their behavior to changes in task or context. To facilitate this adaptation, we often spend substantial periods of time contemplating possible futures before acting. For such planning to be rational, the benefits of planning to future behavior must at least compensate for the time spent thinking. Here we capture these features of human behavior by developing a neural network model where not only actions, but also planning, are controlled by prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences drawn from its own policy, which we refer to as 'rollouts'. Our results demonstrate that this agent learns to plan when planning is beneficial, explaining the empirical variability in human thinking times. Additionally, the patterns of policy rollouts employed by the artificial agent closely resemble patterns of rodent hippocampal replays recently recorded in a spatial navigation task, in terms of both their spatial statistics and their relationship to subsequent behavior. Our work provides a new theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by - and in turn adaptively affect - prefrontal dynamics.

Studies show that abuse, neglect or trauma during childhood can lead to lifelong struggles including with mental health. Fortunately research also indicates that solutions and interventions at various stages of life can be developed to help. But even among people who remain resilient or do not experience acute stresses, a lack of opportunity early in life due to poverty or systemic racism can still constrain their ability to realize their full potential. In what ways are health and other outcomes affected by early life difficulty? What can individuals and institutions do to enhance opportunity?" "This daylong event will feature talks by neuroscientists, policy experts, physicians, educators and activists as they discuss how our experiences and biology work together to affect how our minds develop and what can be accomplished in helping people overcome early disadvantages.

I will discuss work with Jack Lindsey modeling reinforcement learning for action selection in the basal ganglia. I will argue that the presence of multiple brain regions, in addition to the basal ganglia, that contribute to motor control motivates the need for an off-policy basal ganglia learning algorithm. I will then describe a biological implementation of such an algorithm that predicts tuning of dopamine neurons to a quantity we call "action surprise," in addition to reward prediction error. In the same model, an implementation of learning from a motor efference copy also predicts a novel solution to the problem of multiplexing feedforward and efference-related striatal activity. The solution exploits the difference between D1 and D2-expressing medium spiny neurons and leads to predictions about striatal dynamics.

Humans perceive the world in terms of objects and relations between them. In fact, for any given pair of objects, there is a myriad of relations that apply to them. How does the cognitive system learn which relations are useful to characterize the task at hand? And how can it use these representations to build a relational policy to interact effectively with the environment? In this paper we propose that this problem can be understood through the lens of a sub-field of symbolic machine learning called relational reinforcement learning (RRL). To demonstrate the potential of our approach, we build a simple model of relational policy learning based on a function approximator developed in RRL. We trained and tested our model in three Atari games that required to consider an increasingly number of potential relations: Breakout, Pong and Demon Attack. In each game, our model was able to select adequate relational representations and build a relational policy incrementally. We discuss the relationship between our model with models of relational and analogical reasoning, as well as its limitations and future directions of research.

New tasks are often similar in structure to old ones. Animals that take advantage of such conserved or “abstract” task structures can master new tasks with minimal training. To understand the neural basis of this abstraction, we developed a novel behavioural paradigm for mice: the “ABCD” task, and recorded from their medial frontal neurons as they learned. Animals learned multiple tasks where they had to visit 4 rewarded locations on a spatial maze in sequence, which defined a sequence of four “task states” (ABCD). Tasks shared the same circular transition structure (… ABCDABCD …) but differed in the spatial arrangement of rewards. As well as improving across tasks, mice inferred that A followed D (i.e. completed the loop) on the very first trial of a new task. This “zero-shot inference” is only possible if animals had learned the abstract structure of the task. Across tasks, individual medial Frontal Cortex (mFC) neurons maintained their tuning to the phase of an animal’s trajectory between rewards but not their tuning to task states, even in the absence of spatial tuning. Intriguingly, groups of mFC neurons formed modules of coherently remapping neurons that maintained their tuning relationships across tasks. Such tuning relationships were expressed as replay/preplay during sleep, consistent with an internal organisation of activity into multiple, task-matched ring attractors. Remarkably, these modules were anchored to spatial locations: neurons were tuned to specific task space “distances” from a particular spatial location. These newly discovered “Spatially Anchored Task clocks” (SATs), suggest a novel algorithm for solving abstraction tasks. Using computational modelling, we show that SATs can perform zero-shot inference on new tasks in the absence of plasticity and guide optimal policy in the absence of continual planning. These findings provide novel insights into the Frontal mechanisms mediating abstraction and flexible behaviour.

SES neuroscience, using imaging and other methods, has revealed generalizations of interest for population neuroscience and the study of individual differences. But beyond its scientific interest, SES is a topic of societal importance. Does neuroscience offer any useful insights for promoting socioeconomic justice and reducing the harms of poverty? In this talk I will use research from my own lab and others’ to argue that SES neuroscience has the potential to contribute to policy in this area, although its application is premature at present. I will also attempt to forecast the ways in which practical solutions to the problems of poverty may emerge from SES neuroscience. Bio: Martha Farah has conducted groundbreaking research on face and object recognition, visual attention, mental imagery, and semantic memory and - in more recent times - has been at the forefront of interdisciplinary research into neuroscience and society. This deals with topics such as using fMRI for lie detection, ethics of cognitive enhancement, and effects of social deprivation on brain development.

The basal ganglia are a collection of brain areas that are connected by a variety of synaptic pathways and are a site of significant reward-related dopamine release. These properties suggest a possible role for the basal ganglia in action selection, guided by reinforcement learning. In this talk, I will discuss a framework for how this function might be performed and computational results using an upward mapping to identify putative low-dimensional control ensembles that may be involved in tuning decision policy. I will also present some recent experimental results and theory – related to effects of extracellular ion dynamics -- that run counter to the classical view of basal ganglia pathways and suggest a new interpretation of certain aspects of this framework. For those not so interested in the basal ganglia, I hope that the upward mapping approach and impact of extracellular ion dynamics will nonetheless be of interest!

Existing studies show that there is a positive association between pro-migrant integration policies and the subjective well-being of immigrants. However, there is a lack of research elucidating the relations between migrant integration policies and the subjective well-being of the host (i.e., non-migrant) population. This study is based on European data and uses multilevel analysis to clarify the relations between migrant integration policy (both as a whole and its 8 separate components such as: Labour market mobility and Family reunion) and the subjective well-being of the non-immigrant population in European countries. We examined relations between the Migrant Integration Policy Index (MIPEX) for 22 countries in Europe and subjective well-being, as assessed by the European Social Survey (ESS) data. The results demonstrated that there is a positive relation between the MIPEX and subjective well-being for non-immigrants. Considering different components of the MIPEX separately, we found most of them being positively related to the subjective well-being of non-immigrants. As no negative relationship was identified between any of the eight MIPEX components and subjective well-being, policies in favour of immigrant integration also seem to benefit the non-immigrant population.

In the deep reinforcement learning (RL) community, motor control problems are usually approached from a reward-based learning perspective. However, humans are often believed to learn motor control through directed error-based learning. Within this learning setting, the control system is assumed to have access to exact error signals and their gradients with respect to the control signal. This is unlike reward-based learning, in which errors are assumed to be unsigned, encoding relative successes and failures. Here, we try to understand the relation between these two approaches, reward- and error- based learning, and ballistic arm reaches. To do so, we test canonical (deep) RL algorithms on a well-known sensorimotor perturbation in neuroscience: mirror-reversal of visual feedback during arm reaching. This test leads us to propose a potentially novel RL algorithm, denoted as model-based deterministic policy gradient (MB-DPG). This RL algorithm draws inspiration from error-based learning to qualitatively reproduce human reaching performance under mirror-reversal. Next, we show MB-DPG outperforms the other canonical (deep) RL algorithms on a single- and a multi- target ballistic reaching task, based on a biomechanical model of the human arm. Finally, we propose MB-DPG may provide an efficient computational framework to help explain error-based learning in neuroscience.

COVID-19 non-pharmaceutical intervention (NPI) policies have been one of the most important and contentious decisions of our time. Beyond even the "normal" inherent difficulties in impact evaluation with observational data, COVID-19 NPI policy evaluation is complicated by additional challenges related to infectious disease dynamics and lags, lack of direct observation of key outcomes, and a multiplicity of interventions occurring on an accelerated time scale. Randomized controlled trials also suffer from what is feasible and ethical to randomize as well as the sheer scale, scope, time, and resources required for an NPI trial to be informative (or at least not misinformative). In this talk, Dr. Haber will discuss the challenges in generating useful evidence for COVID-19 NPIs, the landscape of the literature, and highlight key controversies in several high profile studies over the course of the pandemic. Chasing after unknowables poses major problems for the metascience/replicability movement, institutional research science, and decision makers. If the only choices for informing an important topic are "weak study design" vs "do nothing," when is "do nothing" the best choice?

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioral policies and indirect learning via value functions. Policy learning and value learning employ distinct algorithms that depend upon evaluation of errors in performance and reward prediction errors, respectively. In mammals, behavioral learning and the role of mesolimbic dopamine signaling have been extensively evaluated with respect to reward prediction errors; but there has been little consideration of how direct policy learning might inform our understanding. I’ll discuss our recent work on classical conditioning in naïve mice (https://www.biorxiv.org/content/10.1101/2021.05.31.446464v1) that provides multiple lines of evidence that phasic dopamine signaling regulates policy learning from performance errors in addition to its well-known roles in value learning. This work points towards new opportunities for unraveling the mechanisms of basal ganglia control over behavior under both adaptive and maladaptive learning conditions.

A central issue in reinforcement learning (RL) is the ‘curse-of-dimensionality’, arising when the degrees-of-freedom are much larger than the number of training samples. In such circumstances, the learning process becomes too slow to be plausible. In the brain, higher cognitive functions (such as abstraction or metacognition) may be part of the solution by generating low dimensional representations on which RL can operate. In this talk I will discuss a series of studies in which we used functional magnetic resonance imaging (fMRI) and computational modeling to investigate the neuro-computational basis of efficient RL. We found that people can learn remarkably complex task structures non-consciously, but also that - intriguingly - metacognition appears tightly coupled to this learning ability. Furthermore, when people use an explicit (conscious) policy to select relevant information, learning is accelerated by abstractions. At the neural level, prefrontal cortex subregions are differentially involved in separate aspects of learning: dorsolateral prefrontal cortex pairs with metacognitive processes, while ventromedial prefrontal cortex with valuation and abstraction. I will discuss the implications of these findings, in particular new questions on the function of metacognition in adaptive behavior and the link with abstraction.

Many flexible behaviors are thought to rely on internal representations of an animal’s spatial relationship to its environment and of the consequences of its actions in that environment. While such representations—e.g. of head direction and value—have been extensively studied, how they are combined to guide behavior is not well understood. I will discuss how we are exploring these questions using a classical visual learning paradigm for the fly. I’ll begin by describing a simple policy that, when tethered to an internal representation of heading, captures structured behavioral variability in this task. I’ll describe how ambiguities in the fly’s visual surroundings affect its perception and, when coupled to this policy, manifest in predictable changes in behavior. Informed by newly-released connectomic data, I’ll then discuss how these computations might be carried out and combined within specific circuits in the fly’s central brain, and how perception and action might interact to shape individual differences in learning performance.

Though studies show that abuse, neglect or trauma during childhood can lead to lifelong lifelong struggles including in mental health, research also indicates that solutions and interventions at various stages of life can be developed to help. And while many people manage to remain resilient, a lack of opportunity early in life, including because of poverty and systemic racism, can constrain their ability to realize their full potential. In what ways are health and other outcomes affected? How can systems instead restore opportunity? "The Picower Institute for Learning and Memory's biennial spring symposium, 'Early Life Stress & Mental Health,' will examine these issues. The daylong event will feature talks by neuroscientists, policy experts, physicians, educators and activists as they discuss how our experiences and biology work together to affect how our minds develop and what can be accomplished in helping people overcome early disadvantages.

This panel discussion aims to generate meaningful dialogue between emerging leaders in basic and clinical neuroscience. It promises to talk about the ground realities and what acts as a hindrance in people to people connection in the field. It aims to advocate for policy change that will revolutionize the field of neuroscience, allowing neuroscientists to collaborate with clinicians wherein the new research can be made available for public use

Mental Health problems among youth constitutes an area of significant social, educational, clinical, policy and public health concern. Understanding processes and mechanisms that underlie the development of mental health problems during childhood and adolescence requires theoretical and methodological integration across multiple scientific domains, including developmental science, neuroscience, genetics, education and prevention science. The primary focus of this presentation is to examine the relative role of genetic and family environmental influences on children’s emotional and behavioural development. Specifically, a complementary array of genetically sensitive and longitudinal research designs will be employed to examine the role of early environmental adversity (e.g. inter-parental conflict, negative parenting practices) relative to inherited factors in accounting for individual differences in children’s symptoms of psychopathology (e.g. depression, aggression, ADHD ). Examples of recent applications of this research to the development of evidence-based intervention programmes aimed at reducing psychopathology in the context of high-risk family settings will also be presented.

Concerns about the impact of social media use on adolescent well-being and mental health are common. While the amount of research in this area has increased rapidly over the last 5 years, most outputs are still marred by a multitude of limitations. These shortcomings have left our understanding of social media effects severely limited, holding back both scientific discovery and policy interventions. This talk discusses how developmental, cognitive and neuroscientific approaches might provide a new and improved way of studying social media effects. It will detail new studies in support of this idea, and raise potential avenues for collaborative work across the Cambridge Neuroscience community. As the digital world now (re)shapes what it means for us to live, communicate and develop, only an interdisciplinary approach will allow us to truly understand its impacts.

While meta-learning algorithms are often viewed as algorithms that learn to learn, an alternative viewpoint frames meta-learning as inferring a hidden task variable from experience consisting of observations and rewards. From this perspective, learning to learn is learning to infer. This viewpoint can be useful in solving problems in meta-RL, which I’ll demonstrate through two examples: (1) enabling off-policy meta-learning, and (2) performing efficient meta-RL from image observations. I’ll also discuss how this perspective leads to an algorithm for few-shot image segmentation.

This presentation will include a brief resume of research in older populations led from Cambridge that have informed current clinical understanding and policy regarding services and prevention for and of dementia. These population studies have more recently been ‘re-purposed’ with enthusiasm from participants into a trial platform, and this also has enabled ongoing follow-up by telephone during the COVID pandemic. Although there are no formal outputs from these latter developments general impressions will be shared.

Understanding the function and dysfunction of the brain remains one of the key challenges of our time. However, an overwhelming majority of brain research is carried out in the Global North, by a minority of well-funded and intimately interconnected labs. In contrast, with an estimated one neuroscientist per million people in Africa, news about neuroscience research from the Global South remains sparse. Clearly, devising new policies to boost Africa’s neuroscience landscape is imperative. However, the policy must be based on accurate data, which is largely lacking. Such data must reflect the extreme heterogeneity of research outputs across the continent’s 54 countries. We have analysed all of Africa’s Neuroscience output over the past 21 years and uniquely verified the work performed in African laboratories. Our unique dataset allows us to gain accurate and in-depth information on the current state of African Neuroscience research, and to put it into a global context. The key findings from this work and recommendations on how African research might best be supported in the future will be discussed.

Model-based approaches to control and decision making have long held the promise of being more powerful and data efficient than model-free counterparts. However, success with model-based methods has been limited to those cases where a perfect model can be queried. The game of Go was mastered by AlphaGo using a combination of neural networks and the MCTS planning algorithm. But planning required a perfect representation of the game rules. I will describe new algorithms that instead leverage deep neural networks to learn models of the environment which are then used to plan, and update policy and value functions. These new algorithms offer hints about how brains might approach planning and acting in complex environments.