Topic: Reward-learning

Humans and other animals approach reward and avoid punishment and pay attention to cues predicting these events. Such motivated behavior thus appears to be guided by value, which directs behavior towards or away from positively or negatively valenced outcomes. Moreover, it is facilitated by (top-down) salience, which enhances attention to behaviorally relevant learned cues predicting the occurrence of valenced outcomes. Using human neuroimaging, we recently separated value (ventral striatum, posterior ventromedial prefrontal cortex) from salience (anterior ventromedial cortex, occipital cortex

Humans and animals are not always rational. They not only rationally exploit rewards but also explore an environment owing to their curiosity. However, the mechanism of such curiosity-driven irrational behavior is largely unknown. Here, we developed a decision-making model for a two-choice task based on the free energy principle, which is a theory integrating recognition and action selection. The model describes irrational behaviors depending on the curiosity level. We also proposed a machine learning method to decode temporal curiosity from behavioral data. By applying it to rat behavioral da

The regulation of body weight relies on homeostatic mechanisms that use a combination of internal signals and external cues to initiate and terminate food intake. Homeostasis depends on intricate communication between the body and the hypothalamus involving numerous neural and hormonal signals. However, there is growing evidence that higher-level cognitive function may also influence energy balance. For instance, research has shown that BMI is consistently linked to various brain, cognitive, and personality measures, implicating executive, reward, and attentional systems. Moreover, the rise in

Gaining insight into the overlapping neural circuits that regulate positive and negative emotion is an important step towards understanding the heterogeneity in the aetiology of anxiety and depression and developing new treatment targets. Determining the core contributions of the functionally heterogenous prefrontal cortex to these circuits is especially illuminating given its marked dysregulation in affective disorders. This presentation will review a series of studies in a new world monkey, the common marmoset, employing pathway-specific chemogenetics, neuroimaging, neuropharmacology and beh

While it is generally accepted that neurons control complex behavior and brain computation, the role of non-neuronal cells in this context remains unclear. Astrocytes, glial cells of the central nervous system, exhibit complex forms of chemical excitation, most prominently calcium transients, evoked by local and projection neuron activity. In this talk, I will provide mechanistic links between astrocytes’ spatiotemporally complex activity patterns, neuronal molecular signaling, and behavior. Using a visual detection task, in vivo calcium imaging, robust statistical analyses, and machine learni

Visual perceptual learning (VPL) is defined as improved performance on a visual task due to visual experience. It was once argued that attention to a visual feature is necessary for VPL of the feature to occur. Contrary to this view, a phenomenon called task-irrelevant VPL demonstrated that VPL can occur due to exposure to a feature which is sub-threshold and task-irrelevant, and therefore, unattended. A series of findings based on task-irrelevant VPL has indicated the following two mechanisms. First, attention to a feature facilitates VPL of the feature while inhibiting VPL of unattended and

The role of attention in working memory remains controversial, but there is some agreement on the notion that the focus of attention holds mnemonic representations in a privileged state of heightened accessibility in working memory, resulting in better memory performance for items that receive focused attention during retention. Closely related, representations held in the focus of attention are often observed to be robust and protected from degradation caused by either perceptual interference (e.g., Makovski & Jiang, 2007; van Moorselaar et al., 2015) or decay (e.g., Barrouillet et al., 2007)

Human society operates on large-scale cooperation and shared norms of fairness. However, individual differences in cooperation and incentives to free-riding on others’ cooperation make large-scale cooperation fragile and can lead to reduced social-welfare. Deciphering the neural codes representing potential rewards/costs for self and others is crucial for understanding social decision-making and cooperation. I will first talk about how we integrate computational modeling with functional magnetic resonance imaging to investigate the neural representation of social value and the modulation by ox

Transient changes in dopamine activity in response to reward and punishment have been known to regulate reward-related learning. However, the cellular basis that detects the transient dopamine signaling has long been unclear. Using two-photon microscopy and optogenetics, I have shown that transient increases and decreases of dopamine modulate plasticity of dopamine D1 and D2 receptor-expressing cells in the nucleus accumbens, respectively. At the behavioral level, I characterized that these D1 and D2 cells cooperatively tune learning by generalization and discrimination learning. Interestingly

Recent years have seen a considerable surge of research on interest-based engagement, examining how and why people are engaged in activities without relying on extrinsic rewards. However, the field of inquiry has been somewhat segregated into three different research traditions which have been developed relatively independently --- research on curiosity, interest, and trait curiosity/interest. The current talk sets out an integrative perspective; the reward-learning framework of knowledge acquisition. This conceptual framework takes on the basic premise of existing reward-learning models of in

Recent years have seen a considerable surge of research on interest-based engagement, examining how and why people are engaged in activities without relying on extrinsic rewards. However, the field of inquiry has been somewhat segregated into three different research traditions which have been developed relatively independently -- research on curiosity, interest, and trait curiosity/interest. The current talk sets out an integrative perspective; the reward-learning framework of knowledge acquisition. This conceptual framework takes on the basic premise of existing reward-learning models of inf

Drosophila can learn to associate odours with reward or punishment and the resulting memories direct odour-specific approach or avoidance behaviours. Recent progress has revealed a straightforward model for learning in which reinforcing dopaminergic neurons assign valence to odour representations in the neural ensemble of the mushroom bodies. Dopamine directed synaptic depression alters the route of odour-driven activity through the mushroom body output network. This circuit configuration and influence of internal state guide the expression of appropriate behaviour. Importantly, learned behavi

Much of our daily behavior is driven by rewards. The ability to learn to pursue rewarding experiences is, in fact, an essential metric of mental health. Conversely, reduced capacity to engage in adaptive goal-oriented behavior is the hallmark of apathy, and present in the psychotic disorder. The search for its underlying mechanisms has resulted in findings of profound impairments in learning from rewards and the associated blunted activation in key reward areas of the brain of patients with psychosis. An emerging research field has been relying on digital phenotyping tools and ecological momen

While meta-learning algorithms are often viewed as algorithms that learn to learn, an alternative viewpoint frames meta-learning as inferring a hidden task variable from experience consisting of observations and rewards. From this perspective, learning to learn is learning to infer. This viewpoint can be useful in solving problems in meta-RL, which I’ll demonstrate through two examples: (1) enabling off-policy meta-learning, and (2) performing efficient meta-RL from image observations. I’ll also discuss how this perspective leads to an algorithm for few-shot image segmentation.

The neurotransmitter dopamine is essential for normal reward learning and motivational arousal processes. Indeed these core functions are implicated in the major neurological and psychiatric dopamine disorders such as schizophrenia, substance abuse disorders/addiction and Parkinson's disease. Over the years, we have made significant strides in understanding the dopamine system across multiple levels of description, and I will focus on our recent advances in the computational description, and brain circuit mechanisms that facilitate the dual role of dopamine in learning and performance. I will

The actions of dopamine within the striatum are central to the selection of cortical and perhaps thalamic inputs that mediate learning throughout life, including during operant conditioning, reward and avoidance learning and the establishment of motor patterns. Dysfunction of these synaptic circuits during maturation or aging underlies many neurological, psychiatric and neurodevelopment disorders. We will discuss the biological sequences by which these synapses are altered as an animal interacts with the environment.

Perceptual learning (PL) is defined as long-term performance improvement on a perceptual task as a result of perceptual experience (Sasaki, Nanez& Watanabe, 2011, Nat Rev Neurosci, 2011). We first found that PL occurs for task-irrelevant and subthreshold features and that pairing task-irrelevant features with rewards is the key to form task-irrelevant PL (TIPL) (Watanabe, Nanez & Sasaki, Nature, 2001; Watanabe et al, 2002, Nature Neuroscience; Seitz & Watanabe, Nature, 2003; Seitz, Kim & Watanabe, 2009, Neuron; Shibata et al, 2011, Science). These results suggest that PL occurs as a result of

Learning the consequences our choices have as we interact with our world is critical for flexible behavior. Relational knowledge of one’s environment gives structure to otherwise-individual one-to-one stimulus-outcome mappings, providing a substrate to globally update behavioral contingencies in the face of changes in the landscape of reward. In the brain, this relational knowledge is thought to be encoded in the hippocampus (HPC) in the form of a cognitive map, while prefrontal regions, such as orbitofrontal cortex (OFC), are thought to instantiate subjective estimates of location on the map,

Impulsivity and compulsivity are behavioural traits that underlie many aspects of decision-making and form the characteristic symptoms of Obsessive Compulsive Disorder (OCD) and Gambling Disorder (GD). The neural underpinnings of aspects of reward and avoidance learning under the expression of these traits and symptoms are only partially understood. " "The present study combined behavioural modelling and neuroimaging technique to examine brain activity associated with critical phases of reward and loss processing in OCD and GD. " "Forty-two healthy controls (HC), forty OCD and twenty-three GD

How are actions linked with subsequent outcomes to guide choices? The nucleus accumbens (NAc), which is implicated in this process, receives glutamatergic inputs from the prelimbic cortex (PL) and midline regions of the thalamus (mTH). However, little is known about what is represented in PL or mTH neurons that project to NAc (PL-NAc and mTH-NAc). By comparing these inputs during a reinforcement learning task in mice, we discovered that i) PL-NAc preferentially represents actions and choices, ii) mTH-NAc preferentially represents cues, iii) choice-selective activity in PL-NAc is organized in s