Reinforcement Learning ePosters

Because the world is dynamic and only imperfectly observable, many of the decisions we make are necessarily uncertain. How do we navigate such uncertainty? From the perspective of cognitive neuroscience, the classic answer would be that we evaluate the benefits of each potential choice and then lean towards the one…

Juvenile male zebra finches learn to produce a single, highly stereotyped song and maintain this song over the course of their adult lives using auditory feedback. Much of what we know of the underlying learning process comes from studies of adult male zebra finches adapting to white noise feedback triggered by either…

Foraging, the set of behaviors associated with seeking valuable resources (e.g., food, water) while avoiding danger (e.g., predators), is ubiquitous among organisms, but its neural circuit basis is presently unclear. Most existing work on navigation, a key component of foraging, involves animals in small, fully…

During navigation, animals dynamically create rich representations of the environment, forming personalized cognitive maps. The hippocampal area CA1 features spatial cells that adapt based on behavior and internal states. Computational models have usually obtained spatial tuning by training a deep recurrent network…

In autonomous navigation, an agent's knowledge of its environment is represented as sensory inputs (states) linked together by actions. A popular approach for acquiring this knowledge is reinforcement learning (RL), where learning is stereotyped, i.e. an agent learns to react to a state with the action associated with…

By causally mapping neural activity to behavior [1], Brain-Computer Interfaces (BCI) offer a means to study the dynamics of sensorimotor learning. Here, we combine computational modeling and data analysis to study the neural learning algorithm [2] monkeys use to adapt to a changed output mapping in a center-out…

To survive in a changing world, animals often need to suppress a previously learnt, but now obsolete, behavior and acquire a new behavior. This process is known as reversal learning (RL). The neural mechanisms underlying RL in spatial navigation has received limited attention and it remains unclear what neural…

Dopaminergic Reward Prediction Errors (RPEs) are a key motivation and inspiration for model free, temporal difference reinforcement learning methods. Originally, the correlation of RPEs with model free temporal difference errors was seen as a strong indicator for model free reinforcement learning in brains. The…

Animals often make use of information from multiple sensory systems, such as vision, proprioception, olfaction, and audition to guide their behavior. They must thus integrate these different sensory inputs or select one of them in order to effectively make decisions in the environment. How exactly the brain…

There has been a long-lasting debate on whether the function of the hippocampal formation is to store and retrieve episodic memory, supported by experiments in humans, or to code for space, suggested by spatially tuned cells (place cells, grid cells, etc.) recorded in nonhuman animals. In an attempt to reconcile the…

The relationship between vergence and accommodation remains relatively unexplored. The frequent co-occurrence of non-strabismic accommodation and vergence disorders suggests a link between the two processes. Experimental data also reveal that vergence and accommodation have reciprocal effects, where their reflexes…

Central place foraging insects like the honeybee (Apis mellifera) are the masters of visual navigation of the insect world: They are able to reliably return to their nest under a wide range of visual conditions after only a few learning flights, (a feat so far unrivalled by any artificial system). Furthermore, they…

Error-related potentials (ErrPs) represent the neural signature of error processing in the brain and numerous studies have demonstrated their reliable detection using non-invasive techniques such as electroencephalography (EEG). Over recent decades, the brain-computer interface (BCI) community has shown growing…

Contemporary Bayesian theories of the mind highlight the role of prediction errors as a driving force in learning processes and the updating of our models of the world and of ourselves. This process is postulated to manifest both at explicit levels (e.g., subjective responses) and at implicit levels (e.g., eye…

Beliefs influence the intensity of felt pain. We previously applied regression models to two independent pain-cueing datasets (https://osf.io/5r6z7/) and found that pain intensity reports were a function of the error between participants’ belief, as determined by cues that informed participants about the level of…

How does the motor cortex (MC) achieve generalizable and purposeful movements from the complex non-linear musculoskeletal system? This understanding can help us uncover how does the cortical activity relates to the movement. To elucidate this relationship, we build a model of the MC with realistic properties. To mimic…

When an agent is trained on a complex episodic task, different task parts may be learned at different rates. How do we determine which part of the task challenged the agent the most? Since reward is provided usually only at the end of each trial, it cannot be used to infer within-trial learning trends. Behavioral…

The representation of “what happened when” is central to encoding episodic and working memories. Recently discovered hippocampal “time cells” are theorized to provide the neural substrate for such representations by forming distinct sequences that encode both time elapsed and sensory content. However, while multiple…

Saccades are fast eye movements that change gaze direction in a ballistic fashion. Human saccadic behaviour has been studied extensively in neuroscience and several methods have been proposed to model the saccadic system. For example, John et al. (2021) adopted a model-based approach, based on a physical model of the…

The successor representation (SR) is a promising candidate principle for hippocampal function. Theory proposes that each place cell encodes the expected state occupancy of its target location in the near future. This framework has desirable consequences on the generalisability and efficiency of reinforcement learning…

Consider a task where you're accumulating noisy evidence about two options, and the longer you collect evidence the more likely you are to choose the correct one. You get $1,000 for choosing the correct option and $900 for choosing the incorrect one. How long should you wait before making a decision? Tasks like this…

One-shot learning can be achieved by animals and algorithms, but how animals do it is poorly understood as most of the algorithms are not biologically plausible. Experiments studying one-shot learning in rodents have shown that after initial gradual learning of associations between cues and locations, new associations…

It is well established that midbrain dopaminergic neurons support reinforcement learning (RL) in the basal ganglia by transmitting a reward prediction error (RPE) to the striatum. In particular, different computational models and experiments have shown that a striatum-wide RPE signal can support RL over a small…

In reinforcement learning (RL), we try to learn a function f that maps a state representation s to actions a: a=f(s). Learning f is hard, and is typically performed in each environment. What if, instead, we learn how to rapidly construct s out of general building blocks, for which the same f always works? We…