Understanding reward-guided learning using large-scale datasets

Understanding the neural mechanisms of reward-guided learning is a long-standing goal of computational neuroscience. Recent methodological innovations enable us to collect ever larger neural and behavioral datasets. This presents opportunities to achieve greater understanding of learning in the brain at scale, as well as methodological challenges. In the first part of the talk, I will discuss our recent insights into the mechanisms by which zebra finch songbirds learn to sing. Dopamine has been long thought to guide reward-based trial-and-error learning by encoding reward prediction errors. However, it is unknown whether the learning of natural behaviours, such as developmental vocal learning, occurs through dopamine-based reinforcement. Longitudinal recordings of dopamine and bird songs reveal that dopamine activity is indeed consistent with encoding a reward prediction error during naturalistic learning. In the second part of the talk, I will talk about recent work we are doing at DeepMind to develop tools for automatically discovering interpretable models of behavior directly from animal choice data. Our method, dubbed CogFunSearch, uses LLMs within an evolutionary search process in order to "discover" novel models in the form of Python programs that excel at accurately predicting animal behavior during reward-guided learning. The discovered programs reveal novel patterns of learning and choice behavior that update our understanding of how the brain solves reinforcement learning problems.

Understanding reward-guided learning using large-scale datasets

Understanding the neural mechanisms of reward-guided learning is a long-standing goal of computational neuroscience. Recent methodological innovations enable us to collect ever larger neural and behavioral datasets. This presents opportunities to achieve greater understanding of learning in the brain at scale, as well as methodological challenges. In the first part of the talk, I will discuss our recent insights into the mechanisms by which zebra finch songbirds learn to sing. Dopamine has been long thought to guide reward-based trial-and-error learning by encoding reward prediction errors. However, it is unknown whether the learning of natural behaviours, such as developmental vocal learning, occurs through dopamine-based reinforcement. Longitudinal recordings of dopamine and bird songs reveal that dopamine activity is indeed consistent with encoding a reward prediction error during naturalistic learning. In the second part of the talk, I will talk about recent work we are doing at DeepMind to develop tools for automatically discovering interpretable models of behavior directly from animal choice data. Our method, dubbed CogFunSearch, uses LLMs within an evolutionary search process in order to "discover" novel models in the form of Python programs that excel at accurately predicting animal behavior during reward-guided learning. The discovered programs reveal novel patterns of learning and choice behavior that update our understanding of how the brain solves reinforcement learning problems.

The neural basis of exploration and decision-making in individuals and groups

On cognitive maps and reinforcement learning in large-scale animal behaviour

Bats are extreme aviators and amazing navigators. Many bat species nightly commute dozens of kilometres in search of food, and some bat species annually migrate over thousands of kilometres. Studying bats in their natural environment has always been extremely challenging because of their small size (mostly <50 gr) and agile nature. We have recently developed novel miniature technology allowing us to GPS-tag small bats, thus opening a new window to document their behaviour in the wild. We have used this technology to track fruit-bats pups over 5 months from birth to adulthood. Following the bats’ full movement history allowed us to show that they use novel short-cuts which are typical for cognitive-map based navigation. In a second study, we examined how nectar-feeding bats make foraging decisions under competition. We show that by relying on a simple reinforcement learning strategy, the bats can divide the resource between them without aggression or communication. Together, these results demonstrate the power of the large scale natural approach for studying animal behavior.

Linking dimensionality to computation in neural networks

The link between behavior, learning and the underlying connectome is a fundamental open problem in neuroscience. In my talk I will show how it is possible to develop a theory that bridges across these three levels (animal behavior, learning and network connectivity) based on the geometrical properties of neural activity. The central tool in my approach is the dimensionality of neural activity. I will link animal complex behavior to the geometry of neural representations, specifically their dimensionality; I will then show how learning shapes changes in such geometrical properties and how local connectivity properties can further regulate them. As a result, I will explain how the complexity of neural representations emerges from both behavioral demands (top-down approach) and learning or connectivity features (bottom-up approach). I will build these results regarding neural dynamics and representations starting from the analysis of neural recordings, by means of theoretical and computational tools that blend dynamical systems, artificial intelligence and statistical physics approaches.

On cognitive maps and reinforcement learning in large-scale animal behaviour

Bats are extreme aviators and amazing navigators. Many bat species nightly com-mute dozens of kilometres in search of food, and some bat species annually migrate over thousands of kilometres. Studying bats in their natural environment has al-ways been extremely challenging because of their small size (mostly <50 gr) and agile nature. We have recently developed novel miniature technology allowing us to GPS-tag small bats, thus opening a new window to document their behaviour in the wild. We have used this technology to track fruit-bats pups over 5 months from birth to adulthood. Following the bats’ full movement history allowed us to show that they use novel short-cuts which are typical for cognitive-map based naviga-tion. In a second study, we examined how nectar-feeding bats make foraging deci-sions under competition. We show that by relying on a simple reinforcement learn-ing strategy, the bats can divide the resource between them without aggression or communication. Together, these results demonstrate the power of the large scale natural approach for studying animal behavior.

Neuroscience in the mud: interplay between lab and field research for understanding animal behavior

Investigations of the neurophysiological processes underlying animal behaviors are almost exclusively done inside the laboratory, typically using few animal models born and reared under artificially stabilized conditions. Yet, animals living in the wild have to cope with much complex and variable environments. Thus, while the laboratory provides the technical possibilities for physiological research, the field offers a more realistic perspective about the animal´s behavioral abilities. We study neural circuits underlying the visually guided prey and predator behaviors in a semiterrestrial crab. By combining lab and field experiments we have, for example, found that the level of predation risk experienced by the animals in the wild affects the responsiveness of identified neurons involved in the animal escape response. Using this and other results from my lab I will illustrate and discuss the importance of complementing lab with field studies in wild animals for understanding the neural mechanisms subserving behavior.

Neuroscience tools for the 99%: On the low-fi development of high-tech lab gear for hands-on neuroscience labs and exploratory research

The public has a fascination with the brain, but little attention is given to neuroscience education prior to graduate studies in brain-related fields. One reason may be the lack of low cost and engaging teaching materials. To address this, we have developed a suite of open-source tools which are appropriate for amateurs and for use in high school, undergraduate, and graduate level educational and research programs. This lecture will provide an overview of our mission to re-engineer research-grade lab equipment using first principles and will highlight basic principles of neuroscience in a "DIY" fashion: neurophysiology, functional electrical stimulation, micro-stimulation effect on animal behavior, neuropharmacology, even neuroprosthesis and optogenetics! Finally, with faculty academic positions becoming a scarce resource, I will discuss an alternative academic career path: entrepreneurship. It is possible to be an academic, do research, publish papers, present at conferences and train students all outside the traditional university setting. I will close by discussing my career path from graduate student to PI/CEO of a startup neuroscience company.