“Development and application of gaze control models for active perception”

Gaze shifts in humans serve to direct high-resolution vision provided by the fovea towards areas in the environment. Gaze can be considered a proxy for attention or indicator of the relative importance of different parts of the environment. In this talk, we discuss the development of generative models of human gaze in response to visual input. We discuss how such models can be learned, both using supervised learning and using implicit feedback as an agent interacts with the environment, the latter being more plausible in biological agents. We also discuss two ways such models can be used. First, they can be used to improve the performance of artificial autonomous systems, in applications such as autonomous navigation. Second, because these models are contingent on the human’s task, goals, and/or state in the context of the environment, observations of gaze can be used to infer information about user intent. This information can be used to improve human-machine and human robot interaction, by making interfaces more anticipative. We discuss example applications in gaze-typing, robotic tele-operation and human-robot interaction.

Brain-Wide Compositionality and Learning Dynamics in Biological Agents

Biological agents continually reconcile the internal states of their brain circuits with incoming sensory and environmental evidence to evaluate when and how to act. The brains of biological agents, including animals and humans, exploit many evolutionary innovations, chiefly modularity—observable at the level of anatomically-defined brain regions, cortical layers, and cell types among others—that can be repurposed in a compositional manner to endow the animal with a highly flexible behavioral repertoire. Accordingly, their behaviors show their own modularity, yet such behavioral modules seldom correspond directly to traditional notions of modularity in brains. It remains unclear how to link neural and behavioral modularity in a compositional manner. We propose a comprehensive framework—compositional modes—to identify overarching compositionality spanning specialized submodules, such as brain regions. Our framework directly links the behavioral repertoire with distributed patterns of population activity, brain-wide, at multiple concurrent spatial and temporal scales. Using whole-brain recordings of zebrafish brains, we introduce an unsupervised pipeline based on neural network models, constrained by biological data, to reveal highly conserved compositional modes across individuals despite the naturalistic (spontaneous or task-independent) nature of their behaviors. These modes provided a scaffolding for other modes that account for the idiosyncratic behavior of each fish. We then demonstrate experimentally that compositional modes can be manipulated in a consistent manner by behavioral and pharmacological perturbations. Our results demonstrate that even natural behavior in different individuals can be decomposed and understood using a relatively small number of neurobehavioral modules—the compositional modes—and elucidate a compositional neural basis of behavior. This approach aligns with recent progress in understanding how reasoning capabilities and internal representational structures develop over the course of learning or training, offering insights into the modularity and flexibility in artificial and biological agents.

Richly structured reward predictions in dopaminergic learning circuits

Theories from reinforcement learning have been highly influential for interpreting neural activity in the biological circuits critical for animal and human learning. Central among these is the identification of phasic activity in dopamine neurons as a reward prediction error signal that drives learning in basal ganglia and prefrontal circuits. However, recent findings suggest that dopaminergic prediction error signals have access to complex, structured reward predictions and are sensitive to more properties of outcomes than learning theories with simple scalar value predictions might suggest. Here, I will present recent work in which we probed the identity-specific structure of reward prediction errors in an odor-guided choice task and found evidence for multiple predictive “threads” that segregate reward predictions, and reward prediction errors, according to the specific sensory features of anticipated outcomes. Our results point to an expanded class of neural reinforcement learning algorithms in which biological agents learn rich associative structure from their environment and leverage it to build reward predictions that include information about the specific, and perhaps idiosyncratic, features of available outcomes, using these to guide behavior in even quite simple reward learning tasks.

Fantastic windows of sensitivity and where to find them

Mapping the brain’s remaining terra incognita

In this webinar, Dr Ye Tian and A/Prof Andrew Zalesky will present new research on mapping the functional architecture of the human subcortex. They used 3T and 7T functional MRI from more than 1000 people to map one of the most detailed functional atlases of the human subcortex to date. Comprising four hierarchical scales, the new atlas reveals the complex topographic organisation of the subcortex, which dynamically adapts to changing cognitive demands. The atlas enables whole-brain mapping of connectomes and has been used to optimise targeting of deep brain stimulation. This joint work with Professors Michael Breakspear and Daniel Margulies was recently published in Nature Neuroscience. In the second part of the webinar, Dr Ye Tian will present her current research on the biological ageing of different body systems, including the human brain, in health and degenerative conditions. Conducted in more than 30,000 individuals, this research reveals associations between the biological ageing of different body systems. She will show the impact of lifestyle factors on ageing and how advanced ageing can predict the risk of mortality. Associate Professor Andrew Zalesky is a Principal Researcher with a joint appointment between the Faculties of Engineering and Medicine at The University of Melbourne. He currently holds a NHMRC Senior Research Fellowship and serves as Associate Editor for Brain Topography, Neuroimage Clinical and Network Neuroscience. Dr Zalesky is recognised for the novel tools that he has developed to analyse brain networks and their application to the study of neuropsychiatric disorders. Dr Ye Tian is a postdoctoral researcher at the Department of Psychiatry, University of Melbourne. She received her PhD from the University of Melbourne in 2020, during which she established the Melbourne Subcortex Atlas. Dr Tian is interested in understanding brain organisation and using brain imaging techniques to unveil neuropathology underpinning neuropsychiatric disorders.

Using Markov Decision Processes to benchmark the performance of artificial and biological agents

COSYNE 2022

Using Markov Decision Processes to benchmark the performance of artificial and biological agents

COSYNE 2022