Brain Imaging seminars

The cognitive neuroscience of conscious perception has seen considerable growth over the past few decades. Confirming an influential hypothesis driven by earlier studies of neuropsychological patients, we have found that the lateral and polar prefrontal cortices play important causal roles in the generation of subjective experiences. However, this basic empirical finding has been hotly contested by researchers with different theoretical commitments, and the differences are at times difficult to resolve. To address the controversies, I suggest one alternative venue may be to look for clinical applications derived from current theories. I outline an example in which we used closed-loop fMRI combined with machine learning to nonconsciously manipulate the physiological responses to threatening stimuli, such as spiders or snakes. A clinical trial involving patients with phobia is currently taking place. I also outline how this theoretical framework may be extended to other diseases. Ultimately, a truly meaningful understanding of the fundamental nature of our mental existence should lead to useful insights for our colleagues on the clinical frontlines. If we use this as a yardstick, whoever loses the esoteric theoretical debates, both science and the patients will always win.

Understanding speech in noisy backgrounds requires selective attention to a particular speaker. Humans excel at this challenging task, while current speech recognition technology still struggles when background noise is loud. The neural mechanisms by which we process speech remain, however, poorly understood, not least due to the complexity of natural speech. Here we describe recent progress obtained through applying machine-learning to neuroimaging data of humans listening to speech in different types of background noise. In particular, we develop statistical models to relate characteristic features of speech such as pitch, amplitude fluctuations and linguistic surprisal to neural measurements. We find neural correlates of speech processing both at the subcortical level, related to the pitch, as well as at the cortical level, related to amplitude fluctuations and linguistic structures. We also show that some of these measures allow to diagnose disorders of consciousness. Our findings may be applied in smart hearing aids that automatically adjust speech processing to assist a user, as well as in the diagnosis of brain disorders.

Over 60 years of research has established that medial temporal lobe structures, including the hippocampus and entorhinal cortex, are necessary for the formation of episodic memories (i.e. memories of specific personal events that occur in spatial and temporal context). While prior work to establish the neural mechanisms underlying episodic memory has largely focused on questions related spatial context, recently we have begun to investigate how these brain structures could be involved in encoding aspects of temporal context. In particular, we have focused on how medial entorhinal cortex, a structure well known for its role in spatial memory, may also be involved in encoding interval time. To answer this question we have developed an instrumental paradigm for head-fixed mice that requires both immobile interval timing and locomotion-dependent navigation behavior. By combining this behavioral paradigm with large-scale cellular resolution functional imaging and optogenetic-mediated inactivation, our results suggest that MEC is required for learning of interval timing behavior and that interval timing could be mediated through regular, sequential neural activity of a distinct subpopulation of neurons in MEC that encode elapsed time during periods of immobility (Heys and Dombeck, 2018; Heys et al, 2020; Issa et al., 2020). In this talk, I will discuss these findings and discuss our on-going work to investigate the principles underlying the role of medial temporal lobe structures in timing behavior and episodic memory.

In recent years, artificial neural networks have demonstrated human-like or super-human performance in many tasks including image or speech recognition, natural language processing (NLP), playing Go, chess, poker and video-games. One remarkable feature of the resulting models is that they can develop very intuitive latent representations of their inputs. In these latent spaces, simple linear operations tend to give meaningful results, as in the well-known analogy QUEEN-WOMAN+MAN=KING. We postulate that human brain representations share essential properties with these deep learning latent spaces. To verify this, we test whether artificial latent spaces can serve as a good model for decoding brain activity. We report improvements over state-of-the-art performance for reconstructing seen and imagined face images from fMRI brain activation patterns, using the latent space of a GAN (Generative Adversarial Network) model coupled with a Variational AutoEncoder (VAE). With another GAN model (BigBiGAN), we can decode and reconstruct natural scenes of any category from the corresponding brain activity. Our results suggest that deep learning can produce high-level representations approaching those found in the human brain. Finally, I will discuss whether these deep learning latent spaces could be relevant to the study of consciousness.

Initial social perceptions are often thought to reflect direct “read outs” of facial features. Instead, we outline a perspective whereby initial perceptions emerge from an automatic yet gradual process of negotiation between the perceptual cues inherent to a person (e.g., facial cues) and top-down social cognitive processes harbored within perceivers. This perspective argues that perceivers’ social-conceptual knowledge in particular can have a fundamental structuring role in perceptions, and thus how we think about social groups, emotions, or personality traits helps determine how we visually perceive them in other people. Integrative evidence from real-time behavioral paradigms (e.g., mouse-tracking), multivariate fMRI, and computational modeling will be discussed. Together, this work shows that the way we use facial cues to categorize other people into social groups (e.g., gender, race), perceive their emotion (e.g., anger), or infer their personality (e.g., trustworthiness) are all fundamentally shaped by prior social-conceptual knowledge and stereotypical assumptions. We find that these top-down impacts on initial perceptions are driven by the interplay of higher-order prefrontal regions involved in top-down predictions and lower-level fusiform regions involved in face processing. We argue that the perception of social categories, emotions, and traits from faces can all be conceived as resulting from an integrated system relying on domain-general cognitive properties. In this system, both visual and social cognitive processes are in a close exchange, and initial social perceptions emerge in part out of the structure of social-conceptual knowledge.

The brain is remarkably flexible, and appears to instantly reconfigure its processing depending on what’s needed to solve a task at hand: fMRI studies indicate that distal brain areas appear to fluidly couple and decouple with one another depending on behavioral context. But the structural architecture of the brain is comprised of long-range axonal projections that are relatively fixed by adulthood. How does the global dynamism evident in fMRI recordings manifest at a cellular level? To bridge the gap between the activity of single neurons and cortex-wide networks, we correlated electrophysiological recordings of individual neurons in primary visual (V1) and retrosplenial (RSP) associational cortex with activity across dorsal cortex, recorded simultaneously using widefield calcium imaging. We found that individual neurons in both cortical areas independently engaged in different distributed cortical networks depending on the animal’s behavioral state, suggesting that locomotion puts cortex into a more sensory driven mode relevant for navigation.

In many learning and decision scenarios, especially sequential settings like mazes or games, it is easy to state an objective function but difficult to compute it, for instance because this can require enumerating many possible future trajectories. This, in turn, motivates a variety of more tractable approximations which then raise resource-rationality questions about whether and when an efficient agent should invest time or resources in computing decision variables more accurately. Previous work has used a simple all-or-nothing version of this reasoning as a framework to explain many phenomena of automaticity, habits, and compulsion in humans and animals. Here, I present a more finegrained theoretical analysis of deliberation, which attempts to address not just whether to deliberate vs. act, but which of many possible actions and trajectories to consider. Empirically, I first motivate and compare this account to nonlocal representations of spatial trajectories in the rodent place cell system, which are thought to be involved in planning. I also consider its implications, in humans, for variation over time and situations in subjective feelings of mental effort, boredom, and cognitive fatigue. Finally, I present results from a new study using magnetoencephalography in humans to measure subjective consideration of possible trajectories during a sequential learning task, and study its relationship to rational prioritization and to choice behavior.

Sequential temporal ordering and patterning are key features of natural signals, used by the brain to decode stimuli and perceive them as sensory objects. Touch is one sensory modality where temporal patterning carries key information, and the rodent whisker system is a prominent model for understanding neuronal coding and plasticity underlying touch sensation. Neurons in this system are precise encoders of fluctuations in whisker dynamics down to a timescale of milliseconds, but it is not clear whether they can refine their encoding abilities as a result of learning patterned stimuli. For example, can they enhance temporal integration to become better at distinguishing sequences? To explore how cortical coding plasticity underpins sequence discrimination, we developed a task in which mice distinguished between tactile ‘word’ sequences constructed from distinct vibrations delivered to the whiskers, assembled in different orders. Animals licked to report the presence of the target sequence. Optogenetic inactivation showed that the somatosensory cortex was necessary for sequence discrimination. Two-photon imaging in layer 2/3 of the primary somatosensory “barrel” cortex (S1bf) revealed that, in well-trained animals, neurons had heterogeneous selectivity to multiple task variables including not just sensory input but also the animal’s action decision and the trial outcome (presence or absence of the predicted reward). Many neurons were activated preceding goal-directed licking, thus reflecting the animal’s learnt action in response to the target sequence; these neurons were found as soon as mice learned to associate the rewarded sequence with licking. In contrast, learning evoked smaller changes in sensory response tuning: neurons responding to stimulus features were already found in naïve mice, and training did not generate neurons with enhanced temporal integration or categorical responses. Therefore, in S1bf sequence learning results in neurons whose activity reflects the learnt association between target sequence and licking, rather than a refined representation of sensory features. Taken together with results from other laboratories, our findings suggest that neurons in sensory cortex are involved in task-specific processing and that an animal does not sense the world independently of what it needs to feel in order to guide behaviour.

Neural responses in the visual system are usually not purely visual but depend on behavioural and internal states such as arousal. This dependence is seen both in primary visual cortex (V1) and in subcortical brain structures receiving direct retinal input. In this talk, I will show that modulation by behavioural state arises as early as in the output of the retina.To measure retinal activity in the awake, intact brain, we imaged the synaptic boutons of retinal axons in the superficial superior colliculus (sSC) of mice. The activity of about half of the boutons depended not only on vision but also on running speed and pupil size, regardless of retinal illumination. Arousal typically reduced the boutons’ visual responses to preferred direction and their selectivity for direction and orientation.Arousal may affect activity in retinal boutons by presynaptic neuromodulation. To test whether the effects of arousal occur already in the retina, we recorded from retinal axons in the optic tract. We found that, in darkness, more than one third of the recorded axons was significantly correlated with running speed. Arousal had similar effects postsynaptically, in sSC neurons, independent of activity in V1, the other main source of visual inputs to colliculus. Optogenetic inactivation of V1 generally decreased activity in collicular neurons but did not diminish the effects of arousal. These results indicate that arousal modulates activity at every stage of the visual system. In the future, we will study the purpose and the underlying mechanisms of behavioural modulation in the early visual system

My lab seeks to understand how the mammalian brain performs the computations that underlie cognitive functions, including decision-making, short-term memory, and spatial navigation, at the level of the building blocks of the nervous system, cell types and neural populations organized into circuits. We have developed methods to measure, manipulate, and analyze neural circuits across various spatial and temporal scales, including technology for virtual reality, optical imaging, optogenetics, intracellular electrophysiology, molecular sensors, and computational modeling. I will present recent work that uses large scale calcium imaging to reveal the functional organization of the mouse posterior cortex for flexible decision-making during spatial navigation in virtual reality. I will also discuss work that uses optogenetics and calcium imaging during a variety of decision-making tasks to highlight how cognitive experience and context greatly alter the cortical circuits necessary for navigation decisions.

fMRI results have shown that the supplementary motor area (SMA) and the basal ganglia, most often discussed in their roles in generating action, are engaged by beat-based timing even in the absence of movement. Some have argued that the motor system is “recruited” by beat-based timing tasks due to the presence of motor-like timescales, but a deeper understanding of the roles of these motor structures is lacking. Reviewing a body of motor neurophysiology literature and drawing on the “active inference” framework, I argue that we can see the motor and timing functions of these brain areas as examples of dynamic sub-second prediction informed by sensory event timing. I hypothesize that in both cases, sub-second dynamics in SMA predict the progress of a temporal process outside the brain, and direct pathway activation in basal ganglia selects temporal and sensory predictions for the upcoming interval -- the only difference is that in motor processes, these predictions are made manifest through motor effectors. If we can unify our understanding of beat-based timing and motor control, we can draw on the substantial motor neuroscience literature to make conceptual leaps forward in the study of predictive timing and musical rhythm.

Although substance use disorders (SUDs) occur commonly in patients with schizophrenia and significantly worsen their clinical course, the neurobiological basis of SUDs in schizophrenia is not well understood. Therefore, there is a critical need to understand the mechanisms underlying SUDs in schizophrenia in order to identify potential targets for therapeutic intervention. Since drug use usually begins in adolescence, it is also important to understand the long-term effects of adolescent drug exposure on schizophrenia- and reward- related behaviors and circuitry. This talk will combine pharmacological, behavioral, electrophysiologic (local field potential recordings) and pre-clinical magnetic resonance imaging (resting-state functional connectivity and magnetic resonance spectroscopy) approaches to study these topics with an eye toward developing better treatment approaches.

Alzheimer’s Disease is characterised by the accumulation of misfolded proteins, namely amyloid and tau, however it is synapse loss which leads to the cognitive impairments associated with the disease. Many studies have focussed on single time points to determine the effects of pathology on synapses however this does not inform on the plasticity of the synapses, that is how they behave in vivo as the pathology progresses. Here we used in vivo two-photon microscopy to assess the temporal dynamics of axonal boutons and dendritic spines in mouse models of tauopathy[1] (rTg4510) and amyloidopathy[2] (J20). This revealed that pre- and post-synaptic components are differentially affected in both AD models in response to pathology. In the Tg4510 model, differences in the stability and turnover of axonal boutons and dendritic spines immediately prior to neurite degeneration was revealed. Moreover, the dystrophic neurites could be partially rescued by transgene suppression. Understanding the imbalance in the response of pre- and post-synaptic components is crucial for drug discovery studies targeting the synapse in Alzheimer’s Disease. To investigate how sub-types of synapses are affected in human tissue, the Multi-‘omics Atlas Project, a UKDRI initiative to comprehensively map the pathology in human AD, will determine the synaptome changes using imaging and synaptic proteomics in human post mortem AD tissue. The use of multiple brain regions and multiple stages of disease will enable a pseudotemporal profile of pathology and the associated synapse alterations to be determined. These data will be compared to data from preclinical models to determine the functional implications of the human findings, to better inform preclinical drug discovery studies and to develop a therapeutic strategy to target synapses in Alzheimer’s Disease[3].

Humans and other primates exhibit rich and versatile behaviour, switching nimbly between tasks as the environmental context requires. I will discuss the neural coding patterns that make this possible in humans and deep networks. First, using deep network simulations, I will characterise two distinct solutions to task acquisition (“lazy” and “rich” learning) which trade off learning speed for robustness, and depend on the initial weights scale and network sparsity. I will chart the predictions of these two schemes for a context-dependent decision-making task, showing that the rich solution is to project task representations onto orthogonal planes on a low-dimensional embedding space. Using behavioural testing and functional neuroimaging in humans, we observe BOLD signals in human prefrontal cortex whose dimensionality and neural geometry are consistent with the rich learning regime. Next, I will discuss the problem of continual learning, showing that behaviourally, humans (unlike vanilla neural networks) learn more effectively when conditions are blocked than interleaved. I will show how this counterintuitive pattern of behaviour can be recreated in neural networks by assuming that information is normalised and temporally clustered (via Hebbian learning) alongside supervised training. Together, this work offers a picture of how humans learn to partition knowledge in the service of structured behaviour, and offers a roadmap for building neural networks that adopt similar principles in the service of multitask learning. This is work with Andrew Saxe, Timo Flesch, David Nagy, and others.

The ability to encode the direction of image motion is fundamental to our sense of vision. Direction selectivity along the four cardinal directions is thought to originate in direction-selective ganglion cells (DSGCs), due to directionally-tuned GABAergic suppression by starburst cells. Here, by utilizing two-photon glutamate imaging to measure synaptic release, we reveal that direction selectivity along all four directions arises earlier than expected, at bipolar cell outputs. Thus, DSGCs receive directionally-aligned glutamatergic inputs from bipolar cell boutons. We further show that this bouton-specific tuning relies on cholinergic excitation and GABAergic inhibition from starburst cells. In this way, starburst cells are able to refine directional tuning in the excitatory visual pathway by modulating the activity of DSGC dendrites and their axonal inputs using two different neurotransmitters.

Every day when we fall asleep we lose consciousness, we are not there. And then, every morning, when we wake up, we regain it. What mechanisms give rise to consciousness, and how can we explain consciousness in the realm of the physical world of atoms and matter? For centuries, philosophers and scientists have aimed to crack this mystery. Much progress has been made in the past decades to understand how consciousness is instantiated in the brain, yet critical questions remain: can we develop a consciousness meter? Are computers conscious? What about other animals and babies? We have embarked in a large-scale, multicenter project to test, in the context of an open science, adversarial collaboration, two of the most prominent theories: Integrated information theory (IIT) and Global Neuronal Workspace (GNW) theory. We are collecting over 500 datasets including invasive and non-invasive recordings of the human brain, i.e.. fMRI, MEG and ECoG. We hope this project will enable theory-driven discoveries and further explorations that will help us better understand how consciousness fits inside the human brain.

Spontaneous and sensory-evoked cortical activity is highly state-dependent, promoting the functional flexibility of cortical circuits underlying perception and cognition. Using neural recordings in combination with behavioral state monitoring, we find that arousal and motor activity have complementary roles in regulating local cortical operations, providing dynamic control of sensory encoding. These changes in encoding are linked to altered performance on perceptual tasks. Neuromodulators, such as acetylcholine, may regulate this state-dependent flexibility of cortical network function. We therefore recently developed an approach for dual mesoscopic imaging of acetylcholine release and neural activity across the entire cortical mantle in behaving mice. We find spatiotemporally heterogeneous patterns of cholinergic signaling across the cortex. Transitions between distinct behavioral states reorganize the structure of large-scale cortico-cortical networks and differentially regulate the relationship between cholinergic signals and neural activity. Together, our findings suggest dynamic state-dependent regulation of cortical network operations at the levels of both local and large-scale circuits.