Using Fast Periodic Visual Stimulation to measure cognitive function in dementia
Fast periodic visual stimulation (FPVS) has emerged as a promising tool for assessing cognitive function in individuals with dementia. This technique leverages electroencephalography (EEG) to measure brain responses to rapidly presented visual stimuli, offering a non-invasive and objective method for evaluating a range of cognitive functions. Unlike traditional cognitive assessments, FPVS does not rely on behavioural responses, making it particularly suitable for individuals with cognitive impairment. In this talk I will highlight a series of studies that have demonstrated its ability to detect subtle deficits in recognition memory, visual processing and attention in dementia patients using EEG in the lab, at home and in clinic. The method is quick, cost-effective, and scalable, utilizing widely available EEG technology. FPVS holds significant potential as a functional biomarker for early diagnosis and monitoring of dementia, paving the way for timely interventions and improved patient outcomes.
Stability of visual processing in passive and active vision
The visual system faces a dual challenge. On the one hand, features of the natural visual environment should be stably processed - irrespective of ongoing wiring changes, representational drift, and behavior. On the other hand, eye, head, and body motion require a robust integration of pose and gaze shifts in visual computations for a stable perception of the world. We address these dimensions of stable visual processing by studying the circuit mechanism of long-term representational stability, focusing on the role of plasticity, network structure, experience, and behavioral state while recording large-scale neuronal activity with miniature two-photon microscopy.
Brain and Behavior: Employing Frequency Tagging as a Tool for Measuring Cognitive Abilities
Frequency tagging based on fast periodic visual stimulation (FPVS) provides a window into ongoing visual and cognitive processing and can be leveraged to measure rule learning and high-level categorization. In this talk, I will present data demonstrating highly proficient categorization as living and non-living in preschool children, and characterize the development of this ability during infancy. In addition to associating cognitive functions with development, an intriguing question is whether frequency tagging also captures enduring individual differences, e.g. in general cognitive abilities. First studies indicate high psychometric quality of FPVS categorization responses (XU et al., Dzhelyova), providing a basis for research on individual differences. I will present results from a pilot study demonstrating high correlations between FPVS categorization responses and behavioral measures of processing speed and fluid intelligences. Drawing upon this first evidence, I will discuss the potential of frequency tagging for diagnosing cognitive functions across development.
A draft connectome for ganglion cell types of the mouse retina
The visual system of the brain is highly parallel in its architecture. This is clearly evident in the outputs of the retina, which arise from neurons called ganglion cells. Work in our lab has shown that mammalian retinas contain more than a dozen distinct types of ganglion cells. Each type appears to filter the retinal image in a unique way and to relay this processed signal to a specific set of targets in the brain. My students and I are working to understand the meaning of this parallel organization through electrophysiological and anatomical studies. We record from light-responsive ganglion cells in vitro using the whole-cell patch method. This allows us to correlate directly the visual response properties, intrinsic electrical behavior, synaptic pharmacology, dendritic morphology and axonal projections of single neurons. Other methods used in the lab include neuroanatomical tracing techniques, single-unit recording and immunohistochemistry. We seek to specify the total number of ganglion cell types, the distinguishing characteristics of each type, and the intraretinal mechanisms (structural, electrical, and synaptic) that shape their stimulus selectivities. Recent work in the lab has identified a bizarre new ganglion cell type that is also a photoreceptor, capable of responding to light even when it is synaptically uncoupled from conventional (rod and cone) photoreceptors. These ganglion cells appear to play a key role in resetting the biological clock. It is just this sort of link, between a specific cell type and a well-defined behavioral or perceptual function, that we seek to establish for the full range of ganglion cell types. My research concerns the structural and functional organization of retinal ganglion cells, the output cells of the retina whose axons make up the optic nerve. Ganglion cells exhibit great diversity both in their morphology and in their responses to light stimuli. On this basis, they are divisible into a large number of types (>15). Each ganglion-cell type appears to send its outputs to a specific set of central visual nuclei. This suggests that ganglion cell heterogeneity has evolved to provide each visual center in the brain with pre-processed representations of the visual scene tailored to its specific functional requirements. Though the outline of this story has been appreciated for some time, it has received little systematic exploration. My laboratory is addressing in parallel three sets of related questions: 1) How many types of ganglion cells are there in a typical mammalian retina and what are their structural and functional characteristics? 2) What combination of synaptic networks and intrinsic membrane properties are responsible for the characteristic light responses of individual types? 3) What do the functional specializations of individual classes contribute to perceptual function or to visually mediated behavior? To pursue these questions, we label retinal ganglion cells by retrograde transport from the brain; analyze in vitro their light responses, intrinsic membrane properties and synaptic pharmacology using the whole-cell patch clamp method; and reveal their morphology with intracellular dyes. Recently, we have discovered a novel ganglion cell in rat retina that is intrinsically photosensitive. These ganglion cells exhibit robust light responses even when all influences from classical photoreceptors (rods and cones) are blocked, either by applying pharmacological agents or by dissociating the ganglion cell from the retina. These photosensitive ganglion cells seem likely to serve as photoreceptors for the photic synchronization of circadian rhythms, the mechanism that allows us to overcome jet lag. They project to the circadian pacemaker of the brain, the suprachiasmatic nucleus of the hypothalamus. Their temporal kinetics, threshold, dynamic range, and spectral tuning all match known properties of the synchronization or "entrainment" mechanism. These photosensitive ganglion cells innervate various other brain targets, such as the midbrain pupillary control center, and apparently contribute to a host of behavioral responses to ambient lighting conditions. These findings help to explain why circadian and pupillary light responses persist in mammals, including humans, with profound disruption of rod and cone function. Ongoing experiments are designed to elucidate the phototransduction mechanism, including the identity of the photopigment and the nature of downstream signaling pathways. In other studies, we seek to provide a more detailed characterization of the photic responsiveness and both morphological and functional evidence concerning possible interactions with conventional rod- and cone-driven retinal circuits. These studies are of potential value in understanding and designing appropriate therapies for jet lag, the negative consequences of shift work, and seasonal affective disorder.
Functional Divergence at the Mouse Bipolar Cell Terminal
Research in our lab focuses on the circuit mechanisms underlying sensory computation. We use the mouse retina as a model system because it allows us to stimulate the circuit precisely with its natural input, patterns of light, and record its natural output, the spike trains of retinal ganglion cells. We harness the power of genetic manipulations and detailed information about cell types to uncover new circuits and discover their role in visual processing. Our methods include electrophysiology, computational modeling, and circuit tracing using a variety of imaging techniques.
Colour processing in the mouse brain for vision and beyond
Colour vision plays important roles in regulating animal behaviour, yet understanding of how such information is processed in the brain is still incomplete. Here I discuss our work addressing this issue in mice where, despite aspects of retinal organisation that might suggest limited capacity for colour vision, we find evidence of extensive cone-dependent spectral opponency across subcortical visual centres. In particular, our data both reveals important contributions of such colour signals to non-image-forming functions (regulation of the circadian system) but also indicate surprisingly sophisticated support for more conventional aspects of colour vision.
Novel Object Detection and Multiplexed Motion Representation in Retinal Bipolar Cells
Detection of motion is essential for survival, but how the visual system processes moving stimuli is not fully understood. Here, based on a detailed analysis of glutamate release from bipolar cells, we outline the rules that govern the representation of object motion in the early processing stages. Our main findings are as follows: (1) Motion processing begins already at the first retinal synapse. (2) The shape and the amplitude of motion responses cannot be reliably predicted from bipolar cell responses to stationary objects. (3) Enhanced representation of novel objects - particularly in bipolar cells with transient dynamics. (4) Response amplitude in bipolar cells matches visual salience reported in humans: suddenly appearing objects > novel motion > existing motion. These findings can be explained by antagonistic interactions in the center-surround receptive field, demonstrate that despite their simple operational concepts, classical center-surround receptive fields enable sophisticated visual computations.
Efficient coding and receptive field coordination in the retina
My laboratory studies how the retina processes visual scenes and transmits this information to the brain. We use multi-electrode arrays to record the activity of hundreds of retina neurons simultaneously in conjunction with transgenic mouse lines and chemogenetics to manipulate neural circuit function. We are interested in three major areas. First, we work to understand how neurons in the retina are functionally connected. Second we are studying how light-adaptation and circadian rhythms alter visual processing in the retina. Finally, we are working to understand the mechanisms of retinal degenerative conditions and we are investigating potential treatments in animal models.
Visual processing of feedforward and feedback signals in mouse thalamus
Traditionally, the dorsolateral geniculate nucleus (dLGN) of the thalamus has been considered a feedforward relay station for retinal signals to reach primary visual cortex. The local and long-range circuits of dLGN, however, suggest that this view is not correct. Indeed, besides the thalamo-cortical relay cells, dLGN contains local inhibitory interneurons, and receives not only feedforward input from the retina, but also massive direct and indirect feedback from primary visual cortex. Furthermore, it is one of the earliest processing stages in the visual system that integrates visual information with neuromodulatory signals.
Genetics and Therapy of Inherited Retinal Diseases
Restoring Vision
Neural mechanisms of active vision in the marmoset monkey
Human vision relies on rapid eye movements (saccades) 2-3 times every second to bring peripheral targets to central foveal vision for high resolution inspection. This rapid sampling of the world defines the perception-action cycle of natural vision and profoundly impacts our perception. Marmosets have similar visual processing and eye movements as humans, including a fovea that supports high-acuity central vision. Here, I present a novel approach developed in my laboratory for investigating the neural mechanisms of visual processing using naturalistic free viewing and simple target foraging paradigms. First, we establish that it is possible to map receptive fields in the marmoset with high precision in visual areas V1 and MT without constraints on fixation of the eyes. Instead, we use an off-line correction for eye position during foraging combined with high resolution eye tracking. This approach allows us to simultaneously map receptive fields, even at the precision of foveal V1 neurons, while also assessing the impact of eye movements on the visual information encoded. We find that the visual information encoded by neurons varies dramatically across the saccade to fixation cycle, with most information localized to brief post-saccadic transients. In a second study we examined if target selection prior to saccades can predictively influence how foveal visual information is subsequently processed in post-saccadic transients. Because every saccade brings a target to the fovea for detailed inspection, we hypothesized that predictive mechanisms might prime foveal populations to process the target. Using neural decoding from laminar arrays placed in foveal regions of area MT, we find that the direction of motion for a fixated target can be predictively read out from foveal activity even before its post-saccadic arrival. These findings highlight the dynamic and predictive nature of visual processing during eye movements and the utility of the marmoset as a model of active vision. Funding sources: NIH EY030998 to JM, Life Sciences Fellowship to JY
Stereo vision in humans and insects
Stereopsis – deriving information about distance by comparing views from two eyes – is widespread in vertebrates but so far known in only class of invertebrates, the praying mantids. Understanding stereopsis which has evolved independently in such a different nervous system promises to shed light on the constraints governing any stereo system. Behavioral experiments indicate that insect stereopsis is functionally very different from that studied in vertebrates. Vertebrate stereopsis depends on matching up the pattern of contrast in the two eyes; it works in static scenes, and may have evolved in order to break camouflage rather than to detect distances. Insect stereopsis matches up regions of the image where the luminance is changing; it is insensitive to the detailed pattern of contrast and operates to detect the distance to a moving target. Work from my lab has revealed a network of neurons within the mantis brain which are tuned to binocular disparity, including some that project to early visual areas. This is in contrast to previous theories which postulated that disparity was computed only at a single, late stage, where visual information is passed down to motor neurons. Thus, despite their very different properties, the underlying neural mechanisms supporting vertebrate and insect stereopsis may be computationally more similar than has been assumed.
The neuroscience of color and what makes primates special
Among mammals, excellent color vision has evolved only in certain non-human primates. And yet, color is often assumed to be just a low-level stimulus feature with a modest role in encoding and recognizing objects. The rationale for this dogma is compelling: object recognition is excellent in grayscale images (consider black-and-white movies, where faces, places, objects, and story are readily apparent). In my talk I will discuss experiments in which we used color as a tool to uncover an organizational plan in inferior temporal cortex (parallel, multistage processing for places, faces, colors, and objects) and a visual-stimulus functional representation in prefrontal cortex (PFC). The discovery of an extensive network of color-biased domains within IT and PFC, regions implicated in high-level object vision and executive functions, compels a re-evaluation of the role of color in behavior. I will discuss behavioral studies prompted by the neurobiology that uncover a universal principle for color categorization across languages, the first systematic study of the color statistics of objects and a chromatic mechanism by which the brain may compute animacy, and a surprising paradoxical impact of memory on face color. Taken together, my talk will put forward the argument that color is not primarily for object recognition, but rather for the assessment of the likely behavioral relevance, or meaning, of the stuff we see.
Visual processing beyond (rapid) serial visual presentations
Natural visual stimuli for mice
Vision for escape and pursuit
We want to understand how the visual system detects and tracks salient stimuli in the environment to initiate and guide specific behaviors (i.e., visual neuroethology). Predator avoidance and prey capture are central selection pressures of animal evolution. Mice use vision to detect aerial predators and hunt insects. I will discuss studies from my group that identify specific circuits and pathways in the early visual system (i.e., the retina and its subcortical targets) mediating predator avoidance and prey capture in mice. Our results highlight the importance of subcellular visual processing in the retina and the alignment of viewing strategies with region- and cell-type-specific retinal ganglion cell projection patterns to the brain.
The neural basis of human face identity recognition
The face is the primary source of information for recognizing the identity of people around us, but the neural basis of this astonishing ability remains largely unknown. In this presentation, I will define the fundamental problem of face identity recognition, arguing that there is a specific expertise of the human species at this function. I will then attempt to integrate a large corpus of observations from lesion studies, neuroimaging, human intracerebral recordings and stimulation into a coherent framework to shed light on the neural mechanisms of human face identity recognition.
Using marmosets for the study of the visual cortex: unique opportunities, and some pitfalls
Marmosets (Callithrix jacchus) are small South American monkeys which are being increasingly becoming adopted as animal models in neuroscience. Knowledge about the marmoset visual system has developed rapidly over the last decade. But what are the comparative advantages, and disadvantages involved in adopting this emerging model, as opposed to the more traditionally used macaque monkey? In this talk I will present case studies where the simpler brain morphology and short developmental cycle of the marmoset have been key factors in facilitating discoveries about the anatomy and physiology of the visual system. Although no single species provides the “ideal” animal model for invasive studies of the neural bases of visual processing, I argue that the development of robust methodologies for the study of the marmoset brain provides exciting opportunities to address long-standing problems in neuroscience.
Towards hybrid models of retinal circuits - integrating biophysical realism, anatomical constraints and predictive performance
Visual processing in the retina has been studied in great detail at all levels such that a comprehensive picture of the retina's cell types and the many neural circuits they form is emerging. However, the currently best performing models of retinal function are black-box CNN models which are agnostic to such biological knowledge. Here, I present two of our recent attempts to develop computational models of processing in the inner retina, which both respect biophysical and anatomical constraints yet provide accurate predictions of retinal activity
What the eye tells the brain: Visual feature extraction in the mouse retina
Visual processing begins in the retina: within only two synaptic layers, multiple parallel feature channels emerge, which relay highly processed visual information to different parts of the brain. To functionally characterize these feature channels we perform calcium and glutamate population activity recordings at different levels of the mouse retina. This allows following the complete visual signal across consecutive processing stages in a systematic way. In my talk, I will summarize our recent findings on the functional diversity of retinal output channels and how they arise within the retinal network. Specifically, I will talk about the role of inhibition and cell-type specific dendritic processing in generating diverse visual channels. Then, I will focus on how color – a single visual feature – emerges across all retinal processing layers and link our results to behavioral output and the statistics of mouse natural scenes. With our approach, we hope to identify general computational principles of retinal signaling, thereby increasing our understanding of what the eye tells the brain.
Human reconstruction of local image structure from natural scenes
Retinal projections often poorly represent the structure of the physical world: well-defined boundaries within the eye may correspond to irrelevant features of the physical world, while critical features of the physical world may be nearly invisible at the retinal projection. Visual cortex is equipped with specialized mechanisms for sorting these two types of features according to their utility in interpreting the scene, however we know little or nothing about their perceptual computations. I will present novel paradigms for the characterization of these processes in human vision, alongside examples of how the associated empirical results can be combined with targeted models to shape our understanding of the underlying perceptual mechanisms. Although the emerging view is far from complete, it challenges compartmentalized notions of bottom-up/top-down object segmentation, and suggests instead that these two modes are best viewed as an integrated perceptual mechanism.
Domain Specificity in the Human Brain: What, Whether, and Why?
The last quarter century has provided extensive evidence that some regions of the human cortex are selectively engaged in processing a single specific domain of information, from faces, places, and bodies to language, music, and other people’s thoughts. This work dovetails with earlier theories in cognitive science highlighting domain specificity in human cognition, development, and evolution. But many questions remain unanswered about even the clearest cases of domain specificity in the brain, the selective engagement of the FFA, PPA, and EBA in the perception of faces, places, and bodies, respectively. First, these claims lack precision, saying little about what is computed and how, and relying on human judgements to decide what counts as a face, place, or body. Second, they provide no account of the reliably varying responses of these regions across different “preferred” images, or across different “nonpreferred” images for each category. Third, the category selectivity of each region is vulnerable to refutation if any of the vast set of as-yet-untested nonpreferred images turns out to produce a stronger response than preferred images for that region. Fourth, and most fundamentally, they provide no account of why, from a computational point of view, brains should exhibit this striking degree of functional specificity in the first place, and why we should have the particular visual specializations we do, for faces, places, and bodies, but not (apparently) for food or snakes. The advent of convolutional neural networks (CNNs) to model visual processing in the ventral pathway has opened up many opportunities to address these long-standing questions in new ways. I will describe ongoing efforts in our lab to harness CNNs to do just that.
Blindspots in Computer Vision - How can neuroscience guide AI?
Scientists have worked to recreate human vision in computers for the past 50 years. But how much about human vision do we actually know? And can the brain be useful in furthering computer vision? This talk will take a look at the similarities and differences between (modern) computer vision and human vision, as well as the important crossovers, collaborations, and applications that define the interface between computational neuroscience and computer vision. If you want to know more about how the brain sees (really sees), how computer vision developments are inspired by the brain, or how to apply AI to neuroscience, this talk is for you.