Despite her still poor visual acuity and minimal visual experience, a 2-3 month old baby will reliably respond to facial expressions, smiling back at her caretaker or older sibling. But what if that same baby had been deprived of her early visual experience? Will she be able to appropriately respond to seemingly mundane interactions, such as a peer’s facial expression, if she begins seeing at the age of 10? My work is part of Project Prakash, a dual humanitarian/scientific mission to identify and treat curably blind children in India and then study how their brain learns to make sense of the visual world when their visual journey begins late in life. In my talk, I will give a brief overview of Project Prakash, and present findings from one of my primary lines of research: plasticity of face perception with late sight onset. Specifically, I will discuss a mixed methods effort to probe and explain the differential windows of plasticity that we find across different aspects of distributed face recognition, from distinguishing a face from a nonface early in the developmental trajectory, to recognizing facial expressions, identifying individuals, and even identifying one’s own caretaker. I will draw connections between our empirical findings and our recent theoretical work hypothesizing that children with late sight onset may suffer persistent face identification difficulties because of the unusual acuity progression they experience relative to typically developing infants. Finally, time permitting, I will point to potential implications of our findings in supporting newly-sighted children as they transition back into society and school, given that their needs and possibilities significantly change upon the introduction of vision into their lives.

Vision and visual cognition is experience-dependent with likely multiple sensitive periods, but we know very little about statistics of visual experience at the scale of everyday life and how they might change with development. By traditional assumptions, the world at the massive scale of daily life presents pretty much the same visual statistics to all perceivers. I will present an overview our work on ego-centric vision showing that this is not the case. The momentary image received at the eye is spatially selective, dependent on the location, posture and behavior of the perceiver. If a perceiver’s location, possible postures and/or preferences for looking at some kinds of scenes over others are constrained, then their sampling of images from the world and thus the visual statistics at the scale of daily life could be biased. I will present evidence with respect to both low-level and higher level visual statistics about the developmental changes in the visual input over the first 18 months post-birth.

During development, the complex neuronal circuitry of the brain arises from limited information contained in the genome. After the genetic code instructs the birth of neurons, the emergence of brain regions, and the formation of axon tracts, it is believed that neuronal activity plays a critical role in shaping circuits for behavior. Current AI technologies are modeled after the same principle: connections in an initial weight matrix are pruned and strengthened by activity-dependent signals until the network can sufficiently generalize a set of inputs into outputs. Here, we challenge these learning-dominated assumptions by quantifying the contribution of neuronal activity to the development of visually guided swimming behavior in larval zebrafish. Intriguingly, dark-rearing zebrafish revealed that visual experience has no effect on the emergence of the optomotor response (OMR). We then raised animals under conditions where neuronal activity was pharmacologically silenced from organogenesis onward using the sodium-channel blocker tricaine. Strikingly, after washout of the anesthetic, animals performed swim bouts and responded to visual stimuli with 75% accuracy in the OMR paradigm. After shorter periods of silenced activity OMR performance stayed above 90% accuracy, calling into question the importance and impact of classical critical periods for visual development. Detailed quantification of the emergence of functional circuit properties by brain-wide imaging experiments confirmed that neuronal circuits came ‘online’ fully tuned and without the requirement for activity-dependent plasticity. Thus, contrary to what you learned on your mother's knee, complex sensory guided behaviors can be wired up innately by activity-independent developmental mechanisms.

Understanding Human ability to learn novel concepts from just a few sensory experiences is a fundamental problem in cognitive neuroscience. I will describe a recent work with Ben Sorcher and Surya Ganguli (PNAS, October 2022) in which we propose a simple, biologically plausible, and mathematically tractable neural mechanism for few-shot learning of naturalistic concepts. We posit that the concepts that can be learned from few examples are defined by tightly circumscribed manifolds in the neural firing-rate space of higher-order sensory areas. Discrimination between novel concepts is performed by downstream neurons implementing ‘prototype’ decision rule, in which a test example is classified according to the nearest prototype constructed from the few training examples. We show that prototype few-shot learning achieves high few-shot learning accuracy on natural visual concepts using both macaque inferotemporal cortex representations and deep neural network (DNN) models of these representations. We develop a mathematical theory that links few-shot learning to the geometric properties of the neural concept manifolds and demonstrate its agreement with our numerical simulations across different DNNs as well as different layers. Intriguingly, we observe striking mismatches between the geometry of manifolds in intermediate stages of the primate visual pathway and in trained DNNs. Finally, we show that linguistic descriptors of visual concepts can be used to discriminate images belonging to novel concepts, without any prior visual experience of these concepts (a task known as ‘zero-shot’ learning), indicated a remarkable alignment of manifold representations of concepts in visual and language modalities. I will discuss ongoing effort to extend this work to other high level cognitive tasks.

Visual perception is traditionally studied in isolation from other sensory systems, and while this approach has been exceptionally successful, in the real world, visual objects are often accompanied by sounds, smells, tactile information, or taste. How is visual processing influenced by these other sensory inputs? In this talk, I will review studies from our lab showing that a sound can influence the perception of a visual object in multiple ways. In the first part, I will focus on spatial interactions between sound and sight, demonstrating that co-localized sounds enhance visual perception. Then, I will show that these cross-modal interactions also occur at a higher contextual and semantic level, where naturalistic sounds facilitate the processing of real-world objects that match these sounds. Throughout my talk I will explore to what extent sounds not only improve visual processing but also alter perceptual representations of the objects we see. Most broadly, I will argue for the importance of considering multisensory influences on visual perception for a more complete understanding of our visual experience.

The way we represent and perceive time has crucial implications for studying temporality in conscious experience. Contrasting positions posit that temporal information is separately abstracted out like any other perceptual property, or that time is represented through representations having temporal properties themselves. To add to this debate, we investigated alterations in felt time in conditions where only conscious visual experience is altered while a bistable figure remains physically unchanged. In this talk, I will discuss two studies that we have done in relation to answering this question. In study 1, we investigated whether perceptual switches in fixed intervals altered felt time. In three experiments we showed that a break in visual experience (via a perceptual switch) also leads to a break in felt time. In study 2, we are currently looking at figure-ground perception in ambigous displays. Here, in experiment 1 we show that differences in flicker frequencies on ambigous regions can induce figure-ground segregation. To see if a reverse complementarity exists for felt time, we ask participants to view ambigous regions as figure/ground and show that they have different temporal resolutions for the same region based on whether it is seen as figure or background. Overall, the two studies provide evidence for temporal mirroring and isomorphism in visual experience, arguing for a link between the timing of experience and time perception.

Discriminating distinct objects and concepts from sensory stimuli is essential for survival. Our brains accomplish this feat by forming meaningful internal representations in deep sensory networks with plastic synaptic connections. Experience-dependent plasticity presumably exploits temporal contingencies between sensory inputs to build these internal representations. However, the precise mechanisms underlying plasticity remain elusive. We derive a local synaptic plasticity model inspired by self-supervised machine learning techniques that shares a deep conceptual connection to Bienenstock-Cooper-Munro (BCM) theory and is consistent with experimentally observed plasticity rules. We show that our plasticity model yields disentangled object representations in deep neural networks without the need for supervision and implausible negative examples. In response to altered visual experience, our model qualitatively captures neuronal selectivity changes observed in the monkey inferotemporal cortex in-vivo. Our work suggests a plausible learning rule to drive learning in sensory networks while making concrete testable predictions.

Visual perceptual learning (VPL) is defined as improved performance on a visual task due to visual experience. It was once argued that attention to a visual feature is necessary for VPL of the feature to occur. Contrary to this view, a phenomenon called task-irrelevant VPL demonstrated that VPL can occur due to exposure to a feature which is sub-threshold and task-irrelevant, and therefore, unattended. A series of findings based on task-irrelevant VPL has indicated the following two mechanisms. First, attention to a feature facilitates VPL of the feature while inhibiting VPL of unattended and supra-threshold features. Second, reward paired with a feature enables VPL of the feature irrespective of whether the feature is attended or not. However, we recently found an additional twist; VPL of a task-irrelevant and supra-threshold feature embedded in a natural scene is not subject to the inhibition of attention. This new finding suggests a need to revise the current view or add a new mechanism as to how VPL occurs.

Our visual experience of living in a three-dimensional world is created from the information contained in the two-dimensional images projected into our eyes. The overlapping visual fields of the two eyes mean that their images are highly correlated, and that the small differences that are present represent an important cue to depth. Binocular neurons encode this information in a way that both maximises efficiency and optimises disparity tuning for the depth structures that are found in our natural environment. Neural network models provide a clear account of how these binocular neurons encode the local binocular disparity in images. These models can be expanded to multi-layer models that are sensitive to salient features of scenes, such as the orientations and discontinuities between surfaces. These deep neural network models have also shown the importance of binocular disparity for the segmentation of images into separate objects, in addition to the estimation of distance. These results demonstrate the usefulness of machine learning approaches as a tool for understanding biological vision.

In Europe, the number of people aged 65 and older is increasing dramatically, and research related to ageing is more crucial than ever. The main research dedicated to age-related changes concentrates on cognitive or sensory deficits. This is also the case in vision research. However, the majority of older adults ages without major cognitive or optical or deficits. These are foremost good news, but even in the absence of neurodegenerative or eye diseases changes in visual perception occur. It has been suggested that age-related changes are due to a general decline of cognitive, perceptual and sensory functions. However, more recent studies reveal large individual differences within the ageing population and whereas some functions show age-related deterioration, others are surprisingly unaffected. Overall, it becomes increasingly apparent that perceptual changes in healthy ageing cannot be attributed to one single underlying factor. I will present studies from various areas of visual perception that challenge the view that age-related changes are primarily related to decline. Instead, our findings suggest that age-related changes are the result of visual experience, such that the brain ages optimally given the input it receives.

Despite the recent success of deep learning in artificial intelligence, the lack of biological plausibility and labeled data in natural learning still poses a challenge in understanding biological learning. At the other extreme lies Hebbian learning, the simplest local and unsupervised one, yet considered to be computationally less efficient. In this talk, I would introduce a novel method to infer the form of Hebbian learning from in vivo data. Applying the method to the data obtained from the monkey inferior temporal cortex for the recognition task indicates how Hebbian learning changes the dynamic properties of the circuits and may promote brain oscillation. Notably, recent electrophysiological data observed in rodent V1 showed that the effect of visual experience on direction selectivity was similar to that observed in monkey data and provided strong validation of asymmetric changes of feedforward and recurrent synaptic strengths inferred from monkey data. This may suggest a general learning principle underlying the same computation, such as familiarity detection across different features represented in different brain regions.

The Manookin lab is investigating the structure and function of neural circuits within the retina and developing techniques for treating blindness. Many blinding diseases, such as retinitis pigmentosa, cause death of the rods and cones, but spare other cell types within the retina. Thus, many techniques for restoring visual function following blindness are based on the premise that other cells within the retina remain viable and capable of performing their various roles in visual processing. There are more than 80 different neuronal types in the human retina and these form the components of the specialized circuits that transform the signals from photoreceptors into a neural code responsible for our perception of color, form, and motion, and thus visual experience. The Manookin laboratory is investigating the function and connectivity of neural circuits in the retina using a variety of techniques including electrophysiology, calcium imaging, and electron microscopy. This knowledge is being used to develop more effective techniques for restoring visual function following blindness.

Unlike even the most sophisticated current forms of artificial intelligence, developing biological organisms must build their neural hardware from scratch. Furthermore they must start to evade predators and find food before this construction process is complete. I will discuss an interdisciplinary program of mathematical and experimental work which addresses some of the computational principles underlying neural development. This includes (i) how growing axons navigate to their targets by detecting and responding to molecular cues in their environment, (ii) the formation of maps in the visual cortex and how these are influenced by visual experience, and (iii) how patterns of neural activity in the zebrafish brain develop to facilitate precisely targeted hunting behaviour. Together this work contributes to our understanding of both normal neural development and the etiology of neurodevelopmental disorders.

The development of neural circuits is profoundly impacted by both spontaneous and sensory experience. This is perhaps most well studied in the visual system, where disruption of early spontaneous activity called retinal waves prior to eye opening and visual deprivation after eye opening leads to alterations in the response properties and connectivity in several visual centers in the brain. We address this question in the retina, which comprises multiple circuits that encode different features of the visual scene, culminating in over 40 different types of retinal ganglion cells. Direction-selective ganglion cells respond strongly to an image moving in the preferred direction and weakly to an image moving in the opposite, or null, direction. Moreover, as recently described (Sabbah et al, 2017) the preferred directions of direction selective ganglion cells cluster along four directions that align along two optic flow axes, causing variation of the relative orientation of preferred directions along the retinal surface. I will provide recent progress in the lab that addresses the role of visual experience and spontaneous retinal waves in the establishment of direction selective tuning and direction selectivity maps in the retina.