Vision Science seminars

All sensory perception and every coordinated movement, as well as feelings, memories and motivation, arise from the bustling activity of many millions of interconnected cells in the brain. The ultimate function of this elaborate network is to generate behavior. We use zebrafish as our experimental model, employing a diverse array of molecular, genetic, optical, connectomic, behavioral and computational approaches. The goal of our research is to understand how neuronal circuits integrate sensory inputs and internal state and convert this information into behavioral responses.

The human visual system enables us to recognize objects (e.g., this is a cup) and act upon them (e.g., grasp the cup) with astonishing ease and accuracy. For decades, it was widely accepted that these different functions rely on two separated cortical pathways. The ventral occipitotemporal pathway subserves object recognition, while the dorsal occipitoparietal pathway promotes visually guided actions. In my talk, I will discuss recent evidence from a series of neuropsychological, developmental and neuroimaging studies that were aimed to explore the nature of object representations in the dorsal pathway. The results from these studies highlight the plausible role of the dorsal pathway in object perception and reveal an interplay between shape representations derived by the two pathways. Together, these findings challenge the binary distinction between the two pathways and are consistent with the view that object recognition is not the sole product of ventral pathway computations, but instead relies on a distributed network of regions.

During communication, alignment between signals and sensors can be critical. Signals are often best perceived from specific angles, and sensory systems can also exhibit strong directional biases. However, we know little about how animals establish and maintain such signaling alignment during communication. To investigate this, we characterized the spatial dynamics of visual courtship signal- ing in the jumping spider Habronattus pyrrithrix. The male performs forward-facing displays involving complex color and movement patterns, with distinct long- and short-range phases. The female views displays with 2 distinct eye types and can only perceive colors and fine patterns of male displays when they are presented in her frontal field of view. Whether and how courtship interactions pro- duce such alignment between male display and female field of view is unknown. We recorded relative positions and orientations of both actors throughout courtship and established the role of each sex in maintaining signaling alignment. Males always oriented their displays toward the female. However, when females were free to move, male displays were consistently aligned with female princi- pal eyes only during short-range courtship. When female position was fixed, signaling alignment consistently occurred during both phases, suggesting that female movement reduces communication efficacy. When female models were experimentally rotated to face away during courtship, males rarely repositioned themselves to re-align their display. However, males were more likely to present cer- tain display elements after females turned to face them. Thus, although signaling alignment is a function of both sexes, males appear to rely on female behavior for effective communication

Visual attention refers to a set of processes allowing selection of relevant and filtering out of irrelevant information in the visual environment. A large amount of research on visual attention has involved visual search paradigms, where observers are asked to report whether a single target is present or absent. However, recent studies have revealed that these classic single-target visual search tasks only provide a snapshot of how attention is allocated in the visual environment, and that multitarget visual foraging tasks may capture the dynamics visual attention more accurately. In visual foraging, observers are asked to select multiple instances of multiple target types, as fast as they can. A critical question in foraging research concerns the factors driving the next target selection. Most likely, this would require two steps: (1) identifying a set of candidates for the next selection, and (2) selecting the best option among these candidates. After having briefly described the advantage of visual foraging over visual search, I will review recent visual foraging studies testing the influence of several manipulations (e.g., target crypticity, number of items, selection modality) on foraging behaviour. Overall, these studies revealed that the next target selection during visual foraging is determined by the competition between three factors: target value, target proximity, and priming of features. I will explain how the analysis of individual differences in foraging behaviour can provide important information, with the idea that individuals show by-default internal biases toward value, proximity and priming that determine their search strategy and behaviour.

Gravity is a constant in our lives. It provides an internalized reference to which all other perceptions are related. We can experimentally manipulate the relationship between physical gravity with other cues to the direction of “up” using virtual reality - with either HMDs or specially built tilting environments - to explore how gravity contributes to perceptual judgements. The effect of gravity can also be cancelled by running experiments on the International Space Station in low Earth orbit. Changing orientation relative to gravity - or even just perceived orientation – affects your perception of how far away things are (they appear closer when supine or prone). Cancelling gravity altogether has a similar effect. Changing orientation also affects how much visual motion is needed to perceive a particular travel distance (you need less when supine or prone). Adapting to zero gravity has the opposite effect (you need more). These results will be discussed in terms of their practical consequences and the multisensory processes involved, in particular the response to visual-vestibular conflict.

Each spring, billions of Bogong moths escape hot conditions in different regions of southeast Australia by migrating over 1000 km to a limited number of cool caves in the Australian Alps, historically used for aestivating over the summer. At the beginning of autumn the same individuals make a return migration to their breeding grounds to reproduce and die. To steer migration Bogong moths sense the Earth’s magnetic field and correlate its directional information with visual cues. In this presentation, we will show that a critically important visual cue is the distribution of starlight within the austral night sky. By tethering spring and autumn migratory moths in a flight simulator, we found that under natural dorsally-projected night skies, and in a nulled magnetic field (disabling the magnetic sense), moths flew in their seasonally appropriate migratory directions, turning in the opposite direction when the night sky was rotated 180°. Visual interneurons in the moth’s optic lobe and central brain responded vigorously to identical sky rotations. Migrating Bogong moths thus use the starry night sky as a true compass to distinguish geographic cardinal directions, the first invertebrate known to do so. These stellar cues are likely reinforced by the Earth’s magnetic field to create a robust compass mechanism for long-distance nocturnal navigation.

Our study reveals the most elaborate opsin expression patterns ever described in any animal eye. In mantis shrimp, a pugnacious crustacean renowned for its visual sophistication, we found unexpected retinal expression patterns highlighting the potential for cryptic photoreceptor functional diversity, including single photoreceptors that coexpress opsins from different spectral clades and a single opsin with a putative nonvisual function important in color vision. This study demonstrates the evolutionary potential for increasing visual system functional diversity through opsin gene duplication and diversification, as well as changes in patterns of gene coexpression among photoreceptors and retinula cells. These results have significant implications for the function of other visual systems, particularly in arthropods where large numbers of retinally expressed opsins have been documented.

All sensory information – like what we see, hear and smell – gets encoded in spike trains by sensory neurons and gets sent to the brain. Due to the complexity of neural circuits and the difficulty of quantifying complex animal behavior, it has been exceedingly hard to resolve how the brain decodes these spike trains to drive behavior. We now measure quantal signals originating from sparse photons through the most sensitive neural circuits of the mammalian retina and correlate the retinal output spike trains with precisely quantified behavioral decisions. We utilize a combination of electrophysiological measurements on the most sensitive ON and OFF retinal ganglion cell types and a novel deep-learning based tracking technology of the head and body positions of freely-moving mice. We show that visually-guided behavior relies on information from the retinal ON pathway for the dimmest light increments and on information from the retinal OFF pathway for the dimmest light decrements (“quantal shadows”). Our results show that the distribution of labor between ON and OFF pathways starts already at starlight supporting distinct pathway-specific visual computations to drive visually-guided behavior. These results have several fundamental consequences for understanding how the brain integrates information across parallel information streams as well as for understanding the limits of sensory signal processing. In my talk, I will discuss some of the most eminent consequences including the extension of this “Quantum Behavior” paradigm from mouse vision to monkey and human visual systems.

State-of-the-art machine vision models can predict human recognition memory for complex scenes with astonishing accuracy. In this talk I present work that investigated how memorable scenes are actually remembered and experienced by human observers. We found that memorable scenes were recognized largely based on recollection of specific episodic details but also based on familiarity for an entire scene. I thus highlight current limitations in machine vision models emulating human recognition memory, with promising opportunities for future research. Moreover, we were interested in what observers specifically remember about complex scenes. We thus considered the functional role of eye-movements as a window into the content of memories, particularly when observers recollected specific information about a scene. We found that when observers formed a memory representation that they later recollected (compared to scenes that only felt familiar), the overall extent of exploration was broader, with a specific subset of fixations clustered around later to-be-recollected scene content, irrespective of the memorability of a scene. I discuss the critical role that our viewing behavior plays in visual memory formation and retrieval and point to potential implications for machine vision models predicting the content of human memories.

Colour vision is based on circuit-level comparison of the signals from spectral distinct types of photoreceptors. In our own eyes, the presence of three types of cones enable trichromatic colour vision. However, many phylogenetically ‘older’ vertebrates have four or more cone types, and in almost all their cases the circuits that enable tetra- or possibly even pentachromatic colour vision are not known. This includes the majority of birds, reptiles, amphibians, and bony fish. In the lab we study neuronal circuits for colour vision in non-mammalian vertebrates, with a focus on zebrafish, a tetrachromatic surface dwelling species of teleost. I will discuss how in the case of zebrafish, retinal colour computations are implemented in a fundamentally different, and probably much more efficient way compared to how they are thought to work in humans. I will then highlight how these fish circuits might be linked with those in mammals, possibly providing a new way of thinking about how circuits for colour vision are organized in vertebrates.