Topic: convolutional neural networks

Behavioral studies have demonstrated that certain task features reliably enhance classification performance for challenging visual stimuli. These include extended image presentation time and the valid cueing of attention. Here, I will show how convolutional neural networks can be used as a model of the visual system that connects neural activity changes with such performance changes. Specifically, I will discuss how different anatomical forms of recurrence can account for better classification of noisy and degraded images with extended processing time. I will then show how experimentally-obser

Spatial context plays an important role in visual processing and perception. A rich set of surround effects that involve nonlinear interactions between the center and surround have been found in Primary Visual Cortex (V1). A general summary is that surround suppression is prevalent and the amount of suppression depends on the visual similarity between the center and surround. In recent years, convolutional neural networks (CNNs) have successfully modeled neural behavior across the visual cortices. However, there have been limited studies addressing the multitude of contextual effects that have

An appealing hypothesis states that visual neurons in the brain are sensitive to the statistical properties of natural images. Neurophysiology and fMRI studies have tested neural sensitivity to images altered to exhibit either typical or atypical statistics. A general finding is that visual cortical neurons respond more strongly to images with natural statistics, and that this can vary across the visual hierarchy. Convolutional neural networks (CNNs) have achieved human-level performance in vision tasks and have been used as visual cortex models. However, CNN responses to images with altered s

The human visual cortex processes visual stimuli hierarchically. Early visual areas (V1, V2) of the ventral visual stream feed crude visual features (like orientation and edges) into later visual areas (V4, lateral occipital (Lat Occ), inferior temporal (IT)) that then encode complex visual features (like object form). Previous studies have reported a difference in fMRI responses between natural and urban landscapes in certain parts of the brain. Here we asked if this distinction in the representation of complex natural and man-made visual stimuli extends to the ventral visual stream and if st

Convolutional neural networks currently provide the best models of biological vision. However, their decision behavior, including the facts that they are deterministic and use equal number of computations for easy and difficult stimuli, differs markedly from human decision-making, thus limiting their applicability as models of human perceptual behavior. Here we develop a new neural network, RTNet, that generates stochastic decisions and human-like response time (RT) distributions, and also reproduces all foundational features of human accuracy, RT, and confidence. To test RTNet9s ability to pr

Brain-computer interface (BCI) applications enable innovative communication with and control of external devices for people with motor deficits by providing a bridge between brain signals and motor actions to replace and restore beneficial function. To measure neural response, Electrocorticography (ECoG) has been utilized as an effective modality for brain-computer interfaces (BCIs). Many studies have shown that both motor imagery and movement execution are located in the frontoparietal region of the brain (Hardwick et. al., 2018). In this study, we aim to demonstrate a shared representation o

Surround suppression is ubiquitous in visual processing and perception. Neurophysiology and modeling studies reveal a rich set of spatial context effects in the Primary Visual Cortex, whereby surrounding stimuli nonlinearly modulate (e.g., suppress) the responses of a center target stimulus. Studies suggest that the strength of surround suppression depends on the image statistics of the center and surround pattern. Namely, if the center and surround share similar statistics, the neural responses are more suppressed by the surround. This behavior has been connected to computational benefits suc

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching – and sometimes even surpassing – human accuracy on a variety of visual recognition tasks. In this talk, however, I will show that these neural networks and their recent extensions exhibit a limited ability to solve seemingly simple visual reasoning problems involving incremental grouping, similarity, and spatial relation judgments. Our group has developed a recurrent network model of classical and

Visual textures efficiently represent real-world information that is essential for perceptual tasks like pattern detection, object segmentation and classification. This information appears to be encoded in intermediate areas along the primate ventral visual stream (areas V2-V4), but how textural selectivity emerges at the circuit level, and whether the neural architectures that support texture processing are shared across mammalian species is currently unknown. We addressed these questions in the mouse by first examining the mouse’s perceptual ability to discriminate higher order texture stat

Classically, vision is seen as a cascade of local, feedforward computations. This framework has been tremendously successful, inspiring a wide range of ground-breaking findings in neuroscience and computer vision. Recently, feedforward Convolutional Neural Networks (ffCNNs), inspired by this classic framework, have revolutionized computer vision and been adopted as tools in neuroscience. However, despite these successes, there is much more to vision. I will present our work using visual crowding and related psychophysical effects as probes into visual processes that go beyond the classic fra

The last quarter century has provided extensive evidence that some regions of the human cortex are selectively engaged in processing a single specific domain of information, from faces, places, and bodies to language, music, and other people’s thoughts. This work dovetails with earlier theories in cognitive science highlighting domain specificity in human cognition, development, and evolution. But many questions remain unanswered about even the clearest cases of domain specificity in the brain, the selective engagement of the FFA, PPA, and EBA in the perception of faces, places, and bodies, r

We propose a novel method to predict contextual modulation of local field potential (LFP) gamma synchronization and multi-unit activity (MUA) in a single framework. Convolutional neural networks are state-of-the-art models of the primary visual cortex (V1). Sensory processing in V1 is determined by contextual modulation via feedback and recurrent connections. However, the computational goal and the neural implementation that shape V1 activity are not yet fully understood. We use transfer learning to use an object recognition convolutional neural network (VGG-16) to predict firing rates and gam