Resources
Authors & Affiliations
Varun Wadia, Chrystal Reed, Jeffrey Chung, Lisa Bateman, Adam Mamelak, Ueli Rutishauser, Doris Tsao
Abstract
Mental imagery refers to our remarkable capacity to generate percepts, emotions, and thoughts in the absence of external stimuli. This ability allows us to generate art, simulate actions and outcomes, remember previous experiences, and imagine new ones. Little is known about the mechanisms that enable mental imagery. While neuroimaging data shows that sensory areas are active during imagery (1–4), it remains unknown whether this is due to reactivation of the same neurons that support perception or separate circuitry located in the same regions. Here we examine the single-neuron mechanisms of visual imagery by recording neurons in the visual system, specifically ventral temporal cortex (VTC), of human patients implanted to localize their focal epilepsy (5) as they viewed and subsequently imagined visual objects. We found that, as in non-human primates, human VTC neurons showed robust visual responses (6–9), and ~80\% of the visually responsive neurons recorded were well modelled by linearly combining features in deep layers of a deep network trained to perform object classification (10). This “axis code” emphasizes the geometric picture that neurons project incoming stimuli, formatted as vectors in feature space, onto specific preferred axes and respond proportionally to the projection value. We used this code to decode objects and generate maximally effective synthetic stimuli. Finally, we asked patients to imagine (i.e., visualize) a subset of the objects, while recording responses of the very same VTC neurons. We found that a subset (~40\%) of axis-tuned neurons reactivated during imagery, and the imagined responses of individual neurons to specific objects were proportional to the projection value of those objects onto the neurons’ preferred axes. Our findings demonstrate for the first time that neurons in VTC support imagery by reactivating the visual code, providing single neuron evidence for the implementation of a generative model in the human brain.