ePoster

Perceptography: Reconstruction of visual percepts induced by brain stimulation

Elia Shahbazi,Timothy Ma,Walter Scheirer,Arash Afraz
COSYNE 2022(2022)
Lisbon, Portugal

Conference

COSYNE 2022

Lisbon, Portugal

Resources

Authors & Affiliations

Elia Shahbazi,Timothy Ma,Walter Scheirer,Arash Afraz

Abstract

Local stimulation in high-level cortical visual areas perturbs the contents of visual perception. Understanding how the contents of perception are altered by cortical stimulation is necessary for characterizing visual hallucinations in mental disorders and developing visual prosthetics. Anecdotal verbal reports by human patients constitute the main body of evidence in this area of research. The absence of speaking ability in nonhuman primates severely limits the systematic and high-throughput study of stimulation-induced perceptual events. Here we introduce a novel methodology, perceptography, that allows taking pictures of complex visual events induced by optogenetic stimulation of macaque inferior temporal (IT) cortex. The animals started each trial by fixating on a computer-generated image. Halfway through the 1-second image presentation, we altered the image features for 200ms. A ~1x1mm area of the IT cortex was optogenetically stimulated in half of the trials at random for 200ms using an implanted LED array. The animals were rewarded for successful detection of stimulation trials by looking at one of the two subsequently presented targets. We hypothesized that image alterations that share common features with the stimulation-induced perceptual event increase the chance of false alarms (FA) by the animal. Under the hood, two learning systems were deployed to increase the probability of FA. Ahab, our feature extraction deep network, used the animals’ behavioral responses to guide DaVinci (a generative adversarial network) to achieve this goal. This closed-loop system created altered images that induced 55-85% FA rate, dramatically higher than the baseline 3-7% (cross-validated, p<0.01). We would like to name these images Perceptograms, given that the state of perceiving them is difficult for the animals to discriminate from the state of their IT cortex being stimulated. We have also shown that higher cortical illumination leads to more pronounced alterations in the resulting perceptograms.

Unique ID: cosyne-22/perceptography-reconstruction-visual-ab7f4989