TALK DETAILS
Comparing CNNs and the brain: sensitivity to images altered in the frequency domain
Alexander Claman — Xu Pan, Vanessa Aguiar-Pulido, Odelia Schwartz
28 September 2022
An appealing hypothesis states that visual neurons in the brain are sensitive to the statistical properties of natural images. Neurophysiology and fMRI studies have tested neural sensitivity to images altered to exhibit either typical or atypical statistics. A general finding is that visual cortical neurons respond more strongly to images with natural statistics, and that this can vary across the visual hierarchy. Convolutional neural networks (CNNs) have achieved human-level performance in vision tasks and have been used as visual cortex models. However, CNN responses to images with altered statistics have been less studied in comparison to cortical neurons (although see work on textures and spectrally matched noise). We performed three simulations focusing largely on spatial frequency alterations to assess the effects on CNNs (AlexNet and VGG16). We tried to match our simulations to the experimental designs as much as possible. (1) We altered the slope of the power spectrum, which is typically slightly larger than 1 in natural images. Most layers in the CNNs had a response peak around the natural slope, with some earlier layers revealing a high frequency bias. (2) We tested contrast effects with different power spectrum slopes. Some layers, especially late layers, showed higher contrast saturation for images matching the natural power spectrum slope. (3) We tested box scrambling, by randomly exchanging pixel values in an image. Early layers preferred scrambled images while late layers preferred intact images. All the simulations qualitatively aligned with the behavior found in visual cortices. Although the CNNs were trained on object classification, they captured the sensitivity to image statistics. This work in conjunction with continued analysis of the sensitivity of deep neural networks and visual cortex to targeted image statistics, examining their commonalities and differences, could help to foster a deeper understanding of both biological and artificial systems.
doi.org/10.57736/nmc-0d45-9bb5📋