Visualizing surround suppression in deep convolutional neural networks

Surround suppression is ubiquitous in visual processing and perception. Neurophysiology and modeling studies reveal a rich set of spatial context effects in the Primary Visual Cortex, whereby surrounding stimuli nonlinearly modulate (e.g., suppress) the responses of a center target stimulus. Studies suggest that the strength of surround suppression depends on the image statistics of the center and surround pattern. Namely, if the center and surround share similar statistics, the neural responses are more suppressed by the surround. This behavior has been connected to computational benefits such as improving coding efficiency, and relates to highlighting salient image structure. Though Convolutional Neural Networks (CNNs) have been popular models for biological visual systems, it is unknown if they also have such surround suppression effects. We took advantage of the feature visualization method to visualize the surround pattern that is most suppressive to the center pattern. In detail, we came up with a two-step optimization method: first we found the optimal center pattern, and next the most suppressive surround pattern. Our results match the general understanding of surround suppression effects in the visual cortex. We found that the most suppressive surround is visually similar to the center, while the most facilitative surround is visually distinct from the center. We also found that the most suppressive surround can follow the change in the center, i.e. when presenting a non-optimal center. When the center color was altered, the most suppressive surround was also changed to match the center color. More surprisingly, when the center texture was changed entirely, some neurons in deeper layers could still follow the center texture. Our study not only suggests a new view towards the efficiency of CNNs, but also a new experimental paradigm that could be used in future neurophysiology studies.