A Large Dataset of Macaque V1 Responses to Natural Images Revealed Complexity in V1 Neural Codes

Natural visual environments are full of complex and diverse patterns. Recent neurophysiological experiments based on artificial patterns have suggested that macaque V1 neurons’ receptive fields are more complex and diverse in feature selectivity (Tang et al 2017) rather than just oriented Gabor filters. However, that finding was based on extensive parametric artificial stimuli used and might be biased. Hence, the nature of the neural codes of macaque V1 neurons remains controversial. To resolve this question, we performed 2-photon calcium imaging on three macaque monkeys to obtain 1689 V1 neurons’ responses to 30K-50K natural images. This dataset allows us to characterize the neural code of V1 neurons more generally and comprehensively. Fitting CNN models to this dataset, we found: (1) data size matters – CNN models trained with a larger set of data can generalize better for predicting responses to images that are not in the training set, and more importantly, the receptive fields recovered become more complex and diverse with more and more data; (2) natural images matters – using the CNN models, we showed that complex and diverse tunings can be revealed by testing with natural images, but not easily with white noise stimuli, even at an extremely large sample size; (3) over-completeness better – we showed that overcomplete sparse coding theory, which predicts the development of more diverse and complex tunings, yields filters that fit V1 neurons’ responses better than that provided by standard sparse coding theory. These findings suggest that with big data, V1 neurons’ CNN models capture more accurately the neural code of V1 neurons, providing compelling evidence in support of the complexity and diversity of the neural codes, and overcomplete sparse coding theory. The results also demonstrate that these models can potentially be used as V1 neurons-in-silico for investigating the neural codes and processes in V1.