Computational benefits of normalization in a circuit model

Normalization mechanism has often been used to describe neurons' sublinear responses to multiple stimuli in visual cortex. Neurons in the visual cortex exhibit highly heterogeneous degrees of normalization. Past work has proposed several hypotheses of the functions of normalization in both biological and artificial neural network. For example, normalization can maximize sensitivity in retina neurons, reduce redundancy in sensory cortex, and accelerate and stabilize the learning process in deep neural networks. However, the mechanism underlying the neuronal heterogeneity of normalization and its contribution to neural coding is not understood. Recently, we developed a biophysically-grounded two-layer spiking neuron circuit model which produces a range of normalization strengths across neurons. Our model reproduces the relationship between spike count correlation and normalization strength as observed in experimental recordings from Macaque visual cortex. In this work, we study the computational benefits of heterogeneous normalization strengths in our circuit model. We find that individual neurons in our model exhibit diverse and nonlinear response functions to multiple stimuli, even though the population-averaged rate remains mostly linear. The heterogeneous normalization strength allows the neural population to encode different contrast combinations of the presented images. In particular, neurons with stronger normalization are more sensitive to contrast differences of images; they exhibit much larger changes in firing rates when there is a contrast difference. In addition, neurons with stronger normalization encode more information of stimulus parameters per spike. Further, we demonstrate that neuronal heterogeneity in normalization improves the efficiency of information coding. Networks with more heterogeneous normalization strengths encode more information of contrast with the same number of neurons than networks with homogeneous normalization strengths. Together, our model sheds new light on the computational benefits of normalization and neural heterogeneity.