Social interactions play a major role in primates. They rely in part on our ability to extract relevant information from the voices of our conspecifics. fMRI studies have highlighted Temporal Voice Areas or ‘voice patches’ in the brain of humans, macaques and marmosets : these regions of cortex selectively respond to conspecific vocalizations, but little is known about how individual voice patch neurons encode vocal information, particularly on caller identity. The ‘Norm-based coding’ hypothesis from face perception research suggests that neuronal activity could be modulated by the acoustical distance between a vocalization and an “average” stimulus. To test it, we created an “average coo” by morphing between coos from 16 different macaques, and generated vocalization continua between individual coos and the average coo to parametrically manipulate distance to the average. Stimuli were presented to 2 macaques implanted with Utah arrays in their fMRI-localized voice patches while they performed a pure tone detection task. Results indicate that the spiking activity of many voice-selective neurons (173/239, 72%) is indeed modulated quasi-linearly by distance to mean at early latencies (0 to 150 ms after stimulus onset). However, we also observed an unsuspected rebound in activity specific to the average coo at later latencies (around 200 ms after stimulus onset) in a largely separate neuron population. These results suggest different temporal dynamics by distinct neuronal subpopulations in the encoding of vocal identity information.