Social interactions in primates are possible through the ability to extract relevant information from voices, for example their identity. The anterior Temporal Voice Area (aTVA) is a region in the anterior temporal lobe of humans, macaques and marmosets specialized in the processing of voices, but the exact voice information represented by individual neurons in the aTVA remains obscure.Here we asked how aTVA neurons encode voice identity information, separating different identities and grouping together calls from a same identity. We implanted two rhesus macaques with high-density multi-electrode arrays in their fMRI-localized aTVA. Spiking activity was recorded during an auditory stimulation task in which we presented 50 natural stimuli including 5 different coo calls from each of 5 macaques, and 5 different calls from each of 5 humans.Of 160 auditory-responsive neurons, 77% were modulated by voice identity (one-way ANOVA). Representational Dissimilarity Matrices, capturing pairwise spiking activity differences between the stimuli, showed significant association with an ideal categorical model separating between identities and grouping different calls from each identity, from 50ms post-stimulus onset.A principal components analysis (PCA) applied to the mean population activity over time revealed that population responses to the same identity followed similar trajectories in the multidimensional state space, with the third PC showing marked differentiation of the different identities in the sustained response and allowing higher accuracy in the discrimination between identities than the other PCs. These results contribute to elucidating the mechanisms by which abstract representations of identities, allowing speaker recognition, emerge in the primate brain.