Population encoding and decoding of frontal cortex during natural communication in marmosets

Communication is an inherently interactive behavior involving the coordinated exchange of social signals between two or more individuals. Studies exploring the neural basis of communication in the primate brain have traditionally focused on how these signals are processed or produced, leaving many facets of vocal behavior largely unexplored. One challenge has been limitations of traditional analyses for quantifying the often variable, dynamic nature of natural communication behaviors, such as conversational exchanges. Here we recorded the activity of single neurons in prefrontal and premotor cortex as freely behaving marmosets engaged in natural conversational exchanges. We applied different analysis approaches to determine how these substrates encode the various facets of this vocal behavior. Using peri-stimulus time histograms (PSTHs), we observed neurons responding to hearing calls and neurons showing a compensatory decrease when producing calls. We further performed a generalized linear model (GLM) based analysis. The predicted spike rate from GLM recapitulates observed PSTHs and the GLM kernels and coefficients yield more neurons in the population having a significant change in spike rate for communication events as well as state-related neurons that are modulated by the internal behavior states. By performing dimensionality reduction and clustering analysis on the GLM kernels and coefficients, we observed distinctive clusters in the population playing different functional roles: one cluster driving marmoset call production; one responding to only conversational calls; one responding to hearing calls. Last, we used the GLM framework to decode the animal’s communicating behavior from the neural activities. These results show that the GLM analysis applied here to a continuous natural vocal behavior revealed elements of primate frontal cortex activity that were not evident using more traditional analyses, and suggests that this method may be a powerful tool to better understand the neural basis of communication in the primate brain and other naturally occurring behaviors.