Cross-trial alignment reveals a low-dimensional cortical manifold of naturalistic speech production

Finding a low-dimensional manifold of neural signals is crucial for understanding neural computation and developing a robust brain-computer interface (BCI). Latent variable models have been proposed to find such manifolds, and proved to be successful in various cognitive and behavioral tasks. However, natural speaking is a complex coordination of multiple motor tasks, and due to such complex nature, it has been challenging to find a low-dimensional and behaviorally relevant manifold of the speech cortex. We hypothesize that contents represented on the manifold should be consistent across repeated trials, and propose to disentangle the contents from the noisy signals by cross-trial alignment. To this end, a sequential autoencoder is trained to yield two temporal factors: signal factor to represent all the information including noise, and content factor to represent cross-trial consistent information. This disentanglement is achieved by contrastive alignment loss which is designed to maximize mutual information (MI) between the content factors and the signal factors, where the signal factors are sampled from other trials and temporally aligned by a jointly trained Aligner. We applied the method to high-density intracranial electrocorticography (ECoG) collected while speaking a set of sentences multiple times. Compared to previous models (e.g., LFADS), the content factor has a much lower effective dimensionality, and articulatory behavior can be best decoded from the factors learned by our approach. Furthermore, pre-/post-speech clusters and rotational dynamics during speaking are clearly identifiable on a single-trial manifold of the content factor, which is not as clear in other factors. Our method reveals a low-dimensional cortical manifold without requiring behavioral readouts, and enables us to characterize dynamic structures of complex naturalistic speaking behavior. These would contribute to understanding neural mechanisms of speech production and improving speech BCIs.