Quantitative behavioral analyses have become an essential tool for understanding neural circuits. For example, recent studies have shown that ongoing “spontaneous” behaviors drive large fractions of neural activity in the mouse brain. However, the specific behavioral features encoded and their consistency across mice remain unknown. To study the neural representation of interpretable behavioral features, we recorded the activity of ~50,000 neurons in the mouse visual cortex during spontaneous behaviors. We used simultaneous five-camera video recordings to show that orofacial behaviors contain all of the neurally-related behavioral information. Focusing on the face, we tracked several key points from which we computed pose relationships. These behavioral features predicted a higher proportion of neural activity compared to previous models. To compare against mice, we developed a fast, generalist deep-learning model for tracking 13 distinct points on the mouse face recorded from arbitrary camera angles. The model obtained a median error of less than 3 pixels on test frames from a new mouse with novel camera views. The model was several times faster than state-of-the-art pose estimation tools, making it a powerful tool for closed-loop behavioral experiments. Next, we aligned facial key points across mice in order to train a universal model to predict neural activity from behavior. The universal mouse model could predict neural activity as well as a model fit to a single mouse, showing that neural representations of behaviors are conserved across mice. The latent states extracted from the universal model contained interpretable mouse behaviors, such as grooming and wincing. In summary, we developed a robust end-to-end framework for modeling neural activity based on orofacial behaviors, and found that neural representations of behavior are similar across mice.