Multi-animal pose estimation is essential for studying animals' social behaviors in neuroscience and neuroethology. Advanced approaches have been proposed to support multi-animal estimation and achieve state-of-the-art performance. However, these models rarely exploit unlabeled data during training even though real-world applications have exponentially more unlabeled frames than labeled frames. Manually adding dense annotations for a large number of images is costly and labor-intensive, especially for multiple instances. To reduce the need for laborious human labeling, we propose a semi-supervised framework for multi-animal pose estimation which is critical for sparsely-labeled problems. The proposed work is an extension of a semi-supervised single-animal pose estimation method, DeepGraphPose [1], to the multi-animal scenario in a non-trivial fashion. DeepGraphPose leverages the abundant spatiotemporal structures pervasive in unlabeled frames in behavior videos to enhance training, particularly in the regime of few training labels. However, it heavily relies on a uni-modal assumption on the output tensor of the neural network to predict poses for single animals. In order to construct the loss term for unlabeled frames, the model needs to read out differentiable pseudo pose locations from the uni-modal output, which cannot be easily achieved in the multi-animal setting. We propose to resolve the issue by introducing a separate network branch to generate differentiable pseudo pose locations, instead of reading directly from the output multi-modal tensor. The resulting algorithm provides superior multi-animal pose estimation results on three animal experiments compared to the state-of-the-art baselines and exhibits more predictive power in sparsely-labeled data regimes.