DeepLabCut 3.0: Efficient deep learning for single and multi-animal pose tracking and identification

Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. DeepLabCut was the first tool to innovate in this space, and since the initial publication (Mathis et al. Nature Neuroscience 2018) has been downloaded more than half a million times. While the code-base was actively developed, after 6 years it is time for a major release. Here we present DeepLabCut 3.0, which builds on DeepLabCut but with PyTorch as a deep learning framework. DeepLabCut 3.0 comes with more powerful network architectures for pose estimation, reaching state of the art performance for multi-animal interactions in crowded scenes and outperforming DeepLabCut2.0+ on single and multi-animal benchmarks. Furthermore, it integrates several pre-trained models (SuperAnimal models for mice, quadrupeds and faces) from the DeepLabCut Model Zoo that can be used on over 45 species, without additional labels. These models show excellent performance across several pose estimation benchmarks. We demonstrate how to fine-tune the models (if needed) on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If fine-tuned, SuperAnimal models are 10-100x more data efficient than prior transfer-learning-based approaches. Overall, DeepLabCut 3.0 provides tools for project management, deep learning for animal pose, and simple analysis. Additionally, its outputs plug into many downstream applications for behavioral and neural analysis, such as CEBRA, DLC2Kinematics, DLC2Action, Keypoint-MoSeq. Code: https://github.com/DeepLabCut/DeepLabCut.