ePoster, FENS Forum 2024
DeepLabCut 3.0: Efficient deep learning for single and multi-animal pose tracking and identification
Niels Poulsen, Anastasiia Filipova, Shaokai Ye, Lucas Stoffl, Mu Zhou, Quentin Mace and 7 more
- Messe Wien Exhibition & Congress Center, Vienna, Austria
- Computer Vision, Deep Learning and
Abstract
Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. DeepLabCut was the first tool to innovate in this space, and since the initial publication (Mathis et al. Nature Neuroscience 2018) has been downloaded more than half a million times. While the code-base was actively developed, after 6 years it is time for a major release. Here we present DeepLabCut 3.0, which builds on DeepLabCut but with PyTorch as a deep learning framework. DeepLabCut 3.0 comes with more powerful network architectures for pose estimation, reaching state of the art performance for multi-animal interactions in crowded scenes and outperforming DeepLabCut2.0+ on single and multi-animal benchmarks. Furthermore, it integrates several pre-trained models (SuperAnimal models for mice, quadrupeds and faces) from the DeepLabCut Model Zoo that can be used on over 45 species, without additional labels. These models show excellent performance across several pose estimation benchmarks. We demonstrate how to fine-tune the models (if needed) on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If fine-tuned, SuperAnimal models are 10-100x more data efficient than prior transfer-learning-based approaches. Overall, DeepLabCut 3.0 provides tools for project management, deep learning for animal pose, and simple analysis. Additionally, its outputs plug into many downstream applications for behavioral and neural analysis, such as CEBRA, DLC2Kinematics, DLC2Action, Keypoint-MoSeq. Code: https://github.com/DeepLabCut/DeepLabCut.
Topics
- DeepLabCut 3.0, search World Wide
- PyTorch, search World Wide
- pose estimation, search World Wide
- multi-animal interactions, search World Wide
- SuperAnimal models, search World Wide
- DeepLabCut Model Zoo, search World Wide
- unsupervised video adaptation, search World Wide
- animal behavior, search World Wide
- Keypoint-MoSeq, search World Wide