AVATAR: AI Vision Analysis for Three-dimensional Action in Real-time

Quantification of animal behavior is critical to neuroscience, genetics, and ethology research. Despite the recent development of a deep-learning-based pose estimation model, the usage of those systems has been limited due to the various video recording environment and hardware systems. Here we introduce a novel open-field experimental hardware with multi-vision and a 3D real-time behavior analysis system, based on the efficient and rapid object-detection deep learning algorithm, named AVATAR (AI Vision Analysis for Three-dimensional Action in Real-time). Combining this system with an automated lickometer, we report the behavior patterns of hedonic licking behavior, seeming to be similar in human eyes but distinguishable in the machine-learning algorithm by comparing the water licking and sucrose licking behavior of the mice. We further introduced the real-time closed-loop optogenetic manipulation of the dopaminergic neurons in VTA (ventral tegmental area) during the open-field test and found the altered rearing behavior during the open-field test. These results suggest that the AVATAR system can be used to detect and analyze the unseen, yet significant behavior patterns of animals and easily combined with the various experimental setup, real-time closed-loop optogenetics, or even neural recordings.