ePoster

AI-assisted annotation of rodent behaviors: Collaboration of the human observer and SmartAnnotator software through active learning

Lucas Noldus, Elsbeth van Dam, Loes Ottink, Ruud Tegelenbosch, Marcel van Gerven
FENS Forum 2024(2024)
Messe Wien Exhibition & Congress Center, Vienna, Austria

Conference

FENS Forum 2024

Messe Wien Exhibition & Congress Center, Vienna, Austria

Resources

Authors & Affiliations

Lucas Noldus, Elsbeth van Dam, Loes Ottink, Ruud Tegelenbosch, Marcel van Gerven

Abstract

Although automated systems for rodent behavior annotation exist, specific behaviors are still scored by hand. We introduce a novel AI-assisted annotation tool, SmartAnnotator, which helps the researcher annotate behaviors through active learning. Active learning reduces annotation effort by automatically selecting behavioral events whose annotation will maximally improve classification. Instead of playing a video from start to end and simultaneously scoring behavior, the researcher is presented short video clips, each showing a single behavioral event. The events are derived from clustered low-level behavior features precalculated by EthoVision software. While the researcher is annotating clips, classifiers are trained on the annotated data, and when proven reliable, applied to all data. Clips with low classification certainty are presented to the researcher. This is more efficient than annotating everything and gives more consistent results as it avoids observer drift and unavoidable decision delay in manual scoring. We evaluated this method with mouse ‘stretched attend’ behavior in a dataset of five 5-minute videos. With 15% of the data labeled, we achieved a recall of 76% with 50% precision. Plotting the generated and true events shows that the general pattern over time is recovered. Our preliminary results are promising, but more extensive testing on a wider variety of behaviors is needed, as well as further improvement of classification performance. Pilot results indicate that classification benefits from improved correctness prediction. The AI-assisted approach is a ‘hybrid AI’ solution, where combining the skills of humans and machines yields better performance than using only one of both.

Unique ID: fens-24/ai-assisted-annotation-rodent-behaviors-de4456f6