Classifying behaviors from animal-borne cameras using machine learning: automated identification of breathing events in sea turtles

Animal-borne cameras are increasingly used to study animal behavior. Here, we assessed the utility of three machine-learning models - Resnet-50 (3 epochs), Resnet-50 (10 epochs) and Vision Transformer (ViT) (3 epochs) - for identifying breathing behavior from animal-borne camera footage from green turtles (Chelonia mydas). The ViT model had mean Accuracy (97.2%), Precision (63.3%) and F1 (72.7%) scores that outperformed the Resnet-50 models, while all models had a Recall of >99.9%. Thus, the ViT model correctly identified almost all breathing frames although false positives (apnea frames labeled as breathing) were relatively common and led to an over-estimation of breathing rates. We conclude that ViT models are a promising solution for behavioral classification of animal-borne camera footage and even if not yet capable of the fully automated calculation of breathing events in sea turtles, they can still massively reduce the quantity of footage that needs to be manually checked and labeled.