Thinking Critically About Algorithms for Automated Detection of Behavior: 11 Guidelines for Social and Behavioral Scientists

Developmental psychologists are increasingly leveraging mobile and wearable sensors paired with machine learning and artificial intelligence (AI) to automatically detect the everyday behaviors and interactions theorized to drive development. These technologies provide an opportunity to capture learners' real-world experiences, with wide-ranging implications for basic science and intervention. However, many developmentalists lack the training to critically evaluate the accuracy of models used to automatically detect behavior and may not be aware of various challenges of implementing these approaches in real-world settings. To advance the next wave of research and innovation in this area, we provide readers with a set of 11 practical guidelines that will give researchers the critical perspective necessary to leverage or codesign systems in a way that is technically sound, ethically responsive, and practical. Our guidelines highlight common pitfalls and challenges with using AI for research and intervention, matched with best practices and practical recommendations for researchers working in this field. They cover the limits of model generalizability, recommendations for careful interpretation of accuracy statistics, the importance of real-world feasibility, ethical deployment, and interdisciplinary collaboration with sustained community engagement. Collectively, these guidelines provide a foundation for advancing the rigor, equity, and impact of tools for activity recognition in developmental science.