An automated multimodal framework for capturing dynamic and micro-scale interactions between the built environment and vulnerable older adults

As global populations age, ensuring safe and independent mobility for older adults is a critical priority for urban planning and public health. Mobility in the built environment involves complex, moment-to-moment interactions, yet current assessment methods often rely on subjective surveys or static geospatial data that fail to capture dynamic human responses. Furthermore, these methods rarely address the specific needs of vulnerable individuals, such as those with knee osteoarthritis or a history of falls, who are most sensitive to environmental barriers. Hence, there is a lack of high-resolution, objective tools capable of quantifying how micro-scale built environmental features directly influence the physiological and kinematic states of these populations. Here we present an automated multimodal framework that integrates wearable motion sensors, physiological monitors, eye-tracking, and dual-perspective video to reconstruct a holistic profile of the walking experience. The framework employs SLAM and computer vision techniques to achieve precise tracking and environmental profiling. By synchronizing these inputs, the framework maps physiological and kinematic responses to specific built environment features. This overcomes the limitations of GNSS-denied settings and manual annotation, revealing micro-scale adaptations, including gait speed changes on slopes or during road crossings, that are often missed by conventional aggregate analyses. The resulting multimodal dataset captures the nuanced interplay between the built environment and vulnerable older adults. The computational pipelines and a specialized walkway surface material dataset are made publicly available. This work lays the foundation for large-scale, data-driven assessments of urban walkability, informing evidence-based design interventions to enhance safety and social engagement for aging societies.