Interpolation Methods for MediaPipe Hand Tracking Gaps: A Pose-Based Approach.

INTRODUCTION: Kinematics are a relevant measure of movement quality, especially for physically restricted patients. Therefore, consistent landmark detection is necessary. Automated landmarks detection already has existing solutions, but a problem remains: missing frames, where landmarks cannot be automatically detected. METHODS: This paper evaluates three interpolation strategies to overcome this issue and thus enable kinematic analysis. These strategies are linear interpolation, the Kalman filter, and the developed pose-based approach. Recordings were obtained using the marker-based VICON system. Participants (n = 3) performed three tasks inspired by a standard stroke assessment. This setup allows the precision of marker-less detection and interpolation to be evaluated. RESULTS: In comparison with Percentage Root Mean Square Error (PRMSE) and the Procrustes mean disparity, linear interpolation yields the best results. It achieved a mean PRMSE of 36.5 and a mean disparity of 0.266. The pose-based approach, presented here, follows closely with a mean PRMSE of 36.6 and a mean disparity of 0.273. The use of Kalman filtering resulted in a mean disparity of 0.314 and a PRMSE of 35.7. The results were tested positive for significance using the Kolmogorov-Smirnov test. DISCUSSION: In subsequent optical evaluations, the pose-based approach demonstrated superior performance; it was possible to observe that the linear approach sometimes lags behind the movement, while the pose-based interpolation follows the movement of the subject. Our results demonstrate that a reliable estimate with consistent landmarks is possible, thus enabling kinematic analysis.