From perception to precision: Vision-based mobile robotic manipulation for assembly screwdriving

Flexible manufacturing demands automation that is both precise and adaptable. However, tasks such as screwdriving are typically automated using costly, rigid robotic cells, making this approach impractical for low-volume, high-mix production. As a scalable solution, mobile manipulators offer a flexible alternative, but achieving the required precision for screwdriving remains challenging due to localization uncertainties. This paper addresses these limitations by presenting a vision-guided mobile robotic manipulation system that performs high-precision screwdriving using only monocular RGB imagery. The proposed pipeline integrates stationary and onboard cameras with perception algorithms for object identification and segmentation, pose estimation, and CAD-based screw hole localization, compensating for base misalignment and object placement variability. Experimental validation using ISO 9283 standard’s metrics demonstrates a translational accuracy between 0.21 mm and 0.50 mm across multiple screw positions. Additionally, the system achieves angular estimation errors as low as 0.07°to 0.20°, verifying its capability for sub-degree precision in orientation estimation. In 50 independent experiments involving a total of 400 screw insertions, the system achieved a 100 % success rate, confirming its reliability in practical conditions. These results confirm the feasibility of using RGB-only vision for precision screwdriving and highlight the mobile manipulation system’s scalability for real-world semi-structured manufacturing environments.