GeoCM-Pose: geometry-aware monocular dental 2D/3D registration benchmarked against reference-assisted methods.

To develop GeoCM-Pose, a geometry-aware monocular dental 2D/3D registration method that predicts metric model-to-camera 6DoF pose from one image under weak texture, repetitive anatomy and partial visibility. Approach. GeoCM-Pose comprises cross-modal adaptation (CMA) and geometry-aware pose regression (GAPR). CMA uses fixed-Gaussian frequency separation and a high-frequency structure-preservation module with multi-scale edge extraction, spatial-channel gating and cross-resolution refinement. GAPR uses a ResNet-50/U-Net to predict dense object-coordinate maps; hierarchical geometric feature aggregation and geometry-aware channel refinement feed decoupled unit-quaternion and translation heads. Joint 20-point pose-matching and dense-coordinate losses train the network. The study comprised 30 in vitro training cases, five in vivo validation cases, five in vivo internal-test cases, 10 fully held-out Teeth3DS+ dental surface models and a 200-frame benchmark generated from a single three-dimensional dental model. Main results. GAPR/V0 generated pose estimates for all 200 query frames, with median reprojection, rotation and translation-vector errors of 2.156 px, 1.252° and 0.653 mm, respectively, and a <5 px pass rate of 91.5% across all attempted query frames. GAPR/V0 did not access a reference-image bank during inference. R20 SIFT and ORB each solved 197/200 query frames and achieved <5 px pass rates of 98.0% and 95.5%, respectively, across all attempts, indicating a trade-off among requirements for the reference-image bank, pose-estimation success rate and accuracy. Median GAPR/V0 inference latency, including image I/O, was 11.459 ms. Significance. GeoCM-Pose integrates cross-modal adaptation, dense object-coordinate supervision and metric pose regression for monocular dental 2D/3D registration. Further case-level evaluation using intraoral endoscopic images acquired from independent patient cases and additional image domains, together with robot-integrated experiments, is required to assess generalisation and system-level performance.