High-Fidelity Digital Twins for Bridging the Sim2Real Gap in LiDAR-Based ITS Perception

Sim2Real domain transfer offers a cost-effective and scalable approach for developing LiDAR-based perception (e.g., object detection, tracking, segmentation) in Intelligent Transportation Systems (ITS). However, perception models trained in simulation often underperform on real-world data due to distributional shifts. To address this Sim2Real gap, this paper proposes a high-fidelity digital twin (HiFi DT) framework that incorporates real-world background geometry, lane-level road topology, and sensor-specific specifications and placement. We formalize the domain adaptation challenge underlying Sim2Real learning and present a systematic method for constructing simulation environments that yield in-domain synthetic data. Evaluations are performed for the down-stream task of 3D object detection where off-the-shelf 3D object detectors are trained purely on HiFi DT-generated synthetic data and evaluated on real data. Our experiments show that the DT-trained models outperform the equivalent model trained on real data. To understand the gains, we quantify distributional alignment between synthetic and real data using multiple metrics at both raw-input and latent-feature levels. Results demonstrate that HiFi DTs substantially reduce domain shift and improve generalization across diverse evaluation scenarios. These findings underscore the significant role of digital twins in enabling reliable, simulation-based LiDAR perception for real-world ITS applications.