GS-SVIR: 3D Gaussian Splatting for inverse rendering with spatially varying illumination

Inverse rendering is widely used in digital twins, virtual asset generation, and VR/AR. When reconstructing scene illumination, predominant methods rely on environment maps, which simplify illumination into a distant uniform source, struggling to accurately model spatially varying effects. Inspired by the rapid development and representational power of 3D Gaussian Splatting (3DGS), we propose GS-SVIR, a novel inverse rendering framework that leverages 3DGS for the modeling of spatially varying illumination. The proposed framework is structured around three main parts. Firstly, we introduce a novel lighting representation termed ”Gaussian-emitter,” which models the scene illumination as a collection of 3D Gaussians in space, thereby effectively overcoming the limitation of environment maps in modeling spatially varying and near-field lighting. Secondly, to enable high-fidelity rendering under this lighting model, we employ a differentiable ray tracer that calculates direct illumination from the Gaussian emitters. Thirdly, we account for global illumination by approximating multiple-bounce indirect lighting using spherical harmonics coefficients stored within each Gaussian, which significantly enhances the realism of the reconstructed materials and lighting. Experimental results on benchmark datasets demonstrate that our method achieves an average improvement of 4% in material reconstruction accuracy and 3% in rendering quality compared to environment-map-based methods.