Limits of absolute vector magnetometry with nitrogen-vacancy centers in diamond

The nitrogen-vacancy (NV) center in diamond has become a widely used platform for quantum sensing. The four NV axes in monocrystalline diamond specifically allow for vector magnetometry, with magnetic-field sensitivities reaching down to fT / Hz . The current literature primarily focuses on improving the precision of NV-based magnetometers. Here, we study the experimental accuracy of determining the magnetic field from measured spin-resonance frequencies via solving the NV Hamiltonian. We derive exact, analytical, and fast-to-compute formulas for calculating resonance frequencies from a known magnetic-field vector, and vice versa (formulas for calculating the magnetic-field vector from measured resonance frequencies). Additionally, the accuracy of often-used approximations is assessed. Finally, we promote using the Voigt profile as a fit model to determine the linewidth of measured resonances accurately. An open-source Python package accompanies our analysis.