Real-time intraoperative depth estimation in transsphenoidal surgery using deep learning: A feasibility study

PURPOSE: Endoscopic endonasal and transcranial approaches are used for the resection of various pathological lesions in neurosurgery, especially pituitary adenomas, craniopharyngiomas, chordomas, or meningiomas. The video feed provided by endoscopes is generally two-dimensional, which can hinder depth perception. Thus, generating three-dimensional imaging without the need for special endoscopes using deep learning might be beneficial for enhanced intraoperative orientation. METHODS: DINOv2 is a pre-trained deep-learning model published by Meta in 2023. One of its capabilities is to estimate the depth in two-dimensional images. In this study, we explore the application of DINOv2 to the video feed of eight transsphenoidal endonasal surgeries. The results were evaluated for quality by both a senior neurosurgeon and a resident neurosurgeon. Furthermore, depth estimations from a randomly selected subset of 488 images taken from the videos were semi-quantitatively compared against manual segmentations for the estimation of deep, intermediate, and superficial areas. RESULTS: Using DINOv2, numeric depth maps were generated, and colormaps were created for depth visualization. Although these colormaps were not perfect, they aligned well with the subjective assessment of depth in the video feed by a senior neurosurgeon as well as a resident neurosurgeon. Semi-quantitative validation of the model's estimations yielded a mean overall DICE Similarity Index of 0.48. These semi-quantitative results should be interpreted with caution, as the cutoffs used for model depth predictions and manual segmentation are not standardized. CONCLUSIONS: Through the application of DINOv2, we were able to estimate depth in endoscopic imaging from transsphenoidal endonasal surgeries by generating numeric maps and depth colormaps. This illustrates the potential of deep learning-based depth estimations, which in the future could contribute to improving intraoperative orientation. It also highlights the opportunities in using artificial intelligence to augment endoscopic video feeds.