Rick H J A Volleberg, Doosup Shin, Ruben G A van der Waerden, Craig R Porter, Susan V Thomas, Fernando Sosa, Simone Saitta, Aysun Cetinyurek-Yavuz, Joske van der Zande, Thijs Luttikholt, Pierandrea Cancian, Xiaojin Gu, Leah Heil, Jos Thannhauser, Clara I Sanchez, Bram van Ginneken, Ivana Išgum, Allen Jeremias, Evan Shlofmitz, Richard A Shlofmitz, Ziad A Ali, Niels van Royen
AI-based fully automated evaluation of OCT images is feasible with performances consistent with interobserver variability in a real-world data set of consecutive patients, supporting generalizability of the proposed methodology.
BACKGROUND: Artificial intelligence (AI) allows automated evaluation of intracoronary optical coherence tomography (OCT) images. However, algorithms are mostly developed and validated on well-curated data sets, which may not represent real-world data. We sought to externally validate a previously developed algorithm performing full-vessel segmentation for OCT in an unselected consecutive real-world data set.
METHODS: This was a retrospective, single-center, external validation study comprising 100 consecutive patients undergoing clinically indicated OCT. A previously developed AI algorithm (OCT-AID) was used for automated pixelwise labeling of OCT images, distinguishing among lumen, guidewire artifact, intima, media, lipid plaque, calcium plaque, side branch, plaque rupture, thrombus, microvessel, and background. The AI-based predictions were compared on a frame level to the reference standard obtained through manual OCT image analysis by expert readers.
RESULTS: Among 2560 analyzable frames, the agreement between the automated OCT image analysis and the reference standard was excellent for calcified plaque identification (κ=0.88 [95% CI, 0.84-0.92]) and quantification (intraclass correlation coefficient values ranged between 0.79 and 0.93), with a performance close to interobserver variability. For lipid plaque identification and quantification, the model performance was reasonable (κ=0.68 [95% CI, 0.64-0.72]; intraclass correlation coefficient for lipid arc, 0.79 [95% CI, 0.76-0.81]; intraclass correlation coefficient for minimum fibrous cap thickness, 0.59 [95% CI, 0.55-0.63]) and largely superior to interobserver variability. The algorithm performance for low-prevalence features (e.g., plaque rupture) was limited.
CONCLUSIONS: AI-based fully automated evaluation of OCT images is feasible with performances consistent with interobserver variability in a real-world data set of consecutive patients, supporting generalizability of the proposed methodology.