Muhiddin Dervis, Anushree Burade, Michael Lanuti, Ibiayi Dagogo-Jack, Subba R Digumarthy
In this radiologist-adjudicated evaluation of AI-detected thoracic lymph node candidates, AI demonstrated high performance in station classification and measurement. AI also identified a greater proportion of pathologically sampled stations than were documented in routine radiology reports, suggesting a potential future role for AI in supporting systematic lymph node assessment and reporting.
PURPOSE: To evaluate the performance of an artificial intelligence (AI) system for detecting, classifying, and measuring AI-detected thoracic lymph node candidates on CT and compare AI detection with routine radiology reporting and histopathologic findings.
METHODS: In this HIPAA-compliant, IRB-approved multicenter study, 169 patients with operable thoracic malignancies underwent invasive mediastinal or hilar lymph node sampling and chest CT within 6 weeks (mean, 16 days ± 10). Patients were identified using a query of the EPIC database. CT images were processed using a commercially available AI that detected lymph nodes, measured bidimensional diameters, and assigned nodal stations. An expert thoracic radiologist reviewed all AI-generated candidates, with discrepancies resolved by a second radiologist to establish the reference standard. Performance metrics were calculated using a one-vs-rest framework. Radiology reports and pathology results were reviewed for lymph node detection and classification.
RESULTS: A total of 3,520 lymph nodal candidates. The AI achieved sensitivity of 96.1 %, specificity of 99.7 %, positive predictive value of 95.8 %, negative predictive value of 99.7 %, and accuracy of 99.5 % for station classification. Overall, 3,374 nodes (95.9 %) were correctly classified; 138 (3.9 %) were misclassified and 8 (0.2 %) were false-positive non-lymph-node detections. Lower performance was observed in stations 8 (sensitivity, 81.6 %) and 9 (positive predictive value, 81.5 %). After excluding non-lymph-node findings, 3,512 lymph nodes were analyzed; 3,203 (91.2 %) measured less than 10 mm. AI-radiologist agreement for size categorization was 100 %. Among 169 biopsied lymph nodes, routine radiology reports documented 36 (21.3 %), whereas AI detected 135 (79.9 %). AI detected 87.6 % of benign and 69.4 % of malignant nodes.
CONCLUSION: In this radiologist-adjudicated evaluation of AI-detected thoracic lymph node candidates, AI demonstrated high performance in station classification and measurement. AI also identified a greater proportion of pathologically sampled stations than were documented in routine radiology reports, suggesting a potential future role for AI in supporting systematic lymph node assessment and reporting.