Sushil Kumar Garg, Brayden Mau, Jeffery Hubers, Victor Arce, Nicole Hooper, Jennifer Lindquist, Sarah B. Harper, Ann B. Thayer, Piyush Mukherjee, Brenna Loufek, Joe Melnick, Lauren Rost, Steven A. Robbins, Rahul Gomes, Rajeev Chaudhary
INTRODUCTION: High-quality colonoscopy requires accurate risk stratification for surveillance per the 2020 US Multi-Society Task Force guidelines and adherence to 2024 American College of Gastroenterology/American Society for Gastrointestinal Endoscopy quality benchmarks. Both are operationally challenging in clinical practice. We developed and validated Colon-Pilot, a large language model-powered clinical decision support system using GPT-4o to automate and standardize both functions. METHODS: The system was evaluated in 2 operational modes: (i) a human-in-the-loop clinical decision support validation of surveillance recommendations for 596 colonoscopies, comparing concordance with 2020 US Multi-Society Task Force guidelines against expert consensus, and (ii) an automated administrative audit applying Colon-Pilot to 42,632 colonoscopies across the Mayo Clinic Health System to calculate 2024 American College of Gastroenterology/American Society for Gastrointestinal Endoscopy priority quality indicators. Recommendations were autogenerated unless predefined safety criteria triggered manual review. RESULTS: Colon-Pilot issued recommendations for 522 of 596 cases (87.6%) and flagged 12.4% for manual review. For automated cases, guideline-concordant accuracy was 97.5% (Cohen κ = 0.970) versus 69.7% (κ = 0.781) for original endoscopist recommendations. Discordant artificial intelligence (AI) cases (n = 13) most often recommended longer-than-appropriate intervals (62%). Applied to the enterprise data set, Colon-Pilot calculated performance exceeding 2024 targets: adenoma detection rate 49.8% (≥35%), sessile serrated lesion detection rate 17.7% (≥6%), bowel preparation adequacy 91.8% (≥90%), and cecal intubation rate 97.4% (≥95%). DISCUSSION: Colon-Pilot demonstrated high fidelity in applying surveillance guidelines and automated quality benchmarking, outperforming unassisted endoscopists in guideline adherence. By combining safety protocols with large-scale automated reporting, it offers a scalable solution for improving both efficiency and quality in colorectal cancer prevention.