Where Are We and Where Are We Going? A GME Program Scorecard for Continuous Programmatic Improvement.

BACKGROUND: Graduate medical education (GME) leaders face challenges in evaluating program performance due to fragmented data systems and limited resources. Dashboards have been proposed, but few are scalable, participant-informed, and feasible without new infrastructure. OBJECTIVE: To develop and implement a centralized, participant-informed GME Program Scorecard (GPS) that enables continuous program evaluation and improvement using existing data systems without new data entry. METHODS: The GPS was developed at a large academic medical center in 2022-2023. A consensus development panel of 20 GME administrators identified 42 candidate metrics across 13 systems. Program directors (PDs) from the largest residency programs were surveyed (11; 9 responses, 81.8%) to rate each metric's value. The 17 most supported metrics were integrated into a Tableau-based dashboard. The GPS was piloted with 4 programs (3 Accreditation Council for Graduate Medical Education [ACGME]-accredited, 1 non-ACGME-accredited), involving 8 PDs and program coordinators. Feedback was collected via structured forms and a focus group. Metrics were rated on a 1 to 5 scale, with lower scores indicating higher priority. RESULTS: Pre-intervention survey results guided prioritization, with 92.9% of metrics receiving a mean score <3. Pilot users reported high acceptability and identified design refinements incorporated before institutional launch. Development required 2920 analyst hours and $1,600 in licenses. Since launching in January 2024, 309 users (61.4% of 503 potential users) have accessed the GPS with 3933 total views. CONCLUSIONS: The GPS demonstrated feasibility, requiring 2920 analyst hours and $1,600 in licenses, and with 61.4% of potential users accessing the dashboard within 22 months of launch.