Engaging Choice Architecture: Interventions in the Emergency Medicine Standardized Letter (eSLOE 2.0) to Adjust Rater Behaviors.

BACKGROUND: Residency programs are utilizing standardized letters of evaluation with increasing frequency to obtain concise and discerning information about applicants. However, letters show a narrow distribution of ratings, leading to difficulties differentiating between applicants. OBJECTIVE: To determine if choice architecture, through 2 interventions on the electronic emergency medicine Standardized Letter of Evaluation 2.0 (eSLOE 2.0), can increase the spread of ratings on letters written for emergency medicine applicants. METHODS: On the eSLOE 2.0 Part B for the 2024-2025 application cycle, radio buttons were prepopulated to a "3" on a Likert scale of 1 to 5 (rather than blank), and a pop-up message prompted letter writers to provide narrative justification for ratings of "1" (lowest) or "5" (highest). Ratings of applicants after this intervention were compared with ratings from 2 prior application cycles. RESULTS: A total of 17 727 letters (5938 from 2024-2025; 11 789 from 2022-2024) were analyzed. Post-intervention, there was a decrease in ratings of "5" across all domains in Part B, with the absolute difference in the percentage of students rated "5" ranging from -12.3% to -19.7%. There was a post-intervention increase in percentage of "3" and "4" utilization across all domains ranging from +3.9 to +11.0 and +7.2 to +11.0 respectively. Ratings of "1" and "2" remained stable post-intervention. CONCLUSIONS: Prepopulating SLOE Part B ratings to 3 out of 5 and pop-up messages prompting letter writers to justify top or bottom ratings led to a shift in the distribution of scores and a reduction in the percentage of top ratings in the 2024-2025 cycle, as compared to the prior 2 cycles.