Grading Comparison of Standardized Patients and Pharmacist Faculty during a Mental Health Simulation.

OBJECTIVE: To analyze concordance between pharmacist evaluators and paid actor standardized patient (SP) evaluators when grading student pharmacists' performance in a simulated mental health patient interaction. METHODS: A mental health telehealth simulation was developed for second-year student pharmacists. Performance was concurrently assessed by an SP and a pharmacist evaluator. Differences in rubric-based assessments between the two evaluator types were subsequently analyzed as paired samples using the McNemar test. RESULTS: All fifty-nine enrolled students completed the simulation. There was strong agreement between SP and pharmacist evaluations for communication and professionalism metrics across the cohort. Discrepancies were observed on 8.6% of the rubric items and occurred most frequently on rubric items that required clinical judgment such as "recommended appropriate over-the-counter (OTC) options for the patient" (27% disagreement) or "no/limited use of medical jargon" (22% disagreement). Significant differences were seen with SP's reporting that the student did not complete some tasks, while the pharmacist evaluators reported the task as completed, such as "asked about duration of symptoms," and "recommended appropriate OTC/nonpharmacologic options for the patient". CONCLUSION: This study found strong concordance between SP and pharmacist evaluators for assessing observable communication behaviors and professionalism but identified significant differences in ratings for items requiring clinical judgment.