Evaluating the Quality of Student-Created Multiple-Choice Questions against Faculty-Created Multiple-Choice Questions Using the Item Writing Flaws Evaluation Instrument

Problem-solving pedagogies are becoming more common for the instruction of science courses, and students frequently use online resources to facilitate their instruction. Learner-sourcing is one such pedagogical strategy whereby students design assessment items related to course content. This provides alternative learning opportunities, improved self-directed learning, and the development of course specific test banks. However, although studies have demonstrated how learner-sourcing can support student outcomes, there remains some reluctance from instructors to implement such approaches, citing the frequency of student misconceptions, lack of control of content, and poor-quality items. This study evaluates the quality of student-created multiple-choice (MC) items from a general chemistry 1 course as evaluated using the Item Writing Flaws Evaluation Instrument (IWFEI). These evaluations were then compared against faculty-created items that had been previously implemented on exams. The results from our data set demonstrate that student-created items contain comparable, if not fewer, item writing flaws than those created by faculty. We suggest that courses looking to use learner-sourcing pedagogies should invest in training students to identify item writing flaws, which could mitigate some of the concerns faculty have about the quality of student-created assessment items and provide opportunities for student-created items to support faculty generated exams.