The importance of alignment: only highly targeted assessment items detect pre-post learning from a short biology lesson

Assessment instruments are critical to detecting student learning from instruction and interventions. When selecting an instrument, instructors face tension between using a published assessment that has validity evidence but might not be fully aligned to an instructional period versus constructing an assessment that has not undergone validity optimization but is highly aligned to the instruction. In this study, we examined how alignment between lesson content and assessment questions impacted conceptual learning detected via pre-post assessment. Introductory biology undergraduates took one of two assessments before and after a lesson on the central dogma of molecular biology: a subset of relevant questions from the Central Dogma Concept Inventory (“adapted concept assessment”; n = 39 students) or questions specifically written to target lesson content (“lesson-customized assessment”; n = 57 students). The adapted concept assessment did not detect learning gains, while the lesson-customized assessment detected learning gains. Changes in overall pre-post assessment scores for the lesson-customized assessment group were driven by select items and students with higher exam scores had larger gains. Our findings that proximal learning detected using an adapted concept assessment can differ from a lesson-customized assessment highlight the unique purposes and features of each instrument type. Concept assessments often cover broader areas and are developed with input from the larger community. Lesson-customized assessments can have direct alignment to instructional content, including in the representation of focal areas and familiarity of content scope. This research calls attention to the importance of considering alignment between instructional content and assessments while recognizing the constraints that limit assessment development.