Evaluating the reliability of collaborative problem solving performance among Chilean students: A Generalizability Study.

Drawing on Generalizability Theory, the study examines how communication, creativity, as well as critical thinking manifest in students' problem solving processes, and how these skills can be consistently captured in performance-based assessment. To this end, we analyze 321 solutions produced by 194 primary and secondary school students in Chilean mathematics classrooms, which were then scored by trained judges using a rubric with three overarching skills. The results show that scores vary systematically across grade levels and students (35-45% of the total variance), and reliability coefficients are around 0.75. Higher values can be achieved by sampling additional tasks per student. The results suggest that problem solving performance can be assessed with respect to a variety of applications in educational research and practice, provided that the problems adequately differentiate between low- and high-performing students. We discuss constraints and implications for implementing performance assessments with sufficient scoring precision.