Meryem Yılmaz Soylu, Jeonghyun Lee, Jui-Tse Hung, Christopher Cui, David Joyner
As Artificial Intelligence (AI) tools become increasingly embedded in higher education, their effective integration remains uneven, and their potential to improve learning outcomes is not yet fully realized. AI literacy has emerged as a key factor influencing whether students can engage effectively with these tools. This study investigates how students’ AI literacy and prior exposure to AI technologies influence their perceptions of SM, an interactive, AI-based formative assessment tool. Drawing on Self-Determination Theory and user experience research, we examine how AI literacy, perceived usability, and satisfaction relate to student engagement and perceived learning effectiveness. Data were collected from 309 undergraduates in Computer Science and Business courses using validated surveys. Partial least squares structural equation modeling revealed that AI literacy – particularly self-efficacy, conceptual understanding, and application skills – significantly predicts usability, satisfaction, and engagement. Usability and satisfaction, in turn, strongly predict perceived learning effectiveness. Prior AI exposure showed no significant effect. By clarifying the mechanisms through which AI literacy influences learning outcomes via usability and satisfaction, this study provides empirical evidence from a large, diverse undergraduate sample and offers actionable design recommendations for creating inclusive, motivating, and effective AI learning environments that account for varying literacy levels.