Key elements in student AI literacy development: Insights from psychometric network analyses

Student AI literacy comprises specific beliefs, knowledge, and skills as elements. Abundant research has synthesized the basic components of student AI literacy into constructs, however there exist misestimated key elements within these constructs. Lower-layer elements' interconnections across constructs and the temporal evolvement of student AI literacy remain inadequately quantified in existing content and latent structure analyses. This study conceptualized student AI literacy development according to the complex dynamic systems theory (CDST). Psychometric network analyses were employed to measure student AI literacy as a coherent system, identifying cores and outliers in its development. Two waves of data were collected from 2573 middle school students (51.5% girls) in 2023 and 2024 via a test. Cross-sectional and longitudinal networks of their AI literacy were constructed and compared at the element level. A content analysis of 25 elements' corresponding questions in the test was conducted for generalized discussion at the construct level. Finally, eight cores and six outliers emerged in the networks, along with twenty edges representing their crucial effects. These key elements aggregated diverse influences of three student AI literacy constructs. The “Process in AI” construct showed prominent contributions, whereas the “Knowledge of AI” construct had slight negative effects. The “Impact of AI” construct exhibited inconsistent influences. Additionally, there were conflicting relationships within and between constructs. This study introduces a novel theoretical construction and measurement strategy to assess student AI literacy and its multifaceted progression. It contributes concrete guidance for students' learning process and educators’ curriculum organization.