Objective measurement of AI literacy: Development and validation of the AI competency objective scale (AICOS)

As Artificial Intelligence (AI) becomes increasingly pervasive in everyday life, AI literacy is widely recognized as a set of essential competencies for navigating AI-driven environments safely, responsibly, and effectively. There is a growing need to assess this construct, for example, to inform targeted educational interventions. Although several measurement tools already exist, many show limitations regarding subjective data collection methods, differentiation between target groups, validity, and the integration of recent developments, such as Generative AI literacy. To address these limitations, this study introduces the AI Competency Objective Scale (AICOS) - an instrument grounded in a competency-oriented framework that enables a more objective assessment of AI literacy sub-competencies. AICOS draws on established theoretical models, integrates validated items from prior instruments, and explicitly incorporates Generative AI literacy as a distinct dimension. The AICOS provides a sound and comprehensive measure of AI literacy, and initial analyses indicate the potential for a modular structure. A preliminary short version of the scale has also been developed. Due to its methodological foundation, extensive validation, and incorporation of recent technological advancements, the test represents a valuable tool for scientific research and practical applications in educational and professional contexts. The AICOS contributes to the standardization of AI literacy assessment and supports the targeted development of AI-related competencies across diverse populations.