Writing with AI, thinking with Toulmin: metacognitive gaps and the rhetorical limits of argumentation

Argumentative writing is widely recognized as a cornerstone of academic literacy, yet it remains underdeveloped among many undergraduates, particularly in contexts where dialogic reasoning and reflective strategies are not systematically taught. This exploratory mixed-methods study examined three interrelated dimensions of writing in Indonesian EFL settings: argumentative quality analyzed through Toulmin’s model, metacognitive competence measured by the Metacognitive Awareness Inventory (MAI), and AI literacy assessed with a purpose-designed questionnaire. Data were collected from 30 final-year students across two teacher-education universities. Among Toulmin’s elements, Rebuttal scored lowest (M = 1.87, SD = 0.43), indicating a persistent weakness in counterargumentation. Metacognitive awareness showed relative strength in monitoring and evaluation but continuing weaknesses in planning and strategy adjustment, while students reported using AI tools such as Grammarly and ChatGPT primarily for surface-level corrections with limited ethical or rhetorical reflection. This study is among the first to empirically map metacognitive gaps emerging from AI-assisted argumentative writing, identifying two types: a regulatory gap (misalignment between awareness and enactment) and a critical-AI gap (mismatch between tool use and rhetorical purpose). The findings advance an integrated conceptual framework linking argumentation, metacognition, and AI literacy in digitally mediated writing. Pedagogically and for assessment design, the study suggests that instruction in argumentation, metacognitive scaffolding, and critical AI literacy should be embedded together to cultivate reflective, rhetorically aware, and ethically grounded academic writing. Background • Argumentative writing is a cornerstone of academic literacy, yet it remains underdeveloped among EFL (English as a Foreign Language) students. • Generative AI tools (e.g., ChatGPT, Grammarly) are increasingly used by students, but primarily for surface-level corrections (grammar/paraphrasing) rather than critical reflection or rhetorical development. • Key challenges include difficulties in constructing rebuttals, limited metacognitive regulation, and low ethical awareness regarding AI use. Purpose of the Study • To analyze students’ argumentative quality using the Toulmin framework. • To measure metacognitive awareness with the Metacognitive Awareness Inventory (MAI). • To explore AI literacy through a purpose-designed questionnaire. Method • Design : Convergent mixed-methods (quantitative + qualitative). • Participants : 30 final-year undergraduates from teacher education programs in Indonesia. • Instruments : ○ Argumentative essays (800–1000 words, analyzed with the Toulmin model). ○ MAI (52 items, α = .87). ○ AI literacy questionnaire (3 domains: usefulness, ethics, frequency of use). Key Findings Argumentative Writing Quality • Moderate performance in Claims, Data, and Warrants. • Rebuttals were weakest (M = 1.87; 57% low). • Many students offered only token acknowledgment of opposing views without elaboration. Metacognitive Awareness • Stronger in Monitoring (M = 3.47) and Evaluation (M = 3.35). • Weaker in Planning (M = 3.29) and Debugging (M = 3.31) → gap between awareness and actual strategies. • Weak correlation (r = .32) between Monitoring and argumentative quality. AI Literacy • Most students used AI tools occasionally, mainly for grammar (83%) and paraphrasing (67%). • Less use for idea generation (27%) or structural feedback (23%). • Perceived usefulness (M = 3.40) exceeded ethical concern (M = 3.13). • Students expressed ambivalence: AI was seen as “helpful” but also as undermining authorship. Integrative Observations • Metacognitive gaps : students recognize weaknesses but fail to translate them into strategies. • Weak rebuttals align with low planning and debugging capacity. • Instrumental AI use reinforces limited dialogic reasoning in writing. Pedagogical Implications • Explicit instruction : integrate Toulmin model with focus on counterclaims and rebuttals. • Metacognitive scaffolding : apply goal-setting, reflection journals, and think-aloud protocols. • Critical AI literacy : train students to use AI reflectively rather than only for corrective purposes. Limitations & Future Directions • Small sample size (N = 30) from only two institutions. • Reliance on self-reported data → potential bias. • Future research should adopt longitudinal and experimental designs, with cross-context comparisons in Global South settings.