Xuewei Gao
The instructors rated the materials favorably for academic fidelity (M = 4.24), proficiency appropriateness (M = 4.36), and teachability (M = 4.31), with acceptable inter-rater reliability, ICC(2, k) = 0.84. Automated structural-complexity indicators showed limited separation ( η p 2 = 0 . 043 ), and leave-one-out discriminant analysis classified the intended proficiency labels at 11.1%, below the 33.3% balanced-task benchmark. Material differences were clearest in functional support, including glosses, sentence unpacking, rhetorical cues, claim-evidence notes, and critical prompts. Differentiated-AI exceeded unified-AI most strongly among high-proficiency learners (d = 1.40), followed by a moderate advantage among low-proficiency learners (d = 0.60) and a small difference among intermediate learners (d = 0.16). The omnibus reading-comprehension interaction identified this heterogeneity, F(4, 126) = 7.43, p < 0.001, and η p 2 = 0 . 191 . Supplementary process estimates were descriptive because the process indicators and outcomes were collected in the same session. Immediate unsupported application did not differ by material condition or interaction.
INTRODUCTION: Generative artificial intelligence (GenAI) can adapt English for Academic Purposes (EAP) reading materials by rewriting passages, adding support, or combining both. This study examined whether proficiency-sensitive GenAI adaptation chiefly changed passage-level structural complexity or text-embedded functional support while preserving academic fidelity.
METHODS: A role-prompted workflow combined barrier analysis, adaptation, fidelity checking, and validation. In a 3 × 3 between-subjects design (N = 135; n = 15 per cell), proficiency level (low, intermediate, and high) was crossed with material condition (original, unified-AI, and differentiated-AI). Three EAP instructors evaluated 15 anonymized material versions. Automated structural-complexity indicators, leave-one-out discriminant analysis, within-proficiency planned contrasts, and omnibus outcome models were used to assess material and learner outcomes.
RESULTS: The instructors rated the materials favorably for academic fidelity (M = 4.24), proficiency appropriateness (M = 4.36), and teachability (M = 4.31), with acceptable inter-rater reliability, ICC(2, k) = 0.84. Automated structural-complexity indicators showed limited separation ( η p 2 = 0 . 043 ), and leave-one-out discriminant analysis classified the intended proficiency labels at 11.1%, below the 33.3% balanced-task benchmark. Material differences were clearest in functional support, including glosses, sentence unpacking, rhetorical cues, claim-evidence notes, and critical prompts. Differentiated-AI exceeded unified-AI most strongly among high-proficiency learners (d = 1.40), followed by a moderate advantage among low-proficiency learners (d = 0.60) and a small difference among intermediate learners (d = 0.16). The omnibus reading-comprehension interaction identified this heterogeneity, F(4, 126) = 7.43, p < 0.001, and η p 2 = 0 . 191 . Supplementary process estimates were descriptive because the process indicators and outcomes were collected in the same session. Immediate unsupported application did not differ by material condition or interaction.
DISCUSSION: The findings locate GenAI-supported differentiation primarily in proficiency-specific support-layer design rather than broad changes in passage-level structural complexity. The strongest learner evidence came from comparisons conducted within the same proficiency level.