ChatGPT’s impact on ESP writing proficiency and learner autonomy: An experimental study

This study investigates ChatGPT's efficacy in fostering autonomous ESP writing development across different proficiency levels, while identifying its strengths and limitations in enhancing specific writing competencies. A quasi-experimental pretest-posttest design was employed with 176 Saudi university students (A1-B1 proficiency) over 8 weeks. Standardized email-writing assessments measured progress in Content, Format/Structure, Grammar/Punctuation, and Clarity/Cohesion, with ANOVA and effect size analyses determining significant changes. Results showed significant improvements (p<.001) in rule-based skills: Content (A1: d=2.68; A2: d=1.56; B1: d=2.06) and Format/Structure (A1: d=2.59; A2: d=1.10; B1: d=1.18). Grammar improved for A1 (d=0.74) and A2 (d=0.58) but not B1 (p=.210). Clarity/Cohesion showed no significant gains (p>.05). The model explained 57-74% of variance (R²), with strongest effects for A1 learners. This study provides the first empirical evidence of ChatGPT's proficiency-dependent effectiveness in ESP writing, establishing (1) clear thresholds for AI's optimal utility across skill levels, (2) critical limitations in developing higher-order writing skills, and (3) a foundation for blended AI-human writing pedagogy in ESP contexts.