Manual vs automated identification of English L2 suffixes and complex words across levels of proficiency

Abstract This report presents the preliminary evaluation of Morph , a web-based tool for the automatic counting of 51 English noun derivational suffixes in a controlled corpus of Mexican learners of English with proficiency levels from A2 to C1 on the Common European Framework of Reference for Languages (CEFR) scale. The evaluation consisted of a quantitative analysis of agreement between human annotators and Morph , a qualitative analysis to obtain the sources of disagreement, and finally, the evaluation of the tool’s efficiency using precision, recall, and F1 score metrics. Results displayed a high level of agreement between the human annotators and Morph . The main sources of disagreement were found to be human errors, learner misspellings, and automatic tagger issues. Finally, the efficiency metrics showed the tool to be effective in accurately identifying the target suffixes across proficiency levels, with a significant reduction in time and effort.