Amnaj Prajong, Therdpong Daengsi
This study evaluates the performance of six generative artificial intelligence (AI) systems in solving Thai-language multiple-choice examinations in the subject of Programmable Logic Controllers (PLC), a core component of Industrial Electrical Engineering education. Six large language models (LLMs), including ChatGPT, Claude, DeepSeek, Gemini, Copilot, and Grok, were tested using fifteen sets of PLC examination questions. Statistical analysis was conducted using one-way ANOVA and two-sample t-Tests with Bonferroni correction to examine performance differences. In addition, effect size measures, including Eta-squared (η²) and Cohen‟s d, were calculated to assess the magnitude of the observed differences. The results show that ChatGPT achieved the highest mean score (77.27%), while DeepSeek followed closely (76.73%) and demonstrated the lowest standard deviation (±1.83%), indicating the most consistent performance across test sets. Claude also performed strongly (74.80%), whereas Gemini, Copilot, and Grok achieved similar mid-tier scores ranging from 72.40% to 72.73%. Although all LLMs achieved scores within the passing grade range, ANOVA confirmed statistically significant differences among systems (p-value is 0.0002). However, after applying the Bonferroni correction, only a subset of pairwise differences remained statistically significant, particularly between DeepSeek and several mid-tier LLMs, while the differences among ChatGPT, Claude, and DeepSeek were not statistically significant under the adjusted threshold. Effect size analysis further indicates that some of these differences represent meaningful practical variation in LLM performance. These findings indicate that contemporary LLMs demonstrate baseline comprehension of PLC concepts and can achieve passing-level performance in technical examinations conducted in a non-English language. The study contributes empirical evidence on AI performance in Thai-language technical assessments and highlights the potential role of generative AI as a complementary learning support tool in vocational and engineering education.