Performance of ChatGPT-4, Gemini, and DeepSeek-V3 in answering the prompt-engineered questions from the Taiwan National Dental Technician Licensing Examination over a three-week period.

BACKGROUND/PURPOSE: Large language models (LLMs) have shown promise in answering professional examination questions. This study evaluated the performance of ChatGPT-4, Gemini, and DeepSeek-V3 in answering prompt-engineered questions selected from the 2023 Taiwan National Dental Technician Licensing Examination (TNDTLE) over a three-week period. MATERIALS AND METHODS: A total of 194 Chinese-language text-based multiple-choice questions were selected from the 2023 TNDTLE. The three LLMs answered these questions using a standardized prompt referring to 17 dental technology textbooks. Accuracy rates (ARs) were recorded at baseline and after one, two, and three weeks. Comparisons of ARs over time, among three LLMs, between basic and clinical subjects, and between Chinese-language questions with and without a prompt were performed by McNemar's or chi-square test, where appropriate. RESULTS: Baseline ARs ranged from 70.1% to 75.3% across the three LLMs. None of the three LLMs demonstrated significant improvement over a three-week period. Overall performance was comparable among models, with only one significant difference observed (Gemini outperforming ChatGPT-4 at one week, P = 0.033). ARs were generally higher for basic subjects than for clinical subjects, although most differences were not statistically significant. The prompt-engineering significantly improved the performance of ChatGPT-4 and Gemini at several time points (P < 0.05), whereas DeepSeek-V3 showed no significant improvement over a three-week period. CONCLUSION: ChatGPT-4, Gemini, and DeepSeek-V3 achieve moderate accuracy in answering prompt-engineered dental technician examination questions but do not exhibit significant improvement over a three-week period. The prompt-engineering can enhance performance for ChatGPT-4 and Gemini but not for DeepSeek-V3.