Reflection- and Curriculum-Based Instruction Tuning for Diabetes-Specialized Large Language Models: Model Development and Evaluation Study.

BACKGROUND: Effective diabetes management requires continuous interpretation of glycemic trends, personalized dietary guidance, and sustained patient education. Although large language models (LLMs) are increasingly being explored for health-related applications, existing general-purpose and biomedical models often struggle with diabetes-specific reasoning and instruction-following, limiting their reliability for domain-focused tasks such as clinical question answering and dietary recommendation tasks. OBJECTIVE: This study aimed to develop and evaluate a diabetes-specialized LLM optimized for diabetes-specific reasoning, instruction-following, and dietary recommendation tasks. METHODS: This was a model development and benchmark evaluation study. We developed a model-centric instruction refinement framework using instruction-following difficulty and reversed instruction-following difficulty to identify and replace suboptimal instruction-response pairs during instruction tuning. Curriculum-based instruction tuning was applied by sequencing instructions from lower to higher difficulty. The resulting diabetes-specialized model was evaluated across diabetes-related question answering, natural language inference, information extraction, summarization, clinical answer generation, and dietary recommendation tasks. Performance was compared with biomedical LLMs and general-purpose baselines using diabetes-specific benchmark subsets and simulation-based glycemic evaluation. RESULTS: Across diabetes-related benchmark tasks, the proposed model demonstrated improved performance in question answering, information extraction, and generative tasks. Ablation experiments showed that the full reflection- and curriculum-based instruction tuning strategy improved overall performance by 59.03% relative to the zero-shot LLaMA3.1 8B baseline. For dietary recommendation tasks, the proposed model achieved a 0.66% improvement in the Diet Quality Index-International score compared with ChatGPT (GPT-4). Simulation-based evaluation using the SimGlucose simulator further showed that meal plans generated by the proposed model resulted in a reduced postprandial glycemic burden, as measured by a lower incremental area under the curve, compared with GPT-4-generated meal plans. CONCLUSIONS: This study demonstrates that domain-specific instruction tuning can effectively adapt general-purpose LLMs for diabetes management. By combining reflection-based instruction replacement with curriculum-based instruction tuning, the proposed approach enhances instruction-following, reasoning capability, and dietary guidance for diabetes. The results highlight the potential of specialized LLMs to provide more reliable and clinically aligned support for diabetes-related decision-making and self-management, offering a promising direction for safe and effective AI deployment in chronic disease care.