AI-driven disease classification from unstructured textual symptom descriptions: A multi-model NLP benchmarking study across categorical, UMLS-derived, and synthetic datasets.

This study presents an AI-driven framework for multi-class disease detection from unstructured, patient-reported textual symptom descriptions, combining natural language processing (NLP) with five machine learning classifiers: Neural Networks, Decision Trees, Logistic Regression, Multinomial Naïve Bayes, and Gradient Boosting. The core strength of the proposed framework lies in its unified, lightweight, and interpretable pipeline that integrates TF-IDF-based symptom text representation with classical and neural machine learning models, introduces a controlled text-generation mechanism using positive and negative templates with noise injection, and enables systematic cross-modal comparison across UMLS-derived, structured categorical, and synthetically generated textual symptom datasets. The framework is evaluated across three complementary dataset representations: (i) a UMLS symptom-disease knowledge base covering 149 diseases and 404 symptoms, (ii) a structured Kaggle categorical dataset comprising 42 disease classes and approximately 5000 samples described by 132 symptom features, and (iii) a synthetically generated text-based symptom dataset derived from the Kaggle data. All models are evaluated using accuracy, precision, recall, and F1-score. On the synthetically generated text-based test set under controlled conditions, the Neural Network achieves the highest performance, with an accuracy of 99.3% and F1-score of 99.3%, followed by Multinomial Naïve Bayes with an accuracy of 99.0%. On the Kaggle categorical dataset, Logistic Regression attains the highest accuracy of 98.9%, demonstrating that lightweight and interpretable models can perform competitively on structured symptom representations. It is important to note that the text dataset is synthetically generated and does not comprise authentic unstructured clinical patient narratives; results should therefore be interpreted with appropriate caution regarding real-world generalizability. Although high accuracy values are obtained, these results should be interpreted in light of dataset-level limitations, including class imbalance in the Kaggle categorical dataset, the synthetic nature of the text-based symptom dataset, and the absence of external validation on authentic patient-reported clinical narratives. Overall, the proposed pipeline offers a scalable foundation for telemedicine, mobile health triage, and low-resource clinical settings where patients express symptoms in natural language rather than selecting from predefined lists.