Comparative Analysis of Machine Learning Models for Emotion Prediction in Reviews of Large Language Model-Powered Applications

Purpose: This study compares the performance of conventional machine learning algorithms for multiclass emotion prediction in reviews of Large Language Model (LLM)-powered applications using a unified TF-IDF representation. Research Method: User reviews were collected from the Google Play Store for eight popular LLM applications through web scraping. After text preprocessing, reviews were labeled into five emotion categories (Happy, Sadness, Anger, Fear, and Surprise). Ten machine learning algorithms, namely Perceptron, KNN, Naïve Bayes, Logistic Regression, MLP, SVM (Linear and RBF), Random Forest, XGBoost, and LightGBM, were evaluated using Accuracy, Balanced Accuracy, Precision, F1-score, MCC, and ROC-AUC. Result and Discussion: XGBoost achieved the best performance, with 87.30% Accuracy, 77.84% Balanced Accuracy, 64.28% F1-score, and 0.5626 MCC. ROC-AUC values between 0.91 and 0.96 demonstrate excellent discrimination across all emotion classes. The superior performance of XGBoost indicates that gradient boosting effectively captures informative patterns from sparse TF-IDF features. Implication: The findings provide practical guidance for selecting efficient machine learning models for emotion analysis of LLM application reviews and support AI application improvement through user emotion understanding. Originality: This study presents a comprehensive benchmark of ten conventional machine learning algorithms using a large-scale multi-application LLM review dataset and a unified five-emotion classification framework.