Explainable metaheuristic-optimized gradient boosting decision trees for slope stability prediction

Abstract Slope stability prediction remains challenging due to multi-factor interactions and the limited capacity of traditional methods to capture nonlinear behavior. Existing AI-driven approaches can show dataset-dependent performance and limited interpretability, which hinders adoption. This study proposes three novel hybrid models integrating metaheuristic optimization and explainable artificial intelligence (XAI) to address this challenge. Three metaheuristic algorithms, Alpha evolutionary algorithm (AEA), Sine cosine algorithm (SCA), and Arithmetic optimization algorithm (AOA), are employed to optimize the hyperparameters of gradient boosting decision trees (GBDT), forming hybrid models (AEA-GBDT, SCA-GBDT, AOA-GBDT). A comprehensive database of 436 slope cases with six inputs (H, β, γ, c, φ, and ru) and slope state was compiled. Five-fold cross-validation (CV) was used for all models to ensure fair evaluation and limit overfitting. The Friedman–Nemenyi test analysis identified AEA-GBDT as the top model. AEA-GBDT achieved superior performance with the highest accuracy (0.920), precision (0.934), recall (0.914), F1-score (0.924), Cohen’s kappa (0.840), and AUC (0.960). To contextualize performance, XGBoost, AdaBoost, Random Forest, and SVM were also evaluated. Shapley Additive explanations (SHAP) indicate that φ and c dominate the predictions, while ru exhibits nonlinear interactions with c. This method narrows the gap between black-box AI models and engineering transparency and provides interpretable, data-driven support for slope design and risk management. Highlights Developed three hybrid models (AEA-GBDT, SCA-GBDT, AOA-GBDT) integrating metaheuristic optimization with gradient boosting decision trees for slope stability prediction.Compiled a comprehensive database of 436 soil slopes combining real and simulated engineering cases.The AEA-GBDT model achieved superior performance with the highest accuracy (0.920), precision (0.934), recall (0.914), F1-score (0.924), Cohen’s kappa (0.840), and AUC (0.960).SHAP analysis enhanced interpretability, identifying cohesion (c) and internal friction angle (φ) as the dominant controlling factors.Independent validation on 16 loess slope cases confirmed excellent generalization and engineering applicability.