Alzheimer’s diagnosis from EEG with reliable probabilities: subject-wise, leakage-free evaluation and isotonic calibration

Abstract Background Electroencephalography (EEG) provides a low-cost, non-invasive view of millisecond-scale brain dynamics; however, its clinical value depends on reliable probabilities and deployment-aware evaluation, rather than accuracy alone. Objective This study aimed to deliver a simple EEG approach for Alzheimer’s Diagnosis (AD) that returns calibrated subject-level probabilities and reports pre-specified clinical operating points, all under subject-wise, leakage-free validation. Methods Resting-state, eyes-closed EEG from OpenNeuro ds004504 was pre-processed (resampled to 128 Hz, 0.5–45 Hz band-pass, 50 Hz notch, average reference), segmented into 8-s epochs with 4-s overlap, and quality-controlled ( $$\ge$$ 20 valid epochs per subject). Wavelet Scattering Transform features were then extracted under two configurations ( $$J=6/7, Q=8$$ ), pooled to lobar regions with mild weights, and aggregated into subject-level statistics. Feature learning used an $$\ell _1$$ -penalized logistic selector followed by an $$\ell _2$$ -regularized logistic classifier, with performance estimated via 5-fold GroupKFold to produce subject-wise out-of-fold (OOF) logits. Base probabilities were calibrated using isotonic regression, combined through linear ensembling, and then recalibrated. We summarized discrimination (AUC, PR-AUC), calibration (Brier score, ECE), threshold behavior, and clinically oriented operating points (Sens@Spec, Spec@Sens); uncertainty was quantified using bootstrap confidence intervals derived from OOF predictions. Results After quality control, 59 subjects (31 AD/28 HC; 6,957 epochs) remained. The calibrated ensemble achieved an AUC of 0.930 and a PR-AUC of 0.931; the Brier score improved from 0.107 to 0.102, and the ECE decreased from 0.051 to $$\sim 0.000$$ . Bootstrap resampling ( $$N=1000$$ ) yielded mean AUC $$=0.932$$ (95% CI $$[0.864,\,0.980]$$ ). Thresholds $$t=0.380$$ (F1-optimal) and $$t=0.5$$ produced identical $$F_1=0.839$$ and accuracy $$=0.831$$ . Clinically, $$t=0.375$$ prioritized sensitivity (0.935), whereas $$t=0.667$$ prioritized specificity (1.00; no false positives). Conclusions The lightweight, interpretable EEG workflow produced reliable, calibrated probabilities under subject-wise, leakage-free evaluation and supported explicit clinical operating points. While external, multi-center validation remains necessary, these findings support probability-aware EEG decision support for AD.