What Knowledge Transfers in Tabular Anomaly Detection? A Teacher–Student Distillation Analysis

Anomaly detection on tabular data is widely used in fraud detection, predictive maintenance, and medical screening. While heterogeneous ensembles combining multiple detection paradigms achieve strong performance, their computational cost limits deployment in latency-sensitive or resource-constrained environments. We propose KD-AnomalyNet, a teacher–student framework that distills anomaly knowledge from a high-capacity ensemble into a lightweight neural model for efficient inference. Beyond performance replication, we study how anomaly representations transfer during distillation. To this end, we introduce a noise perturbation analysis that serves as a diagnostic probe for representation stability without introducing additional trainable components. Experiments on ten benchmark datasets show that the distilled model preserves up to 98.5% of the teacher’s AUC-ROC on the nine capacity-sufficient datasets (84.7% mean retention across all ten datasets) while achieving 26–181× inference speedups. Our analysis reveals which forms of anomaly knowledge transfer reliably—global outliers (78% transfer) and isolation-based detection (88% retention)—and which degrade under compression—local outliers (20% transfer) and neighborhood-based detection (76% retention)—providing practical guidance for deploying distilled anomaly detectors.