A memory and retrieval transformer-based unsupervised learning model for anomaly detection and segmentation

Unsupervised learning models have recently advanced anomaly detection, especially for complex vision-based datasets, but most still yield coarse anomaly masks with vague shapes and locations. To address these limitations, this paper presents U nsupervised S egmentation and A nomaly G radient I nterpretation ( USAGI ), a novel transformer-based unsupervised learning approach designed for accurate anomaly detection and segmentation. We propose two novel components, the Memory Transformer and Retrieval Transformer: the former builds a memory bank from normal features during training, while the latter retrieves and compares features during testing to enable fine-grained anomaly reconstruction and segmentation. USAGI achieves salient performance on the MVTec AD dataset with an AUROC of 98.2 % and on the VisA dataset (Visual Anomaly Dataset) with an AUROC of 99.5 %, 98.8 % on Real-IAD, and 96.4 % on MANTA, demonstrating its superior performance in anomaly detection and segmentation.