MemFormer: A memory based unified model for anomaly detection on metro railway tracks

Abstract

Anomaly detection on metro railway tracks is crucial to maintain the safety of train transportation. Unsupervised methods require separate models for various scenarios, rendering them unsuitable for unified detection in the variable trackway scenario. The problem of “abnormal reconstruction” persists in the unified detection model, which preserves abnormal features in the output and hinders the model’s ability to recognize anomalies. In addition, real-time capability is still a huge challenge currently faced. In this study, we present a unified model, MemFormer, that employs a memory module, layer-wise normal queries, and a cascade convolution-based multi-head self-attention mechanism (CC-MHSA) to overcome the aforementioned issues. Firstly, a memory module is constructed to store diverse normal features that aid in the uniform modeling of a decision boundary for various scenarios. Secondly, we utilized the normal features existed in the memory module and layer-wise normal queries to optimize the attention mechanism, which suppresses the reconstruction of abnormal features. Thirdly, the proposed CC-MHSA benefits from cascaded convolution to improve feature representation and reduce parameter size, thereby reshaping self-attention calculation and reducing model inference time. Under the unified case, our method achieved detection accuracy of {99.2%, 97.5%} and localization accuracy of {99.7%, 97.1%}, respectively, on the metro trackway foreign object anomaly detection dataset and MVTec-AD dataset. The model infer speed is 47.6 FPS, exceeding that of the state-of-the-art alternatives.