ITran: A novel transformer-based approach for industrial anomaly detection and localization

Abstract

Anomaly detection is currently an essential quality monitoring process in industrial production. It is often affected by factors such as under or over reconstruction of images and unclear criteria for feature distribution evaluation, thus making it challenging to improve detection performance. To solve the above problems, this paper proposes a novel transformer-based approach, Inductive Transformer (ITran) for industrial anomaly detection and localization, which utilizes a multi-layer pyramid structure and multi-level jump connections to extract multi-scale features of the data, putting the anomaly detection into the feature space and achieving more accurate industrial anomaly detection and localization results. It incorporates inductive bias and convolution operations into the Transformer which helps to break the myth of Transformer being “data hungry”. Compared with the common Transformers, ITran significantly reduces the computational cost and memory usage and makes it work well on small datasets. In addition, we basically eliminate the effect of positional embedding on the proposed Transformer model. Sufficient experiments have been conducted to validate global anomaly detection on three datasets MNIST, Fashion-MNST and Cifar-10, as well as local anomaly detection on the industrial datasets MVTec AD, Concrete Crack Image and BTAD. The proposed ITran achieves outstanding results on all the above datasets.