Reconstruction-based Methods Research Articles

Multi-Layer Feature Restoration and Projection Model for Unsupervised Anomaly Detection

The anomaly detection of products is a classical problem in the field of computer vision. Image reconstruction-based methods have shown promising results in the field of abnormality detection. Most of the existing methods use convolutional neural networks to build encoding–decoding structures to do image restoration. However, the limited receptive field of convolutional neural networks makes the information considered in the image restoration process limited, and the downsampling in the encoder causes information loss, which is not conducive to performing fine-grained restoration of images. To solve this problem, we propose a multi-layer feature restoration and projection model (MLFRP), which enables the restoration process to be carried out on multi-scale feature maps through a block-level feature restoration module that fully considers the detail information and semantic information required for the restoration process. We conducted in-depth experiments on the MvtecAD anomaly detection benchmark dataset, which showed that our model outperforms current state-of-the-art anomaly detection methods.

A novel Transformer-based model with large kernel temporal convolution for chemical process fault detection

Fault detection and diagnosis (FDD) is an essential tool to ensure safety in chemical industries, and nowadays, many reconstruction-based deep learning methods are active in fault detection. However, many algorithms still suffer from not ideal actual performance. Inspired by the core mechanism of Transformer and large kernel convolution, this paper proposes a novel model combining variate-centric Transformer with large kernel temporal convolution. Variate-centric Transformer depends on self-attention to capture the multivariate correlations of input data, and large kernel temporal convolution collects period information to summarize temporal features. A benchmark dataset Tennessee Eastman process (TEP) and experiment data from the microreactor process are used to test the performance of fault detection. Compared with other reconstruction-based methods, results demonstrate that our model achieves a higher fault detection rate and a lower detection latency, and shows a significant potential for process safety.

Multivariate time series anomaly detection via separation, decomposition, and dual transformer-based autoencoder

Multivariate time series usually have entangled temporal patterns and various anomaly types. Meanwhile, they often contain both continuous and discrete features. Many existing methods directly model correlations in complex multivariate time series to conduct anomaly detection. Decomposing time series into different components, such as the overall trend and fluctuations, can contribute to better extracting semantic information and detecting anomalies. Existing decomposition-based anomaly detection methods still have several limitations. First, they directly decompose all features without considering that discrete features are unsuitable for decomposition because they do not have trends or fluctuations. Second, they adopt the same networks for different components with different characteristics, limiting their ability to extract semantic information. Moreover, due to the nature of Transformers, existing reconstruction-based methods using Transformers rarely form information bottlenecks, reducing the differentiation between the reconstruction errors of normal data and anomalies. This paper proposes a multivariate time series anomaly detection method with separation, decomposition, and dual Transformer-based autoencoder (SDDformer). Different from existing methods, SDDformer separates continuous and discrete features and only decomposes continuous features into trend and residual components. Considering the different characteristics of different components, SDDformer adopts Crossformer and the vanilla Transformer as the backbone of two different autoencoders to reconstruct the trend and residual components. Information bottlenecks are better formed using an extra token as the latent variable between the encoder and the decoder. SDDformer regards reconstructing a discrete feature as a classification task and calculates Cross-Entropy as its reconstruction error. Three different metrics are adopted in this paper to compare SDDformer with a variety of typical anomaly detection methods on public data sets, and the experimental results prove that SDDformer can achieve state-of-the-art performance.

A 3D reconstruction-based method using unmanned aerial vehicles for the representation and analysis of road sections

Due to the fast growth of cities worldwide, roads are increasing daily, and pavement maintenance has become very heavy and costly. Despite all efforts made under the pavement management system to keep the road surface in good shape, several road sections need to be in better condition, which presents a danger for drivers and pedestrians. This paper proposes a novel pavement 3D reconstruction and segmentation approach using the structure from motion technique, unmanned aerial vehicle, and digital camera. The method consists of the 3D modeling of the road by using images taken from different perspectives and the structure from motion technique. In this method, points cloud is sampled and cleaned using statistical outlier removal and noise filters. After that, duplicated and isolated points are eliminated to retain only significant data. The normal road plane is estimated using the principal component analysis technique and the remaining points. This plan presents a root mean square less than 0.85 cm. Finally, distances from those points to the normal plane are calculated and clustered to segment the road into distressed and non-distressed areas. The proposed approach presents a similarity rate to the survey measurement passed 95%. It has demonstrated promising results and has the potential for further improvement by optimizing various steps.

Using Deep Learning to Detect Anomalies in On-Load Tap Changer Based on Vibro-Acoustic Signal Features

An On-Load Tap Changer (OLTC) that regulates transformer voltage is one of the most important and strategic components of a transformer. Detecting faults in this component at early stages is, therefore, crucial to prevent transformer outages. In recent years, Hydro Quebec initiated a project to monitor the OLTC’s condition in power transformers using vibro-acoustic signals. A data acquisition system has been installed on real OLTCs, which has been continuously measuring their generated vibration signal envelopes over the past few years. In this work, the multivariate deep autoencoder, a reconstruction-based method for unsupervised anomaly detection, is employed to analyze the vibration signal envelopes generated by the OLTC and detect abnormal behaviors. The model is trained using a dataset obtained from the normal operating conditions of the transformer to learn patterns. Subsequently, kernel density estimation (KDE), a nonparametric method, is used to fit the reconstruction errors (regarding normal data) obtained from the trained model and to calculate the anomaly scores, along with the static threshold. Finally, anomalies are detected using a deep autoencoder, KDE, and a dynamic threshold. It should be noted that the input variables responsible for anomalies are also identified based on the value of the reconstruction error and standard deviation. The proposed method is applied to six different real datasets to detect anomalies using two distinct approaches: individually on each dataset and by comparing all six datasets. The results indicate that the proposed method can detect anomalies at an early stage. Also, three alarms, including ignorable anomalies, long-term changes, and significant alterations, were introduced to quantify the OLTC’s condition.

ADA-GAD: Anomaly-Denoised Autoencoders for Graph Anomaly Detection

Graph anomaly detection is crucial for identifying nodes that deviate from regular behavior within graphs, benefiting various domains such as fraud detection and social network. Although existing reconstruction-based methods have achieved considerable success, they may face the Anomaly Overfitting and Homophily Trap problems caused by the abnormal patterns in the graph, breaking the assumption that normal nodes are often better reconstructed than abnormal ones. Our observations indicate that models trained on graphs with fewer anomalies exhibit higher detection performance. Based on this insight, we introduce a novel two-stage framework called Anomaly-Denoised Autoencoders for Graph Anomaly Detection (ADA-GAD). In the first stage, we design a learning-free anomaly-denoised augmentation method to generate graphs with reduced anomaly levels. We pretrain graph autoencoders on these augmented graphs at multiple levels, which enables the graph autoencoders to capture normal patterns. In the next stage, the decoders are retrained for detection on the original graph, benefiting from the multi-level representations learned in the previous stage. Meanwhile, we propose the node anomaly distribution regularization to further alleviate Anomaly Overfitting. We validate the effectiveness of our approach through extensive experiments on both synthetic and real-world datasets.

PatchCL-AE: Anomaly detection for medical images using patch-wise contrastive learning-based auto-encoder

Anomaly detection is an important yet challenging task in medical image analysis. Most anomaly detection methods are based on reconstruction, but the performance of reconstruction-based methods is limited due to over-reliance on pixel-level losses. To address the limitation, we propose a patch-wise contrastive learning-based auto-encoder for medical anomaly detection. The key contribution is the patch-wise contrastive learning loss that provides supervision on local semantics to enforce semantic consistency between corresponding input–output patches. Contrastive learning pulls corresponding patch pairs closer while pushing non-corresponding ones apart between input and output, enabling the model to learn local normal features better and improve discriminability on anomalous regions. Additionally, we design an anomaly score based on local semantic discrepancies to pinpoint abnormalities by comparing feature difference rather than pixel variations. Extensive experiments on three public datasets (i.e., brain MRI, retinal OCT, and chest X-ray) achieve state-of-the-art performance, with our method achieving over 99% AUC on retinal and brain images. Both the contrastive patch-wise supervision and patch-discrepancy score provide targeted advancements to overcome the weaknesses in existing approaches.

PS-NET: an end-to-end phase space depth estimation approach for computer-generated holograms.

In the present work, an end-to-end approach is proposed for recovering an RGB-D scene representation directly from a hologram using its phase space representation. The proposed method involves four steps. First, a set of silhouette images is extracted from the hologram phase space representation. Second, a minimal 3D volume that describes these silhouettes is extracted. Third, the extracted 3D volume is decomposed into horizontal slices, and each slice is processed using a neural network to generate a coarse estimation of the scene geometry. Finally, a third neural network is employed to refine the estimation for higher precision applications. Experimental results demonstrate that the proposed approach yields faster and more accurate results compared to numerical reconstruction-based methods. Moreover, the obtained RGB-D representation can be directly utilized for alternative applications such as motion estimation.

Industrial Product Surface Anomaly Detection with Realistic Synthetic Anomalies Based on Defect Map Prediction.

The occurrence of anomalies on the surface of industrial products can lead to issues such as decreased product quality, reduced production efficiency, and safety hazards. Early detection and resolution of these problems are crucial for ensuring the quality and efficiency of production. The key challenge in applying deep learning to surface defect detection of industrial products is the scarcity of defect samples, which will make supervised learning methods unsuitable for surface defect detection problems. Therefore, it is a reasonable solution to use anomaly detection methods to deal with surface defect detection. Among image-based anomaly detection, reconstruction-based methods are the most commonly used. However, reconstruction-based approaches lack the involvement of defect samples in the training process, posing the risk of a perfect reconstruction of defects by the reconstruction network. In this paper, we propose a reconstruction-based defect detection algorithm that addresses these challenges by utilizing more realistic synthetic anomalies for training. Our model focuses on creating authentic synthetic defects and introduces an auto-encoder image reconstruction network with deep feature consistency constraints, as well as a defect separation network with a large receptive field. We conducted experiments on the challenging MVTec anomaly detection dataset and our trained model achieved an AUROC score of 99.70% and an average precision (AP) score of 99.87%. Our method surpasses recently proposed defect detection algorithms, thereby enhancing the accuracy of surface defect detection in industrial products.

Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection.

Anomaly detection has recently gained increasing attention in the field of computer vision, likely due to its broad set of applications ranging from product fault detection on industrial production lines and impending event detection in video surveillance to finding lesions in medical scans. Regardless of the domain, anomaly detection is typically framed as a one-class classification task, where the learning is conducted on normal examples only. An entire family of successful anomaly detection methods is based on learning to reconstruct masked normal inputs (e.g. patches, future frames, etc.) and exerting the magnitude of the reconstruction error as an indicator for the abnormality level. Unlike other reconstruction-based methods, we present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. The proposed self-supervised block is extremely flexible, enabling information masking at any layer of a neural network and being compatible with a wide range of neural architectures. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss. Furthermore, we show that our block is applicable to a wider variety of tasks, adding anomaly detection in medical images and thermal videos to the previously considered tasks based on RGB images and surveillance videos. We exhibit the generality and flexibility of SSMCTB by integrating it into multiple state-of-the-art neural models for anomaly detection, bringing forth empirical results that confirm considerable performance improvements on five benchmarks: MVTec AD, BRATS, Avenue, ShanghaiTech, and Thermal Rare Event.

Depth map super-resolution via learned nonlocal model and enhanced local regularization

The appearance of consumer depth cameras makes it easy to obtain depth maps of a scene. However, the quality of depth maps is low and cannot satisfy the practical application. This paper proposes a novel depth map super-resolution method, which combines the advantages of the learning-based method and the reconstruction-based method, and incorporates both the learned nonlocal self-similarity prior and enhanced local geometry prior. On one hand, the nonlocal self-similarity prior is learned from the external examples by the patch group-based Gaussian mixture model (PG-GMM) and then used to explore the nonlocal self-similarity of depth maps. On the other hand, the local geometry prior is explored in both the gradient and spatial domain. For the gradient domain, the hyper-Laplacian prior is introduced to model the gradient distribution. For the spatial domain, a structure similarity-based auto-regressive model is proposed to acquire local correlation in a small area of a depth map. Finally, the learned nonlocal and enhanced local prior are cast into a unified optimization framework to acquire an estimation of the high-resolution image. Compared with state-of-the-art methods, experimental results indicate that our method can achieve better reconstruction performance in terms of both quantitative and qualitative evaluations.

Node and edge dual-masked self-supervised graph representation

Self-supervised graph representation learning has been widely used in many intelligent applications since labeled information can hardly be found in these data environments. Currently, masking and reconstruction-based (MR-based) methods lead the state-of-the-art records in the self-supervised graph representation field. However, existing MR-based methods did not fully consider both the deep-level node and structure information which might decrease the final performance of the graph representation. To this end, this paper proposes a node and edge dual-masked self-supervised graph representation model to consider both node and structure information. First, a dual masking model is proposed to perform node masking and edge masking on the original graph at the same time to generate two masking graphs. Second, a graph encoder is designed to encode the two generated masking graphs. Then, two reconstruction decoders are designed to reconstruct the nodes and edges according to the masking graphs. At last, the reconstructed nodes and edges are compared with the original nodes and edges to calculate the loss values without using the labeled information. The proposed method is validated on a total of 14 datasets for graph node classification tasks and graph classification tasks. The experimental results show that the method is effective in self-supervised graph representation. The code is available at: https://github.com/TangPeng0627/Node-and-Edge-Dual-Mask.

Anomaly detection via gating highway connection for retinal fundus images

Since the labels for medical images are challenging to collect in real scenarios, especially for rare diseases, fully supervised methods cannot achieve robust performance for clinical anomaly detection. Recent research tried to tackle this problem by training the anomaly detection framework using only normal data. Reconstruction-based methods, e.g., auto-encoder, achieved impressive performances in the anomaly detection task. However, most existing methods adopted the straightforward backbone architecture (i.e., encoder-and-decoder) for image reconstruction. The design of a skip connection, which can directly transfer information between the encoder and decoder, is rarely used. Since the existing U-Net has demonstrated the effectiveness of skip connections for image reconstruction tasks, in this paper, we first use the dynamic gating strategy to achieve the usage of skip connections in existing reconstruction-based anomaly detection methods and then propose a novel gating highway connection module to adaptively integrate skip connections into the framework and boost its anomaly detection performance, namely GatingAno. Furthermore, we formulate an auxiliary task, namely histograms of oriented gradients (HOG) prediction, to encourage the framework to exploit contextual information from fundus images in a self-driven manner, which increases the robustness of feature representation extracted from the healthy samples. Last but not least, to improve the model generalization for anomalous data, we introduce an adversarial strategy for the training of our multi-task framework. Experimental results on the publicly available datasets, i.e., IDRiD and ADAM, validate the superiority of our method for detecting abnormalities in retinal fundus images. The source code is available at https://github.com/WentianZhang-ML/GatingAno.

Adaptive Memory Networks With Self-Supervised Learning for Unsupervised Anomaly Detection

Unsupervised anomaly detection aims to build models to effectively detect unseen anomalies by only training on the normal data. Although previous reconstruction-based methods have made fruitful progress, their generalization ability is limited due to two critical challenges. First, the training dataset only contains normal patterns, which limits the model generalization ability. Second, the feature representations learned by existing models often lack representativeness which hampers the ability to preserve the diversity of normal patterns. In this paper, we propose a novel approach called Adaptive Memory Network with Self-supervised Learning (AMSL) to address these challenges and enhance the generalization ability in unsupervised anomaly detection. Based on the convolutional autoencoder structure, AMSL incorporates a self-supervised learning module to learn general normal patterns and an adaptive memory fusion module to learn rich feature representations. Experiments on four public multivariate time series datasets demonstrate that AMSL significantly improves the performance compared to other state-of-the-art methods. Specifically, on the largest CAP sleep stage detection dataset with 900 million samples, AMSL outperforms the second-best baseline by 4%+ in both accuracy and F1 score. Apart from the enhanced generalization ability, AMSL is also more robust against input noise.

Unsupervised Multimodal Anomaly Detection With Missing Sources for Liquid Rocket Engine.

To achieve reliable and automatic anomaly detection (AD) for large equipment such as liquid rocket engine (LRE), multisource data are commonly manipulated in deep learning pipelines. However, current AD methods mainly aim at single source or single modality, whereas existing multimodal methods cannot effectively cope with a common issue, modality incompleteness. To this end, we propose an unsupervised multimodal method for AD with missing sources in LRE system. The proposed method handles intramodality fusion, intermodality fusion, and decision fusion in a unified framework composed of multiple deep autoencoders (AEs) and a skip-connected AE. Specifically, the first module restores missing sources to construct a complete modality, thus advancing the secondary reconstruction. Different from vanilla reconstruction-based methods, the proposed method minimizes reconstruction loss and meanwhile maximizes the dissimilarity of representations in two latent spaces. Utilizing reconstruction errors and latent representation discrepancy, the anomaly score is acquired. At decision level, the model performance can be further enhanced via anomaly score fusion. To demonstrate the effectiveness, extensive experiments are carried out on multivariate time-series data from static ignition of several LREs. The results indicate the superiority and potential of the proposed method for AD with missing sources for LRE.

Noise-to-Norm Reconstruction for Industrial Anomaly Detection and Localization

Anomaly detection has a wide range of applications and is especially important in industrial quality inspection. Currently, many top-performing anomaly detection models rely on feature embedding-based methods. However, these methods do not perform well on datasets with large variations in object locations. Reconstruction-based methods use reconstruction errors to detect anomalies without considering positional differences between samples. In this study, a reconstruction-based method using the noise-to-norm paradigm is proposed, which avoids the invariant reconstruction of anomalous regions. Our reconstruction network is based on M-net and incorporates multiscale fusion and residual attention modules to enable end-to-end anomaly detection and localization. Experiments demonstrate that the method is effective in reconstructing anomalous regions into normal patterns and achieving accurate anomaly detection and localization. On the MPDD and VisA datasets, our proposed method achieved more competitive results than the latest methods, and it set a new state-of-the-art standard on the MPDD dataset.

Unsupervised industrial anomaly detection with diffusion models

Due to the limitations of autoencoders and generative adversarial networks, the performance of reconstruction-based unsupervised image anomaly detection methods are not satisfactory. In this paper, we aim to explore the potential of a more powerful generative model, the diffusion model, in the anomaly detection problem. Specifically, we design a Reconstructed Diffusion Models (RecDMs) based on conditional denoising diffusion implicit models for image reconstruction. To eliminate the stochastic nature of the generation process, our key idea is to use a learnable encoder to extract meaningful semantic representations, which are then used as signal conditions in an iterative denoising process to guide the model in recovering the image, while avoiding falling into an ”identical shortcut” to meaningless image reconstruction. To accurately locate anomaly regions, we introduce a discriminative network to obtain the pixel-level anomaly segmentation map based on the reconstructed image. Our experiments demonstrate the effectiveness of the proposed method, achieving a new state-of-the-art image-level AUC score of 98.1% and a pixel-level AUC score of 94.6% on the MVTec AD dataset, among all reconstruction-based methods. We also show the significant potential and promising future of our method on the challenging real-world dataset, the CHL AD dataset.

Anomaly-prior guided inpainting for industrial visual anomaly detection

Visual anomaly detection aims to identify areas where the appearance deviates from the normal distribution. Reconstruction-based methods detect anomalies through the analysis of reconstruction errors. In particular, Inpainting-based methods aim to reduce the potential for network generalization of anomalies by introducing random masks. However, larger anomalies may still be reconstructed, making them challenging to discriminate. Although enhancing the network’s long-distance modeling capability offers the possibility to mitigate the reconstruction of anomalous regions, there is a possibility that the reconstruction error for normal regions increases and the ability to discriminate small anomalies is reduced due to the random nature of the mask placement. To address this issue, we introduce a correction branch to modify the original reconstruction results obtained from the reconstruction network during testing. This method can alleviate the potential increase in reconstruction errors in normal regions caused by the randomness of the mask to some extent and enhance the network’s responsiveness to anomalous regions. We also propose a reconstruction network constructed with ConvNeXt blocks and the incorporation of the channel attention mechanism, which enhances the long-distance modeling ability, thereby improving the model’s predictive capacity for large masked areas, this network also improves the reconstruction of textures, thereby improving the network’s capacity to discriminate anomalies. The proposed method achieves competitive results on the challenging benchmark MVTec AD and BTAD datasets.

ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection

Anomaly detection in multivariate time series data is of paramount importance for large-scale systems. However, accurately detecting anomalies in such data poses significant challenges due to the need for precise data modeling capability. Existing forecasting and reconstruction-based methods struggle to address these challenges effectively. To overcome these limitations, we propose a novel anomaly detection framework named ImDiffusion, which combines time series imputation and diffusion models to achieve accurate and robust anomaly detection. The imputation-based approach employed by ImDiffusion leverages the information from neighboring values in the time series, enabling precise modeling of temporal and inter-correlated dependencies, reducing uncertainty in the data, thereby enhancing the robustness of the anomaly detection process. ImDiffusion further leverages diffusion models as time series imputers to accurately capture complex dependencies. We leverage the step-by-step denoised outputs generated during the inference process to serve as valuable signals for anomaly prediction, resulting in improved accuracy and robustness of the detection process. We evaluate the performance of ImDiffusion via extensive experiments on benchmark datasets. The results demonstrate that our proposed framework significantly outperforms state-of-the-art approaches in terms of detection accuracy and timeliness. ImDiffusion is further integrated into the real production system in Microsoft and observes a remarkable 11.4% increase in detection F1 score compared to the legacy approach. To the best of our knowledge, ImDiffusion represents a pioneering approach that combines imputation-based techniques with time series anomaly detection, while introducing the novel use of diffusion models to the field.

Anomaly Detection via Progressive Reconstruction and Hierarchical Feature Fusion.

The main challenges in reconstruction-based anomaly detection include the breakdown of the generalization gap due to improved fitting capabilities and the overfitting problem arising from simulated defects. To overcome this, we propose a new method called PRFF-AD, which utilizes progressive reconstruction and hierarchical feature fusion. It consists of a reconstructive sub-network and a discriminative sub-network. The former achieves anomaly-free reconstruction while maintaining nominal patterns, and the latter locates defects based on pre- and post-reconstruction information. Given defective samples, we find that adopting a progressive reconstruction approach leads to higher-quality reconstructions without compromising the assumption of a generalization gap. Meanwhile, to alleviate the network's overfitting of synthetic defects and address the issue of reconstruction errors, we fuse hierarchical features as guidance for discriminating defects. Moreover, with the help of an attention mechanism, the network achieves higher classification and localization accuracy. In addition, we construct a large dataset for packaging chips, named GTanoIC, with 1750 real non-defective samples and 470 real defective samples, and we provide their pixel-level annotations. Evaluation results demonstrate that our method outperforms other reconstruction-based methods on two challenging datasets: MVTec AD and GTanoIC.