Unsupervised Anomaly Detection Research Articles

Robust Unsupervised Anomaly Detection for Surface Defects Based on Stacked Broad Learning System

Robust Unsupervised Anomaly Detection for Surface Defects Based on Stacked Broad Learning System

Physical Invariant Subspace based Unsupervised Anomaly Detection for Internet of Vehicles

Physical Invariant Subspace based Unsupervised Anomaly Detection for Internet of Vehicles

Outlier-Probability-Based Feature Adaptation for Robust Unsupervised Anomaly Detection on Contaminated Training Data

Outlier-Probability-Based Feature Adaptation for Robust Unsupervised Anomaly Detection on Contaminated Training Data

Automated Anomaly Detection for Surface Defects by Dual Generative Networks With Limited Training Data

The manufacturing process in the pharmaceutical industry requires for using vessels or tanks which inner surfaces can undergo fouling and/or may develop visible defects that have to be cleaned away. Surface contamination can present itself in different forms, and there is no available image database of stains. In this paper, we propose to use an unsupervised anomaly detection approach which trains models on one or just a few images of clean surfaces. First, we propose an inpainting GAN with global and local generators to make full use of every single image and to reduce the overfitting caused by a limited number of images in the training dataset. Furthermore, we propose a periodic noise injection technique to increase the number of images for training and to improve the detection performance of the network. The experimental results demonstrate that the proposed network and noise injection technique achieve significant performance gains in the challenging task of anomaly detection on the surfaces of pharmaceutical equipment and other real-world industrial datasets. We also consider possible generalizations of the proposed method.

TUT: Template-Augmented U-Net Transformer for Unsupervised Anomaly Detection

TUT: Template-Augmented U-Net Transformer for Unsupervised Anomaly Detection

Research on deep neural network-based anomaly detection technology in high-dimensional data environment

Abstract With the popularization of information technology, the dramatic growth of data size, the significant rise in data dimensions, the increasing complexity of data types, and the diversity and complexity of the manifestations of anomalies all make anomaly detection more difficult. In this paper, we enhance the structure of the variational self-encoder in deep neural networks to maintain the benefits of anomaly detection technology, which relies on reconstruction error. Utilizing the high reliability of agent supervision to make up for the lack of detection performance of the deep self-encoder when the anomaly data dimension is high, a novel unsupervised anomaly detection model for high-dimensional and high anomaly rate data is proposed. Results from tests on several public datasets show that the UAD-ADC model does better than four other traditional anomaly detection methods in all of the datasets when looking at AUC values. On two datasets, Pendigits and Letter, the method improves by 0.3648 and 0.3004 over the lowest algorithm, and the experimental results confirm its effectiveness.

ADS-Bpois: Poisoning Attacks against Deep Learning-Based Air Traffic ADS-B Unsupervised Anomaly Detection Models

ADS-Bpois: Poisoning Attacks against Deep Learning-Based Air Traffic ADS-B Unsupervised Anomaly Detection Models

Low-Shot Unsupervised Visual Anomaly Detection via Sparse Feature Representation.

Visual anomaly detection is an essential component in modern industrial manufacturing. Existing studies using notions of pairwise similarity distance between a test feature and nominal features have achieved great breakthroughs. However, the absolute similarity distance lacks certain generalizations, making it challenging to extend the comparison beyond the available samples. This limitation could potentially hamper anomaly detection performance in scenarios with limited samples. This article presents a novel sparse feature representation anomaly detection (SFRAD) framework, which formulates the anomaly detection as a sparse feature representation problem; and notably proposes an anomaly score by orthogonal matching pursuit (ASOMP) as a novel detection metric. Specifically, SFRAD calculates the Gaussian kernel distance between the test feature and its sparse representation in the nominal feature space for anomaly detection. Here, the orthogonal matching pursuit (OMP) algorithm is adopted to achieve the sparse feature representation. Moreover, to construct a low-redundancy memory bank storing the basis features for sparse representation, a novel basis feature sampling (BFS) algorithm is proposed by considering both the maximum coverage and the optimum feature representation simultaneously. As a result, SFRAD incorporates both the advantages of absolute similarity and linear representation; and this enhances the generalization in low-shot scenarios. Extensive experiments on the MVTec anomaly detection (MVTec AD), Kolektor surface-defect dataset (KolektorSDD), Kolektor surface-defect dataset 2 (KolektorSDD2), MVTec logical constraints anomaly detection (MVTec LOCO AD), Visual anomaly (VISA), Modified national institute of standards and technology (MNIST), and CIFAR-10 datasets demonstrate that our proposed SFRAD outperforms the previous methods and achieves state-of-the-art unsupervised anomaly detection performance. Notably, significantly improved outcomes and results have also been achieved on low-shot anomaly detection. Code is available at https://github.com/fanghuisky/SFRAD.

Retracted: Unsupervised Anomaly Detection Based on Deep Autoencoding and Clustering

Retracted: Unsupervised Anomaly Detection Based on Deep Autoencoding and Clustering

Mixed noise-guided mutual constraint framework for unsupervised anomaly detection in smart industries

Large-scale sensor and data acquisition systems, integrated with deep learning methodologies, play a pivotal role in enhancing the sustainability and security of smart city environments, exemplifying the critical significance of anomaly detection techniques. Anomaly detection in complex industrial scenarios presents various challenges, such as intricate working environments, limited anomaly samples, and lack of a priori information. Unsupervised anomaly detection based on knowledge distillation enables anomaly detection using only normal samples. However, the similarity in structure between teacher and student models, along with identical input data flow, hampers accurate anomaly detection and localization. To address these issues, we propose MNMC, an unsupervised anomaly detection model consisting of a mixed noise generation module emulating real defects, a mutual constraint module, and an anomaly segmentation module. Firstly, to enhance the student network’s ability to learn robust features, we construct a hybrid noise model comprising dead-leaves noise and perlin noise. This generates features with structural texture and distributional characteristics closer to real anomalies. Secondly, we design a mutual constraint framework to further improve the learning ability of the student network for normal features by constraining representations containing only a single noise. Lastly, for the detection of anomalies at different scales, we propose a new evaluation metric based on equal importance of normal and anomalous regions. Through ablation experiments, we demonstrate the effectiveness of the simulated real defect generation module and the mutual constraints module. Performance experiments on the MVTec dataset show that our method achieves competitive results compared to the current state-of-the-art anomaly detection methods.

Unsupervised fabric defect detection based on multiscale image reconstruction and structural similarity assessment

AbstractFabric defect detection is a crucial aspect of the textile industry. Currently, deep learning methods have demonstrated exceptional performance in fabric defect detection tasks. However, their performance is greatly affected by the number of defect samples, which is a challenge to obtain during actual production. To address this issue, this article proposes an unsupervised anomaly detection method for fabric defects using image reconstruction networks. This method only requires defect‐free samples for training. During the training phase, the model compresses defect‐free samples to obtain a low‐dimensional manifold and reconstruct them. During the inference phase, the method assesses whether a sample is defective by calculating the reconstruction error between the input and output images, and locates the defect region by computing the difference in various patches. Furthermore, since fabric contains rich texture features, with high correlation between neighbouring pixels, a structure similarity index measure combined with mean absolute error is introduced to evaluate the reconstruction error, which enhances the model's representation ability for defect‐free samples. Additionally, considering the diverse texture backgrounds in fabric, a multiscale reconstruction module is designed to optimise the reconstruction effect. Experimental results demonstrate that compared with other related approaches, the proposed method achieves high accuracy (image‐based area under the curve (AUC) up to 98.2% and pixel‐based AUC up to 97.3%) on multiple datasets and has good generalisation ability for different fabric textures.

Bayesian active learning isolation forest (B-ALIF): A weakly supervised strategy for anomaly detection

Due to the high cost of labelling large datasets, unsupervised Anomaly Detection models have been widely applied in many applications. In many scenarios, Anomaly Detection is typically embedded in Decision Support Systems, where users can monitor complex systems with increased efficiency. However, due to the subjective or domain-specific definition of anomalies, unsupervised models tend to perform poorly as they are based on general definitions of anomaly that might not coincide with the end-user expectations. In many cases human tagging of small amount of data can be feasible: For such scenarios, strategies to properly embed such information in the model and to guide the tagging may be extremely useful. Indeed to solve this problem, scientists are proposing models able to interactively query the end-user, training the detector towards the expected anomaly definition while reducing the human labelling effort; such scenario is especially feasible when Decision Support Systems are in place, since users have simple ways to interact with the Anomaly Detection modules. In this context, a novel Active Learning algorithm is here proposed, called Bayesian Active Learning Isolation Forest (B-ALIF), that, starting from the solution provided by a popular detection model named Isolation Forest, it is able to query the user and update by means of a Bayesian strategy. B-ALIF employs the unsupervised solution provided by the Isolation Forest as its prior, and then improves its detection capabilities as the user interacts with it. Experimental results on real datasets show that, with a limited human effort, the proposed approach is able to learn faster than other state-of-art weakly supervised and supervised approaches.

Non-contact sensing for anomaly detection in wind turbine blades: A focus-SVDD with complex-valued auto-encoder approach

The occurrence of manufacturing defects in wind turbine blade (WTB) production can result in significant increases in operation and maintenance costs of WTBs, reduce capacity factors of wind farms, and occasionally lead to severe and disastrous consequences. Therefore, inspection during the manufacturing process is crucial to ensure consistent fabrication of composite materials. Non-contact sensing techniques, such as Frequency Modulated Continuous Wave (FMCW) radar, are becoming increasingly popular as they offer a full view – cross sectional analysis – of these complex structures during assembly and curing. In this paper, we enhance the quality assurance of WTB manufacturing utilising FMCW radar as a non-destructive sensing modality. Additionally, a novel anomaly detection pipeline is developed that offers the following advantages: (1) We use the analytic representation of the Intermediate Frequency signal of the FMCW radar as a feature to disentangle material-specific and round-trip delay information from the received wave. (2) We propose a novel anomaly detection methodology called focus Support Vector Data Description (focus-SVDD). This methodology involves defining the limit boundaries of the dataset after removing healthy data features, thereby focusing on the attributes of anomalies. (3) The proposed method employs a complex-valued autoencoder to remove healthy features and we introduces a new activation function called Exponential Amplitude Decay (EAD). EAD takes advantage of the Rayleigh distribution, which characterises an instantaneous amplitude signal. The effectiveness of the proposed method is demonstrated through its application to collected data, where it shows superior performance compared to other state-of-the-art unsupervised anomaly detection methods. This method is expected to make a significant contribution not only to structural health monitoring but also to the field of deep complex-valued data processing and SVDD application. The code and dataset will be made publicly available. The code and dataset are available here 11https://github.com/FrusqueGaetan/Focus_SVDD_EAD.

Self-supervised pseudo multi-class pre-training for unsupervised anomaly detection and segmentation in medical images.

Unsupervised anomaly detection (UAD) methods are trained with normal (or healthy) images only, but during testing, they are able to classify normal and abnormal (or disease) images. UAD is an important medical image analysis (MIA) method to be applied in disease screening problems because the training sets available for those problems usually contain only normal images. However, the exclusive reliance on normal images may result in the learning of ineffective low-dimensional image representations that are not sensitive enough to detect and segment unseen abnormal lesions of varying size, appearance, and shape. Pre-training UAD methods with self-supervised learning, based on computer vision techniques, can mitigate this challenge, but they are sub-optimal because they do not explore domain knowledge for designing the pretext tasks, and their contrastive learning losses do not try to cluster the normal training images, which may result in a sparse distribution of normal images that is ineffective for anomaly detection. In this paper, we propose a new self-supervised pre-training method for MIA UAD applications, named Pseudo Multi-class Strong Augmentation via Contrastive Learning (PMSACL). PMSACL consists of a novel optimisation method that contrasts a normal image class from multiple pseudo classes of synthesised abnormal images, with each class enforced to form a dense cluster in the feature space. In the experiments, we show that our PMSACL pre-training improves the accuracy of SOTA UAD methods on many MIA benchmarks using colonoscopy, fundus screening and Covid-19 Chest X-ray datasets.

Adaptive Memory Networks With Self-Supervised Learning for Unsupervised Anomaly Detection

Unsupervised anomaly detection aims to build models to effectively detect unseen anomalies by only training on the normal data. Although previous reconstruction-based methods have made fruitful progress, their generalization ability is limited due to two critical challenges. First, the training dataset only contains normal patterns, which limits the model generalization ability. Second, the feature representations learned by existing models often lack representativeness which hampers the ability to preserve the diversity of normal patterns. In this paper, we propose a novel approach called Adaptive Memory Network with Self-supervised Learning (AMSL) to address these challenges and enhance the generalization ability in unsupervised anomaly detection. Based on the convolutional autoencoder structure, AMSL incorporates a self-supervised learning module to learn general normal patterns and an adaptive memory fusion module to learn rich feature representations. Experiments on four public multivariate time series datasets demonstrate that AMSL significantly improves the performance compared to other state-of-the-art methods. Specifically, on the largest CAP sleep stage detection dataset with 900 million samples, AMSL outperforms the second-best baseline by 4%+ in both accuracy and F1 score. Apart from the enhanced generalization ability, AMSL is also more robust against input noise.

A generalizable unsupervised graph‐based approach for anomalous event detection in people living with dementia

AbstractBackgroundSensor‐based remote health monitoring of persons living with dementia (PLwD) can be used to monitor the progression of their condition with minimal intrusion. This helps minimize preventable hospital admissions, while allowing researchers to improve their understanding of dementia. Existing approaches for detecting anomalies in activity and behavior in PLwD are challenged by data noise, drift, label reliance, and poor explainability and generalizability due to wide inter‐individual variability.MethodWe propose and evaluate a solution that combines graph learning with the Contextual Matrix Profile (CMP), an ultra‐fast distance‐based discord detection technique. Daily household movement data collected via passive infrared sensors are used to generate multivariate CMP. This is encoded as a set of time‐evolving graphs. A self‐supervised graph model is trained to separate graphs by distance, so discordant time blocks show up as anomalous graphs. Differences between successive graph embeddings are subject to moving average thresholding to detect anomaly. We compare our method to four state‐of‐the‐art outlier detection algorithms, and demonstrate the generalizability of a distance‐based graph model trained on one cohort, to anomaly detection in unseen individuals.ResultGraph‐enhanced CMP models outperform state‐of‐the‐art algorithms, yielding 74% recall with a 5.7% alert rate and 88% household‐level validity when evaluated on 13,426 days’ data from 42 households with 145 agitation events, collected by the UK Dementia Research Institute between August 2019 and April 2022. Similar results are obtained for a 23‐participant, 5284‐day Falls cohort: 72% recall, 5.85% alert rate and 83% household‐level validity. Pretrained models yield similarly high performance for unseen households in both cohorts. Experiments on the publicly available PAMAP dataset reveal superior top‐k alert performance. Explainability is achieved via the CMP that is the source for graphs.ConclusionWe address the need for generalizable anomaly detection models for multivariate time series sensor data in remote health monitoring. While the CMP allows the ability to denoise and detect patterns at a household level, a graph model constructed in the distance space is generalizable across households. With higher sensitivity, fewer alerts than state‐of‐the‐art methods and inherent explainability, graph‐CMP models offer a clinically meaningful unsupervised anomaly detection technique for dementia and beyond.

A Process Mining-based unsupervised Anomaly Detection technique for the Industrial Internet of Things

Industrial Internet of Things (IIoT) applications in Industry 4.0 collect and process Time Series (TS) originating from heterogeneous sources. Many data-driven techniques have been proposed over the years for unsupervised collective Anomaly Detection (AD) to detect anomalous TS and improve quality of service. These techniques build statistical and/or behavioral models through data-driven algorithms often based on machine and deep learning. However, these algorithms may have black-box behavior, may require too much computational effort, and may not be trustworthy. In order to address these challenges, this paper proposes and evaluates an unsupervised AD technique that pre-processes noisy and complex TS and detects anomalous patterns. Pre-processing handles TS complexity through Feature Extraction and Dimensionality Reduction based on Autoencoders (AEs), whereas anomalous patterns are classified through Process Mining (PM), which unsupervisedly captures TS patterns and compares unknown behavior to such patterns for collective AD. We apply the technique to a scale replica of an assembly plant adopted in a smart automotive factory to evaluate our technique with respect to several configurations to analyze their impact on both detection and timing performance. Our contribution improves and extends the existing state-of-the-art work in the literature regarding the application of PM to IIoT for collective AD in TS. Moreover, it compares the obtained results with a baseline approach based on AEs.

Reconstruction-based anomaly detection for multivariate time series using contrastive generative adversarial networks

The majority of existing anomaly detection methods for multivariate time series are based on Transformers and Autoencoders owing to their superior capabilities. However, these methods are susceptible to overfitting when there is insufficient data. To address this issue, we propose a novel unsupervised anomaly detection framework, which seamlessly integrates contrastive learning and Generative Adversarial Networks. More concretely, we utilize data augmentation techniques that incorporate geometric distribution masks to expand our training data, thereby enhancing its diversity. Then, a Transformer-based Autoencoder is trained within a Generative Adversarial Network framework to capture the underlying distribution for normal points. Additionally, we incorporate a contrastive loss into our discriminator to effectively regulate the GAN and ensure good generalization. Finally, anomalies are detected based on reconstruction errors. Numerous experiments on five real-world datasets demonstrated that our proposed method can effectively mitigates overfitting issues and obtains superior performance compared to state-of-the-art approaches. In particular, our model could achieve an average improvement of 9.28% in Precision, 11.33% in Recall, and 11.73% in F1-score.

Sensing anomaly of photovoltaic systems with sequential conditional variational autoencoder

The market for urban distributed photovoltaics (DPV) is expected to take off in the next decade. However, these systems are often subject to complex urban contexts and sub-optimal conditions, requiring scalable and comprehensive solutions to detect their underperformances. In recent years, deep generative models (DGMs) have exhibited outstanding performance in the anomaly detection domain, dealing with generic high-dimensional time series data. Nevertheless, the existing applications of DGMs in the photovoltaic (PV) sector are still unable to account for environmental information, limiting their performance under various environmental conditions. This study proposes the Sequential Conditional Variational Autoencoder (SCVAE), which can cope with the sequential impacts of the environment on PV power generation. Using real-world data collected from 30 rooftop PV sites located across China, a data processing pipeline is developed to construct the training datasets which contain mostly normal samples for unsupervised SCVAE model training. This work also constructs a synthetic dataset with a wide variety of artificial anomalies in reference to the domain insights and engineering practice of DPV systems. After checking and refining by experts, the synthetic dataset can finally be used to validate the anomaly detection models. The results demonstrate that the SCVAE model outperforms existing state-of-the-art unsupervised anomaly detection models and can be effectively generalized to unseen PV sites. Moreover, the latent variables of SCVAE could be used to identify the type of DPV failure, thereby enabling more targeted diagnostics of anomaly mechanisms.

An Adversarial Time–Frequency Reconstruction Network for Unsupervised Anomaly Detection

Detecting anomalies in massive volumes of multivariate time series data, particularly in the IoT domain, is critical for maintaining stable systems. Existing anomaly detection models based on reconstruction techniques face challenges in distinguishing normal and abnormal samples from unlabeled data, leading to performance degradation. Moreover, accurately reconstructing abnormal values and pinpointing anomalies remains a limitation. To address these issues, we introduce the Adversarial Time–Frequency Reconstruction Network for Unsupervised Anomaly Detection (ATF-UAD). ATF-UAD consists of a time reconstructor, a frequency reconstructor and a dual-view adversarial learning mechanism. The time reconstructor utilizes a parity sampling mechanism to weaken the dependency between neighboring points. Then attention mechanisms and graph convolutional networks (GCNs) are used to update the feature information for each point, which combines points with close feature relationships and dilutes the influence of abnormal points on normal points. The frequency reconstructor transforms the input sequence into the frequency domain using a Fourier transform and extracts the relationship between frequencies to reconstruct anomalous frequency bands. The dual-view adversarial learning mechanism aims to maximize the normal values in the reconstructed sequences and highlight anomalies and aid in their localization within the data. Through dual-view adversarial learning, ATF-UAD minimizes reconstructed value errors and maximizes the identification of residual outliers. We conducted extensive experiments on nine datasets from different domains, and ATF-UAD showed an average improvement of 6.94% in terms of F1 score compared to the state-of-the-art method.