Unsupervised Anomaly Detection Method Research Articles

Generating and Reweighting Dense Contrastive Patterns for Unsupervised Anomaly Detection

Recent unsupervised anomaly detection methods often rely on feature extractors pretrained with auxiliary datasets or on well-crafted anomaly-simulated samples. However, this might limit their adaptability to an increasing set of anomaly detection tasks due to the priors in the selection of auxiliary datasets or the strategy of anomaly simulation. To tackle this challenge, we first introduce a prior-less anomaly generation paradigm and subsequently develop an innovative unsupervised anomaly detection framework named GRAD, grounded in this paradigm. GRAD comprises three essential components: (1) a diffusion model (PatchDiff) to generate contrastive patterns by preserving the local structures while disregarding the global structures present in normal images, (2) a self-supervised reweighting mechanism to handle the challenge of long-tailed and unlabeled contrastive patterns generated by PatchDiff, and (3) a lightweight patch-level detector to efficiently distinguish the normal patterns and reweighted contrastive patterns. The generation results of PatchDiff effectively expose various types of anomaly patterns, e.g. structural and logical anomaly patterns. In addition, extensive experiments on both MVTec AD and MVTec LOCO datasets also support the aforementioned observation and demonstrate that GRAD achieves competitive anomaly detection accuracy and superior inference speed.

Personalized anomaly detection in PPG data using representation learning and biometric identification

Photoplethysmography (PPG) signals, typically acquired from wearable devices, hold significant potential for continuous fitness-health monitoring. In particular, heart conditions that manifest in rare and subtle deviating heart patterns may be interesting. However, robust and reliable anomaly detection within these data remains a challenge due to the scarcity of labeled data and high inter-subject variability. This paper introduces a two-stage framework leveraging representation learning and personalization to improve anomaly detection performance in PPG data. The proposed framework first employs representation learning to transform the original PPG signals into a more discriminative and compact representation. We then apply three different unsupervised anomaly detection methods for movement detection and biometric identification. We validate our approach using two different datasets in both generalized and personalized scenarios. Our results demonstrate significant improvements: for movement detection, in the generalized scenario, AUCs improved from barely 0.5 to above 0.9 with representation learning. Importantly, inter-subject variability was substantially reduced, from around 0.4 to below 0.1. In the personalized scenario, AUCs became close to 1.0, with variability further reduced to below 0.05, indicating the effectiveness of both representation learning and personalization for anomaly detection in PPG data. Similar enhancements were observed in biometric identification, emphasizing how our approach can minimize inter-subject variability and enhance PPG-based health monitoring systems.

Encoder-Decoder Contrast for Unsupervised Anomaly Detection in Medical Images.

Unsupervised anomaly detection (UAD) aims to recognize anomalous images based on the training set that contains only normal images. In medical image analysis, UAD benefits from leveraging the easily obtained normal (healthy) images, avoiding the costly collecting and labeling of anomalous (unhealthy) images. Most advanced UAD methods rely on frozen encoder networks pre-trained using ImageNet for extracting feature representations. However, the features extracted from the frozen encoders that are borrowed from natural image domains coincide little with the features required in the target medical image domain. Moreover, optimizing encoders usually causes pattern collapse in UAD. In this paper, we propose a novel UAD method, namely Encoder-Decoder Contrast (EDC), which optimizes the entire network to reduce biases towards pre-trained image domain and orient the network in the target medical domain. We start from feature reconstruction approach that detects anomalies from reconstruction errors. Essentially, a contrastive learning paradigm is introduced to tackle the problem of pattern collapsing while optimizing the encoder and the reconstruction decoder simultaneously. In addition, to prevent instability and further improve performances, we propose to bring globality into the contrastive objective function. Extensive experiments are conducted across four medical image modalities including optical coherence tomography, color fundus image, brain MRI, and skin lesion image, where our method outperforms all current state-of-the-art UAD methods. Code is available at: https://github.com/guojiajeremy/EDC.

Unsupervised Deep Anomaly Detection for Industrial Multivariate Time Series Data

With the rapid development of deep learning, researchers are actively exploring its applications in the field of industrial anomaly detection. Deep learning methods differ significantly from traditional mathematical modeling approaches, eliminating the need for intricate mathematical derivations and offering greater flexibility. Deep learning technologies have demonstrated outstanding performance in anomaly detection problems and gained widespread recognition. However, when dealing with multivariate data anomaly detection problems, deep learning faces challenges such as large-scale data annotation and handling relationships between complex data variables. To address these challenges, this study proposes an innovative and lightweight deep learning model—the Attention-Based Deep Convolutional Autoencoding Prediction Network (AT-DCAEP). The model consists of a characterization network based on convolutional autoencoders and a prediction network based on attention mechanisms. The AT-DCAEP exhibits excellent performance in multivariate time series data anomaly detection without the need for pre-labeling large-scale datasets, making it an efficient unsupervised anomaly detection method. We extensively tested the performance of AT-DCAEP on six publicly available datasets, and the results show that compared to current state-of-the-art methods, AT-DCAEP demonstrates superior performance, achieving the optimal balance between anomaly detection performance and computational cost.

Unsupervised fabric defect detection based on multiscale image reconstruction and structural similarity assessment

AbstractFabric defect detection is a crucial aspect of the textile industry. Currently, deep learning methods have demonstrated exceptional performance in fabric defect detection tasks. However, their performance is greatly affected by the number of defect samples, which is a challenge to obtain during actual production. To address this issue, this article proposes an unsupervised anomaly detection method for fabric defects using image reconstruction networks. This method only requires defect‐free samples for training. During the training phase, the model compresses defect‐free samples to obtain a low‐dimensional manifold and reconstruct them. During the inference phase, the method assesses whether a sample is defective by calculating the reconstruction error between the input and output images, and locates the defect region by computing the difference in various patches. Furthermore, since fabric contains rich texture features, with high correlation between neighbouring pixels, a structure similarity index measure combined with mean absolute error is introduced to evaluate the reconstruction error, which enhances the model's representation ability for defect‐free samples. Additionally, considering the diverse texture backgrounds in fabric, a multiscale reconstruction module is designed to optimise the reconstruction effect. Experimental results demonstrate that compared with other related approaches, the proposed method achieves high accuracy (image‐based area under the curve (AUC) up to 98.2% and pixel‐based AUC up to 97.3%) on multiple datasets and has good generalisation ability for different fabric textures.

Non-contact sensing for anomaly detection in wind turbine blades: A focus-SVDD with complex-valued auto-encoder approach

The occurrence of manufacturing defects in wind turbine blade (WTB) production can result in significant increases in operation and maintenance costs of WTBs, reduce capacity factors of wind farms, and occasionally lead to severe and disastrous consequences. Therefore, inspection during the manufacturing process is crucial to ensure consistent fabrication of composite materials. Non-contact sensing techniques, such as Frequency Modulated Continuous Wave (FMCW) radar, are becoming increasingly popular as they offer a full view – cross sectional analysis – of these complex structures during assembly and curing. In this paper, we enhance the quality assurance of WTB manufacturing utilising FMCW radar as a non-destructive sensing modality. Additionally, a novel anomaly detection pipeline is developed that offers the following advantages: (1) We use the analytic representation of the Intermediate Frequency signal of the FMCW radar as a feature to disentangle material-specific and round-trip delay information from the received wave. (2) We propose a novel anomaly detection methodology called focus Support Vector Data Description (focus-SVDD). This methodology involves defining the limit boundaries of the dataset after removing healthy data features, thereby focusing on the attributes of anomalies. (3) The proposed method employs a complex-valued autoencoder to remove healthy features and we introduces a new activation function called Exponential Amplitude Decay (EAD). EAD takes advantage of the Rayleigh distribution, which characterises an instantaneous amplitude signal. The effectiveness of the proposed method is demonstrated through its application to collected data, where it shows superior performance compared to other state-of-the-art unsupervised anomaly detection methods. This method is expected to make a significant contribution not only to structural health monitoring but also to the field of deep complex-valued data processing and SVDD application. The code and dataset will be made publicly available. The code and dataset are available here 11https://github.com/FrusqueGaetan/Focus_SVDD_EAD.

THE ADJUSTED HISTOGRAM-BASED OUTLIER SCORE - AHBOS

Histogram is a commonly used tool for visualizing data distribution. It has also been used in semi-supervised and unsupervised anomaly detection tasks. The histogram-based outlier score is a fast unsupervised anomaly detection method that has become more popular because of the rapid increase in the amount of data collected in recent decades. Histogram-based outlier score can be computed using either static or dynamic bin-width histograms. When a histogram contains large gaps, the dynamic bin-width approach is preferred over the static bin-width approach. These gaps in a histogram usually occur as a result of various distributions in real data. When working with a static bin-width histogram, gaps can be utilized to acquire better distinction between outliers and inliers. In this study, we propose an adjusted version of the histogram-based outlier score named adjusted histogram-based outlier score, which considers neighboring bins prior to density estimation. Results from a simulation study and real data application indicate that the adjusted histogram-based outlier score yields a better performance not only in the simulated data but also for various types of real data.

A periodic anomaly detection framework based on matrix profile for condition monitoring of planetary gearboxes

As the core power transmission component of mechanical equipment, planetary gearboxes unavoidably experience faults due to harsh working environments and continuously heavy loads. Therefore, to ensure safe operation of the equipment and reduction of maintenance costs, condition monitoring of planetary gearboxes is critical. Anomaly detection provides a potential approach to overcome the limitations of traditional signal processing technologies and deep learning methods. However, since the monitoring signal of planetary gearboxes contains various harmonics and periodic impulses, current unsupervised anomaly detection methods are difficult to achieve a satisfactory result for the monitoring signal. To overcome the above problem, a novel periodic anomaly detection framework is put forward for planetary gearboxes condition monitoring based on matrix profile (MP). In this work, a periodic anomaly detection method, namely matrix profile guided multiple residuals average (MPGMRA), is first introduced to detect periodic impulses caused by the local damage. Then, to better extract the weak periodic impulses, an adaptive parameter selection strategy based on impulse-to-noise ratio (INR) is proposed. At last, both simulated signal and experimental data from a planetary gearbox are analyzed to demonstrate the applicability of the proposed framework, and the correlation kurtosis (CK) is introduced to evaluate the magnitude of the periodic fault component. The results manifest that this framework is able to recognize the condition of planetary gearboxes successfully and the performance based on CK can be improved by 5.4 times.

A Novel Warning Identification Framework for Risk-Informed Anomaly Detection

Cyber-physical systems are taking on a permanent role in the industry, such as in oil and gas or mining. These systems are expected to perform increasingly autonomous tasks in complex settings removing human operators from remote and potentially hazardous environments. High autonomy necessitates a more extensive use of artificial intelligence methods, such as anomaly detection, to identify unusual occurrences in the monitored environment. The absence of data characterizing potentially hazardous events leads to disruptive noise displayed as false alarms, a common anomaly detection issue for hazard identification applications. Contrastingly, disregarding the false alarms can result in the opposite effect, causing loss of early indications of hazardous occurrences. Existing research introduces simulating and extrapolating less represented data to expand the information on hazards and semi-supervise the methods or by introducing thresholds and rule-based methods to balance noise and meaningful information, necessitating intensive computing resources. This research proposes a novel Warning Identification Framework that evaluates risk analysis objectives and applies them to discern between true and false warnings identified by anomaly detection. We demonstrate the results by analyzing three seismic hazard assessment methods for identifying seismic tremors and comparing the outcomes to anomalies found using the unsupervised anomaly detection method. The demonstrated approach shows great potential in enhancing the reliability and transparency of anomaly detection outcomes and, thus, supporting the operational decision-making process of a cyber-physical system.

A Multivariate KPIs Anomaly Detection Framework With Dynamic Balancing Loss Training

Anomaly detection on multivariate KPIs (Key Performance Indicators, such as CPU utilization, sockets status, and HTTP requests per second) is of utmost importance to the systems’ reliability. Unsupervised methods have been of considerable interests and have significantly progressed due to their superior effectiveness. However, the state-of-art unsupervised anomaly detection methods still suffer from high false or missed alarm rates. To this end, in this paper, we propose MM, a practical Multivariate KPIs anomaly detection framework following the principles of Multi-task learning with the proposed dynamic balancing loss function. To capture KPIs’ characteristics to the most extent, we simultaneously train multiple sequential autoencoders with different connections based on a designed semi-Random Connection Recurrent Neural Network (sRC-RNN). These autoencoders can be treated as different reconstruction tasks while training. Furthermore, we propose a dynamic loss function to adaptively balance the tasks’ weights. Extensive experiments show that MM outperforms the state-of-art unsupervised multivariate KPIs anomaly detection algorithms and achieves an average F1-score of 0.95 on two public machine-level KPIs datasets and 0.96 on an internal container-level KPIs dataset.

Preserving microgrid sustainability through robust islanding detection scheme ensuring cyber-situational awareness

Along with the numerous environmental and operational benefits of renewable energy sources (RESs) integration into modern distribution networks, maintaining system security in these networks during challenging phenomena, especially unintentional islanding can still be problematic in some cases. This paper puts forward a sensitivity-based islanding detection algorithm that utilizes the transient kinetic energy of the RESs to identify the islanding condition in microgrids and also considers cyber-attacks, which may lead to operators making inappropriate operational choices. To be specific, the proposed algorithm finds a generalized index that considers simultaneously the impact of volatility of parameters during islanding and non-islanding conditions and is also robust against data integrity attacks. The generalized index, the so-called sensitivity islanding detection (SID) index, is designed based on the sensitivity of transient kinetic energy over transient influential parameters. Furthermore, a statistical index for islanding detection is introduced based on the standard variation that effectively monitors the volatilities. An unsupervised anomaly detection method that uses kernel density estimation technology and statistical calculations are proposed as a way to find cyber-attacks. Following that, an outlier detection method is used to identify anomalies. The effectiveness of the proposed algorithm is evaluated under various conditions using simulation environments.

A novel unsupervised anomaly detection method for rotating machinery based on memory augmented temporal convolutional autoencoder

During the operation of rotating machinery, the occurrence of unknown fault types makes it impossible for the artificial intelligence-based fault diagnosis model to distinguish. Furthermore, due to the excessive generalization capability of the autoencoder, the unsupervised anomaly detection method based on the autoencoder is difficult to effectively distinguish normal and abnormal samples. To address the above problem, this paper proposed an unsupervised anomaly detection method based on memory augmented temporal convolutional autoencoder (MATCAE). Firstly, a novel temporal convolutional autoencoder model is constructed based on dilated causal convolution, skip connection and autoencoder to facilitate the model to learn the temporal features of the input data, thereby enhancing the model’s ability to capture the complex structure of the data. Then, a memory augmented module is designed using a memory matrix and an attention mechanism to expand the distribution interval between the reconstructed samples of normal and abnormal samples and reduce the sample capacity in the overlapping area. Finally, an anomaly detection module based on Euclidean distance, cosine distance and absolute mean square error is designed to improve the reliability of the metric between the input and reconstructed samples. To verify the effectiveness of the proposed method, experimental validation is carried out on a gearbox anomaly detection dataset. The experimental results show that the proposed method has higher anomaly detection accuracy and better noise robustness than other advanced anomaly detection methods, where the average performance metric is improved by 26.86% at the highest and 2.80% at the lowest.

Texture and semantic convolutional auto‐encoder for anomaly detection and segmentation

AbstractAnomaly detection is a challenging task, especially detecting and segmenting tiny defect regions in images without anomaly priors. Although deep encoder‐decoder‐based convolutional neural networks have achieved good anomaly detection results, existing methods operate uniformly on all extracted image features without considering disentangling these features. To fully explore the texture and semantic information of images, A novel unsupervised anomaly detection method is proposed. Specifically, discriminative features are extracted from images by using a deep pre‐trained network, where shallow and deep features are aggregated into texture and semantic modules, respectively. Then, a feature fusion module is developed to interactively enable feature information in two different modules. The texture and semantic segmentation results are obtained by comparing the texture features and semantic features before and after reconstruction, respectively. Finally, an anomaly segmentation module is designed to generate anomaly detection results by integrating the results of the texture and semantic modules by setting a threshold. Experimental results on benchmark datasets for anomaly detection demonstrate that our proposed method can efficiently and effectively detect anomalies, outperforming some state‐of‐the‐art methods by 2.7% and 0.6% in classification and segmentation.

Combining autoencoder with clustering analysis for anomaly detection in radiotherapy plans.

To develop an unsupervised anomaly detection method to identify suspicious error-prone treatment plans in radiotherapy. A total of 577 treatment plans of breast cancer patients were used in this study. They were labeled as either normal or abnormal plans by experienced clinicians. Multiple features of each plan were extracted and selected by the learning algorithms. The training set consisted of feature samples from 400 normal plans and the testing set consisted of feature samples from 158 normal plans and 19 abnormal plans. Using the k-means clustering algorithm in the training stage, 4 normal plan clusters were formed. The distance between the samples in the testing set and the cluster centers were then determined. To evaluate the effect of dimensionality reduction (DR) on detection accuracy, principal component analysis (PCA) and autoencoder (AE) methods were compared. The sensitivity of the anomaly detection model based on PCA and AE methods were 84.2% (16/19) and 94.7% (18/19), respectively. The specificity of the anomaly detection model based on PCA and AE methods were 64.6% (102/158) and 69.0% (109/158), respectively. The areas under the receiver operating characteristic (ROC) curve (AUCs) based on PCA and AE methods were 0.81 and 0.90, respectively. The unsupervised learning method was effective for detecting anomalies from the feature samples. Accuracy could be improved with the introduction of AE-based DR technique. The combination of AE and k-means clustering methods provides an automated way to identify abnormal plans among clinical treatment plans in radiotherapy.

Anomaly Detection and Concept Drift Adaptation for Dynamic Systems: A General Method with Practical Implementation Using an Industrial Collaborative Robot

Industrial collaborative robots (cobots) are known for their ability to operate in dynamic environments to perform many different tasks (since they can be easily reprogrammed). Due to their features, they are largely used in flexible manufacturing processes. Since fault diagnosis methods are generally applied to systems where the working conditions are bounded, problems arise when defining condition monitoring architecture, in terms of setting absolute criteria for fault analysis and interpreting the meanings of detected values since working conditions may vary. The same cobot can be easily programmed to accomplish more than three or four tasks in a single working day. The extreme versatility of their use complicates the definition of strategies for detecting abnormal behavior. This is because any variation in working conditions can result in a different distribution of the acquired data stream. This phenomenon can be viewed as concept drift (CD). CD is defined as the change in data distribution that occurs in dynamically changing and nonstationary systems. Therefore, in this work, we propose an unsupervised anomaly detection (UAD) method that is capable of operating under CD. This solution aims to identify data changes coming from different working conditions (the concept drift) or a system degradation (failure) and, at the same time, can distinguish between the two cases. Additionally, once a concept drift is detected, the model can be adapted to the new conditions, thereby avoiding misinterpretation of the data. This paper concludes with a proof of concept (POC) that tests the proposed method on an industrial collaborative robot.

Unsupervised anomaly detection for posteroanterior chest X-rays using multiresolution patch-based self-supervised learning

The demand for anomaly detection, which involves the identification of abnormal samples, has continued to increase in various domains. In particular, with increases in the data volume of medical imaging, the demand for automated screening systems has also risen. Consequently, in actual clinical practice, radiologists can focus only on diagnosing patients with abnormal findings. In this study, we propose an unsupervised anomaly detection method for posteroanterior chest X-rays (CXR) using multiresolution patch-based self-supervised learning. The core aspect of our approach is to leverage patch images of different sizes for training and testing to recognize diverse anomalies characterized by unknown shapes and scales. In addition, self-supervised contrastive learning is applied to learn the generalized and robust features of the patches. The performance of the proposed method is evaluated using posteroanterior CXR images from a public dataset for training and testing. The results show that the proposed method is superior to state-of-the-art anomaly detection methods. In addition, unlike single-resolution patch-based methods, the proposed method consistently exhibits a good overall performance regardless of the evaluation criteria used for comparison, thus demonstrating the effectiveness of using multiresolution patch-based features. Overall, the results of this study validate the effectiveness of multiresolution patch-based self-supervised learning for detecting anomalies in CXR images.

A Closed-loop Hybrid Supervision Framework of Cryptocurrency Transactions for Data Trading in IoT

The Device-as-a-service (DaaS) Internet of Things (IoT) business model enables distributed IoT devices to sell collected data to other devices, paving the way for machine-to-machine (M2M) economy applications. Cryptocurrencies are widely used by various IoT devices to undertake the main settlement and payment task in the M2M economy. However, the cryptocurrency market, which lacks effective supervision, has fluctuated wildly in the past few years. These fluctuations are breeding grounds for arbitrage in IoT data trading. Therefore, a practical cryptocurrency market supervision framework is very imperative in the process of IoT data trading to ensure that the trading is completed safely and fairly. The difficulty stems from how to combine these unlabeled daily trading data with supervision strategies to punish abnormal users, who disrupt the data trading market in IoT. In this article, we propose a closed-loop hybrid supervision framework based on the unsupervised anomaly detection to solve this problem. The core is to design the multi-modal unsupervised anomaly detection methods on trading prices to identify malicious users. We then design a dedicated control strategy with three levels to defend against various abnormal behaviors, according to the detection results. Furthermore, to guarantee the reliability of this framework, we evaluate the detection rate, accuracy, precision, and time consumption of single-modal and multi-modal detection methods and the contrast algorithm Adaptive KDE [ 19 ]. Finally, an effective prototype framework for supervising is established. The extensive evaluations prove that our supervision framework greatly reduces IoT data trading risks and losses.

Anomaly detection using a model-agnostic meta-learning-based variational auto-encoder for facility management

In a smart building, various types of sensor data are generated and employed to control devices within the building. Detecting anomalies in these sensor data is critical for effective facility management because it can prevent device malfunction or failure. Although deep learning-based methods have been used to detect anomalies, effective model construction is challenging because abnormal data tend to be rare under real-world conditions. In image classification and load forecasting, model-agnostic meta-learning (MAML) has recently been proposed to alleviate this problem using common knowledge learned from various tasks. This paper proposes an MAML-based unsupervised anomaly detection method called MAVAE for time-series sensor data. The proposed method uses a variational auto-encoder (VAE) as an anomaly detection model and adapts the model to a new target task with few unlabeled anomaly data via MAML. To our knowledge, this is the first study to train a VAE using MAML with time-series data. Extensive experiments on public data reveals that the proposed method outperforms existing anomaly detection methods, achieving an average improvement in prediction performance of 45% compared to state-of-the-art methods. The robustness of the proposed method is also demonstrated by evaluating its performance with a different number of samples for MAML. Our code is available at github.com/17011813/MAVAE.

Is It Worth It? Comparing Six Deep and Classical Methods for Unsupervised Anomaly Detection in Time Series

Detecting anomalies in time series data is important in a variety of fields, including system monitoring, healthcare and cybersecurity. While the abundance of available methods makes it difficult to choose the most appropriate method for a given application, each method has its strengths in detecting certain types of anomalies. In this study, we compare six unsupervised anomaly detection methods of varying complexity to determine whether more complex methods generally perform better and if certain methods are better suited to certain types of anomalies. We evaluated the methods using the UCR anomaly archive, a recent benchmark dataset for anomaly detection. We analyzed the results on a dataset and anomaly-type level after adjusting the necessary hyperparameters for each method. Additionally, we assessed the ability of each method to incorporate prior knowledge about anomalies and examined the differences between point-wise and sequence-wise features. Our experiments show that classical machine learning methods generally outperform deep learning methods across a range of anomaly types.

Unsupervised anomaly detection in brain MRI: Learning abstract distribution from massive healthy brains

PurposeTo develop a general unsupervised anomaly detection method based only on MR images of normal brains to automatically detect various brain abnormalities. Materials and methodsIn this study, a novel method based on three-dimensional deep autoencoder network is proposed to automatically detect and segment various brain abnormalities without being trained on any abnormal samples. A total of 578 normal T2w MR volumes without obvious abnormalities were used for model training and validation. The proposed 3D autoencoder was evaluated on two different datasets (BraTs dataset and in-house dataset) containing T2w volumes from patients with glioblastoma, multiple sclerosis and cerebral infarction. Lesions detection and segmentation performance were reported as AUC, precision-recall curve, sensitivity, and Dice score. ResultsIn anomaly detection, AUCs for three typical lesions were as follows: glioblastoma, 0.844; multiple sclerosis, 0.858; cerebral infarction, 0.807. In anomaly segmentation, the mean Dice for glioblastomas was 0.462. The proposed network also has the ability to generate an anomaly heatmap for visualization purpose. ConclusionOur proposed method was able to automatically detect various brain anomalies such as glioblastoma, multiple sclerosis, and cerebral infarction. This work suggests that unsupervised anomaly detection is a powerful approach to detect arbitrary brain abnormalities without labeled samples. It has the potential to support diagnostic workflow in radiology as an automated tool for computer-aided image analysis.