Unsupervised Anomaly Detection Method Research Articles

Registration and Convolutional Multidimensional Attention Module for Few-Shot Anomaly Detection

Aiming at the shortcomings of most current anomaly detection models, such as low detection accuracy and poor generalization performance, this paper proposes a few-shot anomaly detection model based on a convolutional multidimensional attention module to achieve feature registration (abbreviated as RCM-FSAD), which enhances the model’s perception of the overall image perception ability, using spatial transformer network to obtain the spatial transformation features of the image, improving the sensitivity of the relevant features, so that the whole model learns the commonality between the categories, and enhancing the generalization ability of the model. The spatial transformations and local structures of the input data are captured by deformable convolutional networks v2 to ensure the spatial invariance of the input data. The model is trained with only normal samples to accomplish anomalous regions’ localization and anomaly detection. On the challenging MVTec AD dataset, the unsupervised model not only improves the anomaly detection accuracy but also shows better generalization compared to current state-of-the-art unsupervised anomaly detection methods.

Dynamic Classifier Auditing by Unsupervised Anomaly Detection Methods: An Application in Packaging Industry Predictive Maintenance

Dynamic Classifier Auditing by Unsupervised Anomaly Detection Methods: An Application in Packaging Industry Predictive Maintenance

Automated Scoring of Morphological Changes in Images of Pentaerythritol Tetranitrate

ABSTRACTRecent advances in characterization techniques that generate large datasets of material microstructure images require robust, automated image‐processing. We applied an unsupervised anomaly detection method called feature anomaly detection system (FADS) to automatically detect and quantify microstructure changes in images of the explosive pentaerythritol tetranitrate (PETN) aged at various temperatures. We demonstrated the FADS approach on two‐dimensional images extracted from computed tomography scans, but the same technique can be readily applied to other imaging modalities. FADS calculates anomaly scores on the basis of differences in filter activations of nominal and test data in pretrained convolutional neural networks. The FADS scores successfully differentiated between pristine PETN and PETN aged at a temperature where material coarsening occurred. Morphological metric analysis of segmented images verified observed trends in FADS scores as a function of aging temperature and aging time, specifically by calculating volume fractions, specific boundary lengths, two‐point correlation functions, and local thicknesses. The FADS technique has two important advantages compared to traditional morphological analysis: First, it uses grayscale images as input, rather than images that are segmented to separate the appropriate phases; and second, FADS scores capture any type of changes among image sets, rather than requiring prior knowledge or selection of a relevant set of metrics.

FFD-Clustering: An unsupervised anomaly detection method for aero-engines based on fuzzy fusion of variables and discriminative mapping of features

FFD-Clustering: An unsupervised anomaly detection method for aero-engines based on fuzzy fusion of variables and discriminative mapping of features

Efficient anomaly detection method for offshore wind turbines

Time-series anomaly detection plays a crucial role in the operation of offshore wind turbines. Various wind turbine monitoring systems rely on time-series data to monitor and identify anomalies in real-time, as well as to initiate early warning processes. However, for offshore wind turbines with a high data density, conventional methods have high computational overhead in detecting anomalies while failing to accurately detect anomalies due to variations in data scales. To address this challenge, we propose an efficient anomaly detection method with contrastive learning, called Hawkeye. Hawkeye is based on residual clustering, an unsupervised anomaly detection method for multivariate time-series data. To ensure accurate anomaly detection, a trend-capturing prediction module is also combined with an automatic labeling module. As a result, the most common information can be learned from multivariate time-series data to reconstruct data trends. By evaluating Hawkeye on public datasets and real-world offshore wind turbine operational datasets, the results show that Hawkeye’s F1-score improves by an average of 14% compared with Isolation Forest, and its size shrinks by up to 11.5 times on the largest dataset compared with other methods. The proposed Hawkeye is potential to real-time monitoring and early warning systems for wind turbines, accelerating the development of intelligent operation and maintenance.

Anomaly Sign Detection for Automatic Ticket Gates by the Histogram Limitation Method

It is crucial to appropriately maintain automatic ticket gates (ATGs) to keep transportation operating smoothly in urban areas. Although the average failure rate of new ATGs is extremely low, continuous operation for many years might lead to unstable performance due to deterioration, and the need for periodic maintenance to avoid fatal faults might halt operations for extended periods. To detect anomalies at an early stage, “anomaly signs” can be utilized to flag ATGs for maintenance by service engineers before anomalies occur. In addition, to minimize the cost of ATG monitoring, the necessary computing resources should be minimized, which means using only light-weight statistical methods rather than deep learning or machine learning. In this paper, we focus on the automatic separation modules inside ATGs that separate multiple tickets by complicated mechatronic controls because this module is the major cause of maintenance calls from station attendants. We propose a simple anomaly sign detection, called the histogram limitation method (HLM). We evaluated the anomaly sign scores over time with maintenance timing and compared them with the conventional fast unsupervised anomaly detection method, Histogram-Based Outlier Score (HBOS) widely used in various domains. The experimental results using real field ATG monitoring data show that HLM successfully detected anomaly signs before a maintenance call was necessary, which is better performance compared with HBOS. Despite being a simple modification based on HBOS, HLM also provides anomaly sign scores that agree adequately with assessments by maintenance service engineers.

Structural damage detection and localization via an unsupervised anomaly detection method

This study introduces an unsupervised machine learning framework for damage detection and localization in Structural Health Monitoring (SHM), leveraging dynamic graph convolutional neural networks and Transformer networks. This approach is specifically tailored to overcome the challenge of limited labeled data in SHM, enabling precise analysis and feature synthesis from sensor-derived time series data for accurate damage identification. Incorporating a novel ‘localization score’ enhances the framework’s precision in pinpointing structural damages by integrating data-driven insights with a physics-informed understanding of structural dynamics. Extensive validations on diverse structures, including a benchmark steel structure and a real-world cable-stayed bridge, underscore the framework’s effectiveness in anomaly detection and damage localization, showcasing its potential to safeguard critical infrastructure through advanced data-effective machine learning techniques.

Detecting anomalies with granular-ball fuzzy rough sets

Most of the existing anomaly detection methods are based on a single and fine granularity input pattern, which is susceptible to noisy data and inefficient for detecting anomalies. Granular-ball computing, as a novel multi-granularity representation and computation method, can effectively compensate for these shortcomings. We utilize the fuzzy rough sets to mine the potential uncertainty information in the data efficiently. The combination of granular-ball computing and fuzzy rough sets takes into account the benefits of both methods, providing great application and research value. However, this novel combination still needs to be explored, especially for unsupervised anomaly detection. In this study, we first propose the granular-ball fuzzy rough set model, and the relevant definitions in the model are given. Subsequently, we pioneeringly present an unsupervised anomaly detection method based on granular-ball fuzzy rough sets called granular-ball fuzzy rough sets-based anomaly detection (GBFRD). Our method introduces the granular-ball fuzzy rough granules-based outlier factor to characterize the outlier degree of an object effectively. The experimental results demonstrate that GBFRD exhibits superior performance compared to the state-of-the-art methods. The code is publicly available at https://github.com/Mxeron/GBFRD.

Snow‐ or ice‐covered road detection in winter road surface conditions using deep neural networks

AbstractTraffic accidents occur frequently in cold and snow‐ or ice‐covered regions due to weather changes that occur during the winter season. To detect the snow‐ or ice‐covered roads in road surface conditions, road surface images captured using fixed‐point cameras installed along the route are sufficient. This paper proposes a snow‐ or ice‐covered road detection method that uses the deep convolutional autoencoding Gaussian mixture model (DCAGMM) with structural similarity (SSIM). The DCAGMM method, which is an unsupervised anomaly detection method, is unaffected by imbalance in the training data. In addition, the end‐to‐end convolutional neural network implemented in the DCAGMM enables the capture of the unique characteristics of the road surface images. Finally, by reconstructing the input images as normal images, the comparison of the input and reconstructed images enables identification of snow‐ or ice‐covered road areas without requiring pixel‐level annotations. Furthermore, the road surface images include complex characteristics for reconstruction, and the SSIM‐based reconstruction error allows us to preserve the image quality of the reconstructed image. Experimental results obtained on real‐world road surface images demonstrate the effectiveness of the proposed method.

Graph Transformer Network Incorporating Sparse Representation for Multivariate Time Series Anomaly Detection

Cyber–physical systems (CPSs) serve as the pivotal core of Internet of Things (IoT) infrastructures, such as smart grids and intelligent transportation, deploying interconnected sensing devices to monitor operating status. With increasing decentralization, the surge in sensor devices expands the potential vulnerability to cyber attacks. It is imperative to conduct anomaly detection research on the multivariate time series data that these sensors produce to bolster the security of distributed CPSs. However, the high dimensionality, absence of anomaly labels in real-world datasets, and intricate non-linear relationships among sensors present considerable challenges in formulating effective anomaly detection algorithms. Recent deep-learning methods have achieved progress in the field of anomaly detection. Yet, many methods either rely on statistical models that struggle to capture non-linear relationships or use conventional deep learning models like CNN and LSTM, which do not explicitly learn inter-variable correlations. In this study, we propose a novel unsupervised anomaly detection method that integrates Sparse Autoencoder with Graph Transformer network (SGTrans). SGTrans leverages Sparse Autoencoder for the dimensionality reduction and reconstruction of high-dimensional time series, thus extracting meaningful hidden representations. Then, the multivariate time series are mapped into a graph structure. We introduce a multi-head attention mechanism from Transformer into graph structure learning, constructing a Graph Transformer network forecasting module. This module performs attentive information propagation between long-distance sensor nodes and explicitly models the complex temporal dependencies among them to enhance the prediction of future behaviors. Extensive experiments and evaluations on three publicly available real-world datasets demonstrate the effectiveness of our approach.

Generating and Reweighting Dense Contrastive Patterns for Unsupervised Anomaly Detection

Recent unsupervised anomaly detection methods often rely on feature extractors pretrained with auxiliary datasets or on well-crafted anomaly-simulated samples. However, this might limit their adaptability to an increasing set of anomaly detection tasks due to the priors in the selection of auxiliary datasets or the strategy of anomaly simulation. To tackle this challenge, we first introduce a prior-less anomaly generation paradigm and subsequently develop an innovative unsupervised anomaly detection framework named GRAD, grounded in this paradigm. GRAD comprises three essential components: (1) a diffusion model (PatchDiff) to generate contrastive patterns by preserving the local structures while disregarding the global structures present in normal images, (2) a self-supervised reweighting mechanism to handle the challenge of long-tailed and unlabeled contrastive patterns generated by PatchDiff, and (3) a lightweight patch-level detector to efficiently distinguish the normal patterns and reweighted contrastive patterns. The generation results of PatchDiff effectively expose various types of anomaly patterns, e.g. structural and logical anomaly patterns. In addition, extensive experiments on both MVTec AD and MVTec LOCO datasets also support the aforementioned observation and demonstrate that GRAD achieves competitive anomaly detection accuracy and superior inference speed.

Personalized anomaly detection in PPG data using representation learning and biometric identification

Photoplethysmography (PPG) signals, typically acquired from wearable devices, hold significant potential for continuous fitness-health monitoring. In particular, heart conditions that manifest in rare and subtle deviating heart patterns may be interesting. However, robust and reliable anomaly detection within these data remains a challenge due to the scarcity of labeled data and high inter-subject variability. This paper introduces a two-stage framework leveraging representation learning and personalization to improve anomaly detection performance in PPG data. The proposed framework first employs representation learning to transform the original PPG signals into a more discriminative and compact representation. We then apply three different unsupervised anomaly detection methods for movement detection and biometric identification. We validate our approach using two different datasets in both generalized and personalized scenarios. Our results demonstrate significant improvements: for movement detection, in the generalized scenario, AUCs improved from barely 0.5 to above 0.9 with representation learning. Importantly, inter-subject variability was substantially reduced, from around 0.4 to below 0.1. In the personalized scenario, AUCs became close to 1.0, with variability further reduced to below 0.05, indicating the effectiveness of both representation learning and personalization for anomaly detection in PPG data. Similar enhancements were observed in biometric identification, emphasizing how our approach can minimize inter-subject variability and enhance PPG-based health monitoring systems.

Encoder-Decoder Contrast for Unsupervised Anomaly Detection in Medical Images.

Unsupervised anomaly detection (UAD) aims to recognize anomalous images based on the training set that contains only normal images. In medical image analysis, UAD benefits from leveraging the easily obtained normal (healthy) images, avoiding the costly collecting and labeling of anomalous (unhealthy) images. Most advanced UAD methods rely on frozen encoder networks pre-trained using ImageNet for extracting feature representations. However, the features extracted from the frozen encoders that are borrowed from natural image domains coincide little with the features required in the target medical image domain. Moreover, optimizing encoders usually causes pattern collapse in UAD. In this paper, we propose a novel UAD method, namely Encoder-Decoder Contrast (EDC), which optimizes the entire network to reduce biases towards pre-trained image domain and orient the network in the target medical domain. We start from feature reconstruction approach that detects anomalies from reconstruction errors. Essentially, a contrastive learning paradigm is introduced to tackle the problem of pattern collapsing while optimizing the encoder and the reconstruction decoder simultaneously. In addition, to prevent instability and further improve performances, we propose to bring globality into the contrastive objective function. Extensive experiments are conducted across four medical image modalities including optical coherence tomography, color fundus image, brain MRI, and skin lesion image, where our method outperforms all current state-of-the-art UAD methods. Code is available at: https://github.com/guojiajeremy/EDC.

Unsupervised Deep Anomaly Detection for Industrial Multivariate Time Series Data

With the rapid development of deep learning, researchers are actively exploring its applications in the field of industrial anomaly detection. Deep learning methods differ significantly from traditional mathematical modeling approaches, eliminating the need for intricate mathematical derivations and offering greater flexibility. Deep learning technologies have demonstrated outstanding performance in anomaly detection problems and gained widespread recognition. However, when dealing with multivariate data anomaly detection problems, deep learning faces challenges such as large-scale data annotation and handling relationships between complex data variables. To address these challenges, this study proposes an innovative and lightweight deep learning model—the Attention-Based Deep Convolutional Autoencoding Prediction Network (AT-DCAEP). The model consists of a characterization network based on convolutional autoencoders and a prediction network based on attention mechanisms. The AT-DCAEP exhibits excellent performance in multivariate time series data anomaly detection without the need for pre-labeling large-scale datasets, making it an efficient unsupervised anomaly detection method. We extensively tested the performance of AT-DCAEP on six publicly available datasets, and the results show that compared to current state-of-the-art methods, AT-DCAEP demonstrates superior performance, achieving the optimal balance between anomaly detection performance and computational cost.

Unsupervised fabric defect detection based on multiscale image reconstruction and structural similarity assessment

AbstractFabric defect detection is a crucial aspect of the textile industry. Currently, deep learning methods have demonstrated exceptional performance in fabric defect detection tasks. However, their performance is greatly affected by the number of defect samples, which is a challenge to obtain during actual production. To address this issue, this article proposes an unsupervised anomaly detection method for fabric defects using image reconstruction networks. This method only requires defect‐free samples for training. During the training phase, the model compresses defect‐free samples to obtain a low‐dimensional manifold and reconstruct them. During the inference phase, the method assesses whether a sample is defective by calculating the reconstruction error between the input and output images, and locates the defect region by computing the difference in various patches. Furthermore, since fabric contains rich texture features, with high correlation between neighbouring pixels, a structure similarity index measure combined with mean absolute error is introduced to evaluate the reconstruction error, which enhances the model's representation ability for defect‐free samples. Additionally, considering the diverse texture backgrounds in fabric, a multiscale reconstruction module is designed to optimise the reconstruction effect. Experimental results demonstrate that compared with other related approaches, the proposed method achieves high accuracy (image‐based area under the curve (AUC) up to 98.2% and pixel‐based AUC up to 97.3%) on multiple datasets and has good generalisation ability for different fabric textures.

Non-contact sensing for anomaly detection in wind turbine blades: A focus-SVDD with complex-valued auto-encoder approach

The occurrence of manufacturing defects in wind turbine blade (WTB) production can result in significant increases in operation and maintenance costs of WTBs, reduce capacity factors of wind farms, and occasionally lead to severe and disastrous consequences. Therefore, inspection during the manufacturing process is crucial to ensure consistent fabrication of composite materials. Non-contact sensing techniques, such as Frequency Modulated Continuous Wave (FMCW) radar, are becoming increasingly popular as they offer a full view – cross sectional analysis – of these complex structures during assembly and curing. In this paper, we enhance the quality assurance of WTB manufacturing utilising FMCW radar as a non-destructive sensing modality. Additionally, a novel anomaly detection pipeline is developed that offers the following advantages: (1) We use the analytic representation of the Intermediate Frequency signal of the FMCW radar as a feature to disentangle material-specific and round-trip delay information from the received wave. (2) We propose a novel anomaly detection methodology called focus Support Vector Data Description (focus-SVDD). This methodology involves defining the limit boundaries of the dataset after removing healthy data features, thereby focusing on the attributes of anomalies. (3) The proposed method employs a complex-valued autoencoder to remove healthy features and we introduces a new activation function called Exponential Amplitude Decay (EAD). EAD takes advantage of the Rayleigh distribution, which characterises an instantaneous amplitude signal. The effectiveness of the proposed method is demonstrated through its application to collected data, where it shows superior performance compared to other state-of-the-art unsupervised anomaly detection methods. This method is expected to make a significant contribution not only to structural health monitoring but also to the field of deep complex-valued data processing and SVDD application. The code and dataset will be made publicly available. The code and dataset are available here 11https://github.com/FrusqueGaetan/Focus_SVDD_EAD.

THE ADJUSTED HISTOGRAM-BASED OUTLIER SCORE - AHBOS

Histogram is a commonly used tool for visualizing data distribution. It has also been used in semi-supervised and unsupervised anomaly detection tasks. The histogram-based outlier score is a fast unsupervised anomaly detection method that has become more popular because of the rapid increase in the amount of data collected in recent decades. Histogram-based outlier score can be computed using either static or dynamic bin-width histograms. When a histogram contains large gaps, the dynamic bin-width approach is preferred over the static bin-width approach. These gaps in a histogram usually occur as a result of various distributions in real data. When working with a static bin-width histogram, gaps can be utilized to acquire better distinction between outliers and inliers. In this study, we propose an adjusted version of the histogram-based outlier score named adjusted histogram-based outlier score, which considers neighboring bins prior to density estimation. Results from a simulation study and real data application indicate that the adjusted histogram-based outlier score yields a better performance not only in the simulated data but also for various types of real data.

A periodic anomaly detection framework based on matrix profile for condition monitoring of planetary gearboxes

As the core power transmission component of mechanical equipment, planetary gearboxes unavoidably experience faults due to harsh working environments and continuously heavy loads. Therefore, to ensure safe operation of the equipment and reduction of maintenance costs, condition monitoring of planetary gearboxes is critical. Anomaly detection provides a potential approach to overcome the limitations of traditional signal processing technologies and deep learning methods. However, since the monitoring signal of planetary gearboxes contains various harmonics and periodic impulses, current unsupervised anomaly detection methods are difficult to achieve a satisfactory result for the monitoring signal. To overcome the above problem, a novel periodic anomaly detection framework is put forward for planetary gearboxes condition monitoring based on matrix profile (MP). In this work, a periodic anomaly detection method, namely matrix profile guided multiple residuals average (MPGMRA), is first introduced to detect periodic impulses caused by the local damage. Then, to better extract the weak periodic impulses, an adaptive parameter selection strategy based on impulse-to-noise ratio (INR) is proposed. At last, both simulated signal and experimental data from a planetary gearbox are analyzed to demonstrate the applicability of the proposed framework, and the correlation kurtosis (CK) is introduced to evaluate the magnitude of the periodic fault component. The results manifest that this framework is able to recognize the condition of planetary gearboxes successfully and the performance based on CK can be improved by 5.4 times.

A Novel Warning Identification Framework for Risk-Informed Anomaly Detection

Cyber-physical systems are taking on a permanent role in the industry, such as in oil and gas or mining. These systems are expected to perform increasingly autonomous tasks in complex settings removing human operators from remote and potentially hazardous environments. High autonomy necessitates a more extensive use of artificial intelligence methods, such as anomaly detection, to identify unusual occurrences in the monitored environment. The absence of data characterizing potentially hazardous events leads to disruptive noise displayed as false alarms, a common anomaly detection issue for hazard identification applications. Contrastingly, disregarding the false alarms can result in the opposite effect, causing loss of early indications of hazardous occurrences. Existing research introduces simulating and extrapolating less represented data to expand the information on hazards and semi-supervise the methods or by introducing thresholds and rule-based methods to balance noise and meaningful information, necessitating intensive computing resources. This research proposes a novel Warning Identification Framework that evaluates risk analysis objectives and applies them to discern between true and false warnings identified by anomaly detection. We demonstrate the results by analyzing three seismic hazard assessment methods for identifying seismic tremors and comparing the outcomes to anomalies found using the unsupervised anomaly detection method. The demonstrated approach shows great potential in enhancing the reliability and transparency of anomaly detection outcomes and, thus, supporting the operational decision-making process of a cyber-physical system.

A Multivariate KPIs Anomaly Detection Framework With Dynamic Balancing Loss Training

Anomaly detection on multivariate KPIs (Key Performance Indicators, such as CPU utilization, sockets status, and HTTP requests per second) is of utmost importance to the systems’ reliability. Unsupervised methods have been of considerable interests and have significantly progressed due to their superior effectiveness. However, the state-of-art unsupervised anomaly detection methods still suffer from high false or missed alarm rates. To this end, in this paper, we propose MM, a practical Multivariate KPIs anomaly detection framework following the principles of Multi-task learning with the proposed dynamic balancing loss function. To capture KPIs’ characteristics to the most extent, we simultaneously train multiple sequential autoencoders with different connections based on a designed semi-Random Connection Recurrent Neural Network (sRC-RNN). These autoencoders can be treated as different reconstruction tasks while training. Furthermore, we propose a dynamic loss function to adaptively balance the tasks’ weights. Extensive experiments show that MM outperforms the state-of-art unsupervised multivariate KPIs anomaly detection algorithms and achieves an average F1-score of 0.95 on two public machine-level KPIs datasets and 0.96 on an internal container-level KPIs dataset.