Anomaly Detection Model Research Articles

Multivariate anomaly detection models enhance identification of errors in routine clinical chemistry testing.

Conventional autoverification rules evaluate analytes independently, potentially missing unusual patterns of results indicative of errors such as serum contamination by collection tube additives. This study assessed whether multivariate anomaly detection algorithms could enhance the detection of such errors. Multivariate Gaussian, k-nearest neighbours (KNN) distance, and one-class support vector machine (SVM) anomaly detection models, along with conventional limit checks, were developed using a training dataset of 127,451 electrolyte, urea, and creatinine (EUC) results, with a 5 % flagging rate targeted for all approaches. The models were compared with limit checks for their ability to detect atypical EUC results from samples spiked with additives from collection tubes: EDTA, fluoride, sodium citrate, or acid citrate dextrose (n=200 per contaminant). The study additionally assessed the ability of the models to identify 127,449 single-analyte errors, a potential weakness of multivariate models. The KNN distance and SVM models outperformed limit checks for detecting all contaminants (p-values <0.05). The multivariate Gaussian model did not surpass limit checks for detecting EDTA contamination but was superior for detecting the other additives. All models surpassed limit checks for identifying single-analyte errors, with the KNN distance model demonstrating the highest overall sensitivity. Multivariate anomaly detection models, particularly the KNN distance model, were superior to the conventional approach for detecting serum contamination and single-analyte errors. Developing multivariate approaches to autoverification is warranted to optimise error detection and improve patient safety.

Weakly Supervised anomaly detection with privacy preservation under a Bi-Level Federated learning framework

Training a Machine Learning (ML) model in the industrial field faces special challenges, such as data privacy, lack of data, data imbalance, and unlabeled data. Therefore, it is not realistic to gather production data directly from various companies and use them to train a machine learning model. In this paper, we proposed a novel framework named Bi-level Federated Learning (BFL) to tackle the above challenges. In the first level, a Weakly Supervised Anomaly Detection method named Pairwise Relation prediction-based Ordinal regression Network (PRO) is utilized for training a Deep Anomaly Detection (DAD) model under Federated Learning (FL) mechanism. According to the DAD model, anomalies are identified and labeled with ‘anomaly’ tags. In order to train a multi-classification model that can identify different types of defects not just anomalies, the anomalies are manually labeled according to their respective defect types. In the second level, labeled anomaly and normal data are applied to train a multi-classification model under the FL mechanism. A framework is proposed to classify defects in pre-baked carbon anodes. Its performance is verified through a real case study. The results show that the DAD model achieved an acceptable performance with a 0.988 recall and 0.866 F1 score. The multi-classification model has shown good performance on 0.942 accuracy, 0.913 precision, 0.903 recall, and 0.906 F1. The results of the BFL exhibit improved performance compared to our previous work and classical ML methods.

Separable contextual graph neural networks to identify tailgating-oriented traffic congestion

Identifying largescale traffic congestion with thousands of robots is a complicated task involving structured-data graphical modeling, and it has recently gained more research attention. This study aims to model traffic metrics that exhibit negative correlations in specific spatiotemporal contexts of automated material handling systems in semiconductor manufacturing. Numerous studies have focused on detecting vehicle congestion using multivariate time-series data. However, they faced challenges regarding the understanding of the complex graph structural joint spatiotemporal dynamics of the traffic rails. Existing methods fail to distinguish between congestion caused by high traffic volumes and nondisruptive delivery robot operations, such as parking and hoisting. This study introduces a novel approach to detect tailgating-oriented congestion using separable contextual graph neural networks (SC-GNNs). The proposed method employs deviation-based anomaly detection with attentive GNNs to capture spatial correlations and temporal dependencies. SC-GNNs were proposed for dual metrics with distinct characteristics, and dynamic thresholds were applied for each GNN to ensure reliable and interpretable anomaly detection. To demonstrate the outperformance of the proposed method, we conducted simulation experiments via a high-fidelity simulator similar to actual semiconductor fabrication. The experimental results showed that our method achieved both a 36% reduction in the false positive rate and a 32% decrease in the equal error rate compared to the best performance of existing state-of-the-art anomaly detection models. Overall, the proposed approach significantly improved the accuracy and interpretability of anomaly detection in multivariate time series, particularly effective in the context of identifying anomalies related to traffic congestion tailgating.

VAEAT: Variational AutoeEncoder with adversarial training for multivariate time series anomaly detection

High labor costs and the requirement for significant domain expertise often result in a lack of anomaly labels in most time series. Consequently, employing unsupervised methods becomes critical for practical industrial applications. However, prevailing reconstruction-based anomaly detection algorithms encounter challenges in capturing intricate underlying correlations and temporal dependencies in time series. This study introduces an unsupervised anomaly detection model called Variational AutoeEncoder with Adversarial Training for Multivariate Time Series Anomaly Detection (VAEAT). Its fundamental concept involves adopting a two-phase training strategy to improve anomaly detection precision through adversarial reconstruction of raw data. In the first phase, the model reconstructs raw data to extract its basic features by training two enhanced variational autoencoders (VAEs) that incorporate both the long short-term memory (LSTM) network and the attention mechanism in their common encoder. In the second phase, the model refines reconstructed data to optimize the reconstruction quality. In this manner, this two-phase VAE model effectively captures intricate underlying correlation and temporal dependencies. A large number of experiments are conducted to evaluate the performance on five publicly available datasets, and experimental results illustrate that VAEAT exhibits robust performance and effective anomaly detection capabilities. The source code of the proposed VAEAT can be available at https://github.com/Du-Team/VAEAT.

Adversarial Attacks against Deep-Learning-Based Automatic Dependent Surveillance-Broadcast Unsupervised Anomaly Detection Models in the Context of Air Traffic Management.

Deep learning has shown significant advantages in Automatic Dependent Surveillance-Broadcast (ADS-B) anomaly detection, but it is known for its susceptibility to adversarial examples which make anomaly detection models non-robust. In this study, we propose Time Neighborhood Accumulation Iteration Fast Gradient Sign Method (TNAI-FGSM) adversarial attacks which fully take into account the temporal correlation of an ADS-B time series, stabilize the update directions of adversarial samples, and escape from poor local optimum during the process of iterating. The experimental results show that TNAI-FGSM adversarial attacks can successfully attack ADS-B anomaly detection models and improve the transferability of ADS-B adversarial examples. Moreover, the TNAI-FGSM is superior to two well-known adversarial attacks called the Fast Gradient Sign Method (FGSM) and Basic Iterative Method (BIM). To the best of our understanding, we demonstrate, for the first time, the vulnerability of deep-learning-based ADS-B time series unsupervised anomaly detection models to adversarial examples, which is a crucial step in safety-critical and cost-critical Air Traffic Management (ATM).

A Robust Anomaly Detection Model for Pumps Based on the Spectral Residual With Self-Attention Variational Autoencoder

A Robust Anomaly Detection Model for Pumps Based on the Spectral Residual With Self-Attention Variational Autoencoder

MMD Fence GAN Unsupervised Anomaly Detection Model Based on Maximum Mean Discrepancy

Due to data imbalances and the complexity of data, identifying outliers can be challenging. To address this issue, this paper introduces a new unsupervised model called MMD Fence GAN. An improved loss function is incorporated into the generator to enhance anomaly detection accuracy. In the loss function of the generator, there are three main components: the encirclement loss limits the generated samples to lie at the decision boundary of the normal samples. The central dispersion loss maximizes the decision boundary's range. By measuring the maximum mean discrepancy between the generated samples and the given samples, the sample dispersion loss avoids issues like model collapse and outliers. Two datasets validate the effectiveness of the model: On the 2D synthetic dataset, it converges faster and pinpoints decision boundaries more precisely than Fence GAN. Both F1-score and recall improve over traditional and advanced deep learning methods on the KDD99 dataset.

Digital phenotyping data and anomaly detection methods to assess changes in mood and anxiety symptoms across a transdiagnostic clinical sample.

Clinical assessment of mood and anxiety change often relies on clinical assessment or self-reported scales. Using smartphone digital phenotyping data and resulting markers of behavior (e.g., sleep) to augment clinical symptom scores offers a scalable and potentially more valid method to understand changes in patients' state. This paper explores the potential of using a combination of active and passive sensors in the context of smartphone-based digital phenotyping to assess mood and anxiety changes in two distinct cohorts of patients to assess the preliminary reliability and validity of this digital phenotyping method. Participants from two different cohorts, each n = 76, one with diagnoses of depression/anxiety and the other schizophrenia, utilized mindLAMP to collect active data (e.g., surveys on mood/anxiety), along with passive data consisting of smartphone digital phenotyping data (geolocation, accelerometer, and screen state) for at least 1 month. Using anomaly detection algorithms, we assessed if statistical anomalies in the combination of active and passive data could predict changes in mood/anxiety scores as measured via smartphone surveys. The anomaly detection model was reliably able to predict symptom change of 4 points or greater for depression as measured by the PHQ-9 and anxiety as measured for the GAD-8 for both patient populations, with an area under the ROC curve of 0.65 and 0.80 for each respectively. For both PHQ-9 and GAD-7, these AUCs were maintained when predicting significant symptom change at least 7 days in advance. Active data alone predicted around 52% and 75% of the symptom variability for the depression/anxiety and schizophrenia populations respectively. These results indicate the feasibility of anomaly detection for predicting symptom change in transdiagnostic cohorts. These results across different patient groups, different countries, and different sites (India and the US) suggest anomaly detection of smartphone digital phenotyping data may offer a reliable and valid approach to predicting symptom change. Future work should emphasize prospective application of these statistical methods.

Pedestrian Abnormal Behavior Detection System Using Edge–Server Architecture for Large–Scale CCTVEnvironments

As the deployment of CCTV cameras for safety continues to increase, the monitoring workload has significantly exceeded the capacity of the current workforce. To overcome this problem, intelligent CCTV technologies and server-efficient deep learning analysis models are being developed. However, real-world applications exhibit performance degradation due to environmental changes and limited server processing capacity for multiple CCTVs. This study proposes a real-time pedestrian anomaly detection system with an edge–server structure that ensures efficiency and scalability. In the proposed system, the pedestrian abnormal behavior detection model analyzed by the edge uses a rule-based mechanism that can detect anomalies frequently, albeit less accurately, with high recall. The server uses a deep learning-based model with high precision because it analyzes only the sections detected by the edge. The proposed system was applied to an experimental environment using 20 video streams, 18 edge devices, and 3 servers equipped with 2 GPUs as a substitute for real CCTV. Pedestrian abnormal behavior was included in each video stream to conduct experiments in real-time processing and compare the abnormal behavior detection performance between the case with the edge and server alone and that with the edge and server in combination. Through these experiments, we verified that 20 video streams can be processed with 18 edges and 3 GPU servers, which confirms the scalability of the proposed system according to the number of events per hour and the event duration. We also demonstrate that the pedestrian anomaly detection model with the edge and server is more efficient and scalable than the models with these components alone. The linkage of the edge and server can reduce the false detection rate and provide a more accurate analysis. This research contributes to the development of control systems in urban safety and public security by proposing an efficient and scalable analysis system for large-scale CCTV environments.

Anomaly Detection for Industrial Time Series Data Based on Correlation Analysis and CNN-BiLSTM with Self-attention

This paper aims to propose an anomaly detection model for industrial time series data based on correlation analysis and CNN-BiLSTM with self-attention to solve the problem of abnormal data detection in the field of industrial data analysis. Industrial data anomaly detection is an important task in the industrial field, which can help people to timely understand the production operation status and real-time record and perception of the operating environment. This paper introduces two key technologies: correlation analysis and CNN-BiLSTM with self-attention, and how to combine them to build an effective anomaly detection model for industrial time series data. Through experimental evaluation, this paper proves the effectiveness and superiority of the proposed model in industrial data anomaly detection tasks.

A Non-Intrusive Method for Lonely Death Prevention Using Occupancy Detection and an Anomaly Detection Model

In countries like Japan, Australia, France, Denmark, and South Korea, the numbers of single-person households and older adults living alone have been steadily increasing each year, leading to the social issue of lonely deaths among older adults. Against this backdrop, this study proposes a method to develop a system for preventing lonely deaths based on information technology, including the Internet of Things (IoT). IoT sensor data, which include nine environmental variables such as indoor temperature, relative humidity, CO2 concentration, fine dust particle levels, illuminance, total volatile organic compound levels, and occupancy data collected from passive infrared sensors, provide empirical evidence so that anomalies can be detected in the behavior patterns of older adults when they remain in one place for an unusually long time. Detecting such risky situations for older adults living alone involves anomaly detection through occupancy monitoring. The data from occupancy monitoring were analyzed using four classification models, namely Logistic Regression, k-Nearest Neighbor, Decision Tree, and Random Forest, with the performance of occupancy detection being compared across these models. Furthermore, the method proposed in this study includes data processing for environmental variables to improve the performance of occupancy detection.

Less-supervised learning with knowledge distillation for sperm morphology analysis

ABSTRACT Sperm Morphology Analysis (SMA) is pivotal in diagnosing male infertility. However, manual analysis is subjective and time-intensive. Artificial intelligence presents automated alternatives, but hurdles like limited data and image quality constraints hinder its efficacy. These challenges impede Deep Learning (DL) models from grasping crucial sperm features. A solution enabling DL models to learn sample nuances, even with limited data, would be invaluable. This study proposes a Knowledge Distillation (KD) method to distinguish normal from abnormal sperm cells, leveraging the Modified Human Sperm Morphology Analysis dataset. Despite low-resolution, blurry images, our method yields relevant results. We exclusively utilize normal samples to train the model for anomaly detection, crucial in scenarios lacking abnormal data – a common issue in medical tasks. Our aim is to train an Anomaly Detection model using a dataset comprising unclear images and limited samples, without direct exposure to abnormal data. Our method achieves Receiver ROC/AUC scores of 70.4%, 87.6%, and 71.1% for head, vacuole, and acrosome, respectively, our method matches traditional DL model performance with less than 70% of the data. This less-supervised approach shows promise in advancing SMA despite data scarcity. Furthermore, KD enables model adaptability to edge devices in fertility clinics, requiring less processing power.

MCAD: Multi-classification anomaly detection with relational knowledge distillation

With the wide application of deep learning in anomaly detection (AD), industrial vision AD has achieved remarkable success. However, current AD usually focuses on anomaly localization and rarely investigates anomaly classification. Furthermore, anomaly classification is currently requested for quality management and anomaly reason analysis. Therefore, it is essential to classify anomalies while improving the accuracy of AD. This paper designs a novel multi-classification AD (MCAD) framework to achieve high-accuracy AD with an anomaly classification function. In detail, the proposal model based on relational knowledge distillation consists of two components. The first one employs a teacher–student AD model, utilizing a relational knowledge distillation approach to transfer the interrelationships of images. The teacher–student critical layer feature activation values are used in the knowledge transfer process to achieve anomaly detection. The second component realizes anomaly multi-classification using the lightweight convolutional neural network. Our proposal has achieved 98.95, 96.04, and 92.94% AUROC AD results on MNIST, FashionMNIST, and CIFAR10 datasets. Meanwhile, we earn 97.58 and 98.10% AUROC for AD and localization in the MVTecAD dataset. The average classification accuracy of anomaly classification has reached 76.37% in fifteen categories of the MVTec-AD dataset. In particular, the classification accuracy of the leather category has gained 95.24%. The results on the MVTec-AD dataset show that MCAD achieves excellent detection, localization, and classification results.

A multiscale electricity theft detection model based on feature engineering

With the widespread adoption of smart meters and the growing availability of data mining and machine learning algorithms, there is a pressing demand for methods that are both accurate and explicable in identifying electricity theft patterns among end-users. To address this need, this study proposes a multi-scale anomaly detection model based on feature engineering.Specifically, tsfresh is utilized in feature engineering to extract electricity consumption features from the raw data, and XGBoost is employed to select features that are highly correlated with anomalous behavior, which have clear physical interpretations. Multi-scale convolutional neural networks are then used to analyze and process the data at different temporal and frequency scales. Attention mechanisms are applied to assign weights to different feature channels, and all of the extracted information is fused for anomaly detection. The combination of feature engineering and multi-scale convolutional neural networks not only enhances the interpretability of the model but also improves its performance, as demonstrated by the experimental results, which show that the proposed method outperforms traditional anomaly detection approaches across multiple evaluation metrics.

Reliable fabric defect detection via Bayesian uncertainty modeling

Despite the demonstrated capability of deep learning models in detecting anomalies in textile images, their predictions in real-world applications tend to be overly confident, especially when faced with defect types not previously encountered in the training set or when dealing with low-quality annotations. This excessive confidence in predictions limits the practical application of deep learning methods in textile defect detection, as it fails to provide inspectors with reliable guidance on when to trust the model's predictions and when manual verification is necessary. To address this issue, this paper introduces a Bayesian fabric anomaly detection model that utilizes Variational Inference (VI) to apply Bayesian inference to the widely used U-Net architecture. During the inference phase, the model employs Monte Carlo sampling to perform multiple forward passes, generating three types of uncertainty estimations and per-pixel uncertainty maps, thus providing comprehensive evidence for decision-making. This method not only estimates the uncertainty of model predictions but also improves the F1 score by 2-4% over the baseline U-Net model in the frequency domain. This study proves the Bayesian approach boosts fabric anomaly detection and decision-making by optimizing model performance and reducing reliance on inaccurate predictions.

An Unsupervised Method for Industrial Image Anomaly Detection with Vision Transformer-Based Autoencoder.

Existing industrial image anomaly detection techniques predominantly utilize codecs based on convolutional neural networks (CNNs). However, traditional convolutional autoencoders are limited to local features, struggling to assimilate global feature information. CNNs' generalizability enables the reconstruction of certain anomalous regions. This is particularly evident when normal and abnormal regions, despite having similar pixel values, contain different semantic information, leading to ineffective anomaly detection. Furthermore, collecting abnormal image samples during actual industrial production poses challenges, often resulting in data imbalance. To mitigate these issues, this study proposes an unsupervised anomaly detection model employing the Vision Transformer (ViT) architecture, incorporating a Transformer structure to understand the global context between image blocks, thereby extracting a superior representation of feature information. It integrates a memory module to catalog normal sample features, both to counteract anomaly reconstruction issues and bolster feature representation, and additionally introduces a coordinate attention (CA) mechanism to intensify focus on image features at both spatial and channel dimensions, minimizing feature information loss and thereby enabling more precise anomaly identification and localization. Experiments conducted on two public datasets, MVTec AD and BeanTech AD, substantiate the method's effectiveness, demonstrating an approximate 20% improvement in average AUROC% at the image level over traditional convolutional encoders.

TTSAD: TCN-Transformer-SVDD Model for Anomaly Detection in air traffic ADS-B data

ADS-B (Automatic Dependent Surveillance-Broadcast) is a key technology in the new generation air traffic surveillance system. However, it is vulnerable to various cyber attacks because it broadcasts data in plaintext format and lacks authentication mechanism. Previous research has rarely considered the application scenarios of ATM (Air Traffic Management) in commercial air transport, and there are the problems of low anomaly detection rate and the non-lightweight model. This paper focuses on ADS-B anomaly detection under the background of ATM. We propose the TTSAD (TCN-Transformer-SVDD Model for Anomaly Detection) model, which aims to address the problems of existing ADS-B anomaly detection methods including inadequate considerations of long-term dependencies and distribution characteristic, the non-lightweight model and the poor adaptive threshold. First, ADS-B time series is input into TCN (Temporal Convolutional Network) prediction module which predicts data in an accurate and quick way using causal convolution and dilated convolution. Then, the predicted ADS-B time series is input into Transformer reconstruction module which reconstructs data accurately and quickly based on Self-Attention and Multi-Head Attention mechanism. Finally, the difference values between the reconstructed values and the real values are input into SVDD (Support Vector Data Description) threshold determination module for an optimal threshold. Experimental results show that the TTSAD model can detect ADS-B anomaly data generated from attacks such as altitude slow offset and DOS (Denial of Service). The TTSAD model is superior to other machine learning methods in terms of recall rate, detection rate, accuracy rate, missing detection rate and false alarm rate. Furthermore, compared with other deep learning methods including LSTM, GRU and LSTM-AE, the TTSAD model has a shorter training time and a lightweight characteristic. This approach guarantees the information security of ADS-B, thereby improving the operational security of ATM.

SCA-GANomaly: an unsupervised anomaly detection model of high-speed railway catenary components

SCA-GANomaly: an unsupervised anomaly detection model of high-speed railway catenary components

Skip-GANomaly++: Skip Connections and Residual Blocks for Anomaly Detection (Student Abstract)

Anomaly detection is a critical task across various domains. Fundamentally, anomaly detection models offer methods to identify unusual patterns that do not align with expected behaviors. Notably, in the medical field, detecting anomalies in medical imagery or biometrics can facilitate early diagnosis of diseases. Consequently, we propose the Skip-GANomaly++ model, an enhanced and more efficient version of the conventional anomaly detection models. The proposed model's performance was evaluated through comparative experiments. Experimental results demonstrated superior performance across most classes compared to the previous models.

Counterfactual Graph Learning for Anomaly Detection with Feature Disentanglement and Generation (Student Abstract)

Graph anomaly detection has received remarkable research interests, and various techniques have been employed for enhancing detection performance. However, existing models tend to learn dataset-specific spurious correlations based on statistical associations. A well-trained model might suffer from performance degradation when applied to newly observed nodes with different environments. To handle this situation, we propose CounterFactual Graph Anomaly Detection model, CFGAD. In this model, we design a gradient-based separator to disentangle node features into class features and environment features. Then, we present a weight-varying diffusion model to combine class features and environment features from different nodes to generate counterfactual samples. These counterfactual samples will be adopted to enhance model robustness. Comprehensive experiments demonstrate the effectiveness of our CFGAD.