Anomalous Data Research Articles

A novel approach for multivariate time series interval prediction of water quality at wastewater treatment plants

ABSTRACT This study proposes a novel approach for predicting variations in water quality at wastewater treatment plants (WWTPs), which is crucial for optimizing process management and pollution control. The model combines convolutional bi-directional gated recursive units (CBGRUs) with Adaptive Bandwidth Kernel Function Density Estimation (ABKDE) to address the challenge of multivariate time series interval prediction of WWTP water quality. Initially, wavelet transform (WT) was employed to smooth the water quality data, reducing noise and fluctuations. Linear correlation coefficient (CC) and non-linear mutual information (MI) techniques were then utilized to select input variables. The CBGRU model was applied to capture temporal correlations in the time series, integrating the Multiple Heads of Attention (MHA) mechanism to enhance the model's ability to comprehend complex relationships within the data. ABKDE was employed, supplemented by bootstrap to establish upper and lower bounds of the prediction intervals. Ablation experiments and comparative analyses with benchmark models confirmed the superior performance of the model in point prediction, interval prediction, the analysis of forecast period, and fluctuation detection for water quality data. Also, this study verifies the model's broad applicability and robustness to anomalous data. This study contributes significantly to improved effluent treatment efficiency and water quality control in WWTPs.

Enhanced federated anomaly detection through autoencoders using summary statistics-based thresholding

In Federated Learning, Anomaly Detection poses significant challenges due to the decentralized nature of data, especially under Non-IID distributions. This study proposes a federated threshold calculation method that aggregates summary statistics from normal and anomalous data across clients to create a global threshold for Anomaly Detection with federated Autoencoders, enhancing detection accuracy and robustness while ensuring privacy. Extensive experiments on datasets, including Credit Card Fraud Detection, Shuttle, and Covertype, show that our approach consistently outperforms existing federated and local threshold calculation methods. These findings highlight the potential of summary statistics in improving federated Anomaly Detection under Non-IID conditions.

Research on Oil Well Production Prediction Based on GRU-KAN Model Optimized by PSO

Accurately predicting oil well production volume is of great significance in oilfield production. To overcome the shortcomings in the current study of oil well production prediction, we propose a hybrid model (GRU-KAN) with the gated recurrent unit (GRU) and Kolmogorov–Arnold network (KAN). The GRU-KAN model utilizes GRU to extract temporal features and KAN to capture complex nonlinear relationships. First, the MissForest algorithm is employed to handle anomalous data, improving data quality. The Pearson correlation coefficient is used to select the most significant features. These selected features are used as input to the GRU-KAN model to establish the oil well production prediction model. Then, the Particle Swarm Optimization (PSO) algorithm is used to enhance the predictive performance. Finally, the model is evaluated on the test set. The validity of the model was verified on two oil wells and the results on well F14 show that the proposed GRU-KAN model achieves a Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Coefficient of Determination (R2) values of 11.90, 9.18, 6.0% and 0.95, respectively. Compared to popular single and hybrid models, the GRU-KAN model achieves higher production-prediction accuracy and higher computational efficiency. The model can be applied to the formulation of oilfield-development plans, which is of great theoretical and practical significance to the advancement of oilfield technology levels.

Enhanced robustness in machine learning: Application of an adaptive robust loss function

In the field of machine learning, the selection of a loss function plays a pivotal role in determining the training dynamics and generalization capability of models. Traditional loss functions such as mean square error (MSE) and cross-entropy are often not equipped to handle noisy and anomalous data effectively, which can result in models that perform poorly in practical scenarios. To address these shortcomings, this study introduces a novel adaptive robust loss function, enhanced by a tunable parameter , which allows for flexibility in adjusting the robustness of the function according to the nature of the data being processed. Our research demonstrates that this new loss function significantly improves the performance of linear regression and multilayer perceptron models, particularly in environments laden with noisy and anomalous data. By adapting the parameter , the function can cater to varying levels of data irregularities, thus enhancing the models accuracy and reliability across diverse and complex data environments. This adaptive mechanism not only offers a substantial theoretical contribution to the understanding of robust loss functions but also provides a practical tool for machine learning practitioners to develop models that are resilient to data imperfections. The implications of this research are profound, suggesting a shift towards more adaptive and robust approaches in machine learning model development.

Anomaly detection for bridge health monitoring data based on multiple encoded images and convolutional neural network

In bridge health monitoring systems, abnormal monitoring data inevitably appear due to the complex operational conditions, and these anomalous data will seriously affect the accuracy and reliability of the results of bridge health assessment. Hence, it is vital to detect the abnormal data before further processing. However, the pattern diversity and sample scarcity usually make the accuracy of data anomaly detection unsatisfactory. A novel anomaly detection method for bridge health monitoring data based on encoded images and convolutional neural network (CNN) is proposed in this paper. First, the raw time series are encoded into images to highlight the intrinsic features in the anomaly data, and multiple encoding techniques are adopted to represent the data in different perspectives. Then, the CNN is utilized to establish the mapping between the encoded images and the anomaly patterns for achieving the data anomaly detection. At last, the proposed method is verified with the acceleration data measured from an actual large span cable-stayed bridge. It shows that the proposed method can significantly improve the performance of data anomaly detection while the results are hardly affected by the small sample sizes, and a detection accuracy of over 95% is achieved based on the multiple encoded images.

Application of Machine Learning Models for Malware Classification With Real and Synthetic Datasets

Stacking of multiple Machine Learning (ML) classifiers have gained popularity in addressing anomalous data classification along with Deep Learning (DL) algorithms. This study compares traditional ML classifiers, multi-layer stacking ML classifiers, and DL classifiers using an open-source malware dataset-containing equal numbers of benign and malware samples. The results on the realistic dataset indicate that the DL classifier, utilizing a Bidirectional Long Short-Term Memory (BiLSTM) model, outperformed the stacked classifiers with Logistic Regression (LR) and Support Vector Machine (SVM) as Meta learners by 36.78% and 39.69%, respectively, in terms of classification accuracy and performance. The research work was extended to study the impact of Generative Adversarial Network (GAN) based synthetic dataset of relatively smaller size on deep learning models. It was observed that the Deep Learning Multi-Layer Perceptron (DLMLP) Model had relatively superior performance as compared to complex deep learning models like Long Short-Term Memory LSTM and BiLSTM

Adaptive Online Continual Learning for In-Situ Quality Prediction in Manufacturing Processes

Abstract Manufacturing processes undergo continuous changes in order to meet various requirements, such as process/product changes and variations in tool/workpiece conditions, leading to mixed, heterogenous, or anomalous data. As a result, a quality prediction model trained from previous data may not perform well when new tasks emerge. In order to achieve in-time and accurate product quality prediction, it is crucial to develop a predictive method that adapt to variations in the manufacturing system, capable of learning from new tasks without forgetting previous ones and detecting unknown tasks. This study proposes a deep learning method integrated with continual learning for in-situ quality prediction that is capable of learning from new tasks without forgetting previous ones. To demonstrate this idea, deep CNNs are designed to analyze in-process sensor data, which consist of shared layers to capture the common underlying features across all tasks, and task-specific layers that capture specific characteristics of each individual task. In order to identify the task to which incoming product belongs, a task prediction approach based on task relevancy using filter subspace distance is proposed. When new data come in, the model first identifies the task, followed by predicting the quality of the current product. The proposed method is demonstrated on two case studies, including quality prediction of the workpiece using acoustic emissions during the laser-induced plasma micro-machining process, and quality prediction of the product through thermal images during the hot stamping process.

Multimodal Fusion Anomaly Detection Model for Agricultural Wireless Sensors

ABSTRACTAgricultural Internet of Things has become one of the most important data sources of agricultural big data. However, the wireless sensors equipped with agricultural Internet of Things is affected by many factors, and anomalous data inevitably exists during the data collection process. The existence of anomalous data leads to the deterioration of the agricultural data quality, which hinders the efficient development of agricultural data analysis. In order to detect anomalous data accurately for agricultural wireless sensors, in this article, we propose an anomaly detection model that combines multimodal fusion and error reconstruction. The model first adds Gaussian noise to the input sequence and converts it into standard time series and image data. Subsequently, it is processed by different modal encoders and fuses the image and sequence modal data using Temporal Cross‐modal Attention module to enhance the perception of anomalous modal information. Finally, the fused data is reconstructed, and anomalous data are identified by the image and sequence decoders. Comparison experiments with several baselines prove the validity of the model proposed in this article, ablation experiments prove the necessity of the key modules in the model, and multiple sets of experiments are also designed to discuss the effects of hyperparameters.

Anomaly Detection in Fractal Time Series with LSTM Autoencoders

This study explores the application of neural networks for anomaly detection in time series data exhibiting fractal properties, with a particular focus on changes in the Hurst exponent. The objective is to investigate whether changes in fractal properties can be identified by transitioning from the analysis of the original time series to the analysis of the sequence of Hurst exponent estimates. To this end, we employ an LSTM autoencoder neural network, demonstrating its effectiveness in detecting anomalies within synthetic fractal time series and real EEG signals by identifying deviations in the sequence of estimates. Whittle’s method was utilized for the precise estimation of the Hurst exponent, thereby enhancing the model’s ability to differentiate between normal and anomalous data. The findings underscore the potential of machine learning techniques for robust anomaly detection in complex datasets.

An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contamination.

Most deep anomaly detection models are based on learning normality from datasets due to the difficulty of defining abnormality by its diverse and inconsistent nature. Therefore, it has been a common practice to learn normality under the assumption that anomalous data are absent in a training dataset, which we call normality assumption. However, in practice, the normality assumption is often violated due to the nature of real data distributions that includes anomalous tails, i.e., a contaminated dataset. Thereby, the gap between the assumption and actual training data affects detrimentally in learning of an anomaly detection model. In this work, we propose a learning framework to reduce this gap and achieve better normality representation. Our key idea is to identify sample-wise normality and utilize it as an importance weight, which is updated iteratively during the training. Our framework is designed to be model-agnostic and hyperparameter insensitive so that it applies to a wide range of existing methods without careful parameter tuning. We apply our framework to three different representative approaches of deep anomaly detection that are classified into one-class classification-, probabilistic model-, and reconstruction-based approaches. In addition, we address the importance of a termination condition for iterative methods and propose a termination criterion inspired by the anomaly detection objective. We validate that our framework improves the robustness of the anomaly detection models under different levels of contamination ratios on five anomaly detection benchmark datasets and two image datasets. On various contaminated datasets, our framework improves the performance of three representative anomaly detection methods, measured by area under the ROC curve.

Rapid quantitative analysis of three elements (Al, Mg and Fe) in molten zinc based on laser-induced breakdown spectroscopy combined with machine learning algorithm

Hot-dip galvanizing represents one of the most cost-effective methods for the prevention of metal corrosion, and is therefore employed extensively across a range of fields. Carrying out the research and development of technology and devices for quantitative analysis of chemical elements in hot-dip galvanising process can provide theoretical basis and technical support for the efficiency of hot-dip galvanising process and reduction of energy consumption. A machine learning-assisted LIBS combined with a programmable logic controller (PLC) for simultaneous on-line/in-site monitoring of multiple elements in hot-dip galvanising solution (molten zinc) was developed in the current study. The LIBS spectral data of the on-site hot-dip galvanising solution was collected under optimised experimental conditions. In order to further reduce the influence of experimental noise on the analysis performance, the on-site LIBS spectral data were preprocessed and anomalous spectral data were screened based on normalisation and principal component analysis-mahalanobis distance (PCA-MD). On the basis of the optimised data, component prediction models for three key elements of on-site hot-dip galvanising solution were constructed. The storage and re-call of the model was achieved based on python combined with LabVIEW, thus real-time prediction of the on-site component content of hot-dip galvanising solution was achieved. The results show that the random forest model presents the best prediction results, in which the R2 is 0.9978 and the RMSE is 0.0013% for Al, the R2 is 0.9984 and the RMSE is 0.0011% for Mg, and the R2 is 0.9932 and the RMSE is 0.0001% for Fe. From the on-site analysis results of the constructed model, its MRE of Al, Mg and Fe is 0.0098, 0.0236, and 0.2102, respectively. In summary, the in-situ/on-line analysis system of hot-dip galvanising solution combined with machine learning constructed in this study shows excellent performance, which can satisfy the needs of hot-dip galvanising solution production site. This study is expected to provide theoretical basis and technical reference for quality control and process optimisation in other production sites in the metallurgical field.

Advanced data-driven FBG sensor-based pavement monitoring system using multi-sensor data fusion and an unsupervised learning approach

Pavement health monitoring and assessment using sensor technology provides valuable insights into the condition of roads and helps in effective maintenance and management. This paper presents an advanced fiber Bragg grating (FBG) sensor-based pavement monitoring system that leverages multi-sensor data fusion techniques and an unsupervised learning approach for enhanced structural health assessment. The study commences with the fusion of data from various FBG sensors based on specific considerations and employing a novel function to improve the accuracy of pavement condition assessment. A dedicated data management system is introduced to address the challenges associated with continuous monitoring. This system automates data organization, pre-processing, and storage, facilitating historical trend analysis and post-processing. Then, the research introduces a novel unsupervised learning approach utilizing the density-based spatial clustering of applications with noise (DBSCAN) algorithm. This algorithm identifies abnormal patterns in the FBG sensor data, which may indicate structural anomalies or unexpected events, aiding the operator by automatically flagging abnormal data. By incorporating principle component analysis (PCA)-based feature fusion, the system reduces the dimensionality of the data, which enhances the efficiency of the clustering process. Various anomalies were detected in the data collected for four months from the test track. The cause and relevance of these anomalies were assessed to facilitate informed decision-making. This approach enables the operator to achieve early detection of potential structural issues and distinguish abnormal damage data from other anomalous data, thereby contributing to the overall success of the pavement monitoring system. In conclusion, the integration of advanced FBG sensor technology with multi-sensor data fusion and unsupervised learning techniques results in a sophisticated and robust pavement monitoring system. The presented approach enhances accuracy by detecting anomalies at any strain level, computational time using multi-sensor data fusion, and reliability of structural health assessment through anomaly evaluation. It also provides a framework for proactive decision-making, ultimately improving the longevity and structural integrity of critical infrastructure.

Enhanced capability of model fitting to nuclear level density and heat capacity: testing on 93−98Mo nuclei

We propose an improved fitting approach that improves reliability in studying the nuclear level density (NLD) and thermodynamic quantities. The proposed method, which relies on the fact that experimental fluctuations or outliers, if they exist, should not be involved in the fitting process, is validated with a set of data artificially generated with anomalous data points being intentionally inserted. In order to showcase the advantages of the proposed technique, we have applied it to re-investigate the back-shifted Fermi gas (BSFG) level density parameters and thermodynamic quantities, particularly the heat capacities, of 93−98Mo isotopes. We have found that the range of values for the level density parameter of 93Mo (approximately from 8.5 to 9.0 MeV−1) is notably smaller than that obtained for the other isotopes of Mo (approximately from 10.5 to 11.5 MeV−1). This observation is different from previous predictions, in which the values of level density parameter of all Mo isotopes are in the same range. This is because among the Mo isotopes under examination, 93Mo (N = 51 neutrons) has the smallest number of valence neutrons, namely only a single neutron away from the closed N = 50 shell. In addition, thanks to the proposed method, we have discussed the effects of data fluctuations on the BSFG NLDs and thermodynamic quantities of 93−98Mo isotopes, from which our recommendation for future works is announced. On top of that, we should notice that the proposed approach can be further applied to any work involving the fitting of a phenomenological model to empirical data.

Utilizing anomalous signals for element identification in macromolecular crystallography.

AlphaFold2 has revolutionized structural biology by offering unparalleled accuracy in predicting protein structures. Traditional methods for determining protein structures, such as X-ray crystallography and cryo-electron microscopy, are often time-consuming and resource-intensive. AlphaFold2 provides models that are valuable for molecular replacement, aiding in model building and docking into electron density or potential maps. However, despite its capabilities, models from AlphaFold2 do not consistently match the accuracy of experimentally determined structures, need to be validated experimentally and currently miss some crucial information, such as post-translational modifications, ligands and bound ions. In this paper, the advantages are explored of collecting X-ray anomalous data to identify chemical elements, such as metal ions, which are key to understanding certain structures and functions of proteins. This is achieved through methods such as calculating anomalous difference Fourier maps or refining the imaginary component of the anomalous scattering factor f''. Anomalous data can serve as a valuable complement to the information provided by AlphaFold2 models and this is particularly significant in elucidating the roles of metal ions.

The survey of industrial anomaly detection for industry 5.0

ABSTRACT With the impetus of Industry 5.0, smart manufacturing would witness the significant advancement of automation and digital transformation. Even though Deep Learning (DL) has been widely investigated in smart manufacturing to release the automation in industry 5.0, the difficulties still emerge in collecting the abundant defective labels and the appearance of new anomalies in the manufacturing system. Anomaly detection (AD) has emerged as a fundamental challenge in identifying irregularities within the manufacturing processes without rely on the large anomalous label data, but the discussion on the AD for smart manufacturing is still insufficient in industry 5.0. This research first discusses the context and definition of anomaly detection. Then, it focusses on DL-based AD method in industry areas and discusses the network design, advantages, and applications of the DL-based AD on industrial images. Additionally, the typical metrics used to evaluate DL-based methods and popular AD datasets in the industrial field have been described. Finally, the challenges and future prospects have been discussed.

VAE-WACGAN: An Improved Data Augmentation Method Based on VAEGAN for Intrusion Detection.

To address the class imbalance issue in network intrusion detection, which degrades performance of intrusion detection models, this paper proposes a novel generative model called VAE-WACGAN to generate minority class samples and balance the dataset. This model extends the Variational Autoencoder Generative Adversarial Network (VAEGAN) by integrating key features from the Auxiliary Classifier Generative Adversarial Network (ACGAN) and the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP). These enhancements significantly improve both the quality of generated samples and the stability of the training process. By utilizing the VAE-WACGAN model to oversample anomalous data, more realistic synthetic anomalies that closely mirror the actual network traffic distribution can be generated. This approach effectively balances the network traffic dataset and enhances the overall performance of the intrusion detection model. Experimental validation was conducted using two widely utilized intrusion detection datasets, UNSW-NB15 and CIC-IDS2017. The results demonstrate that the VAE-WACGAN method effectively enhances the performance metrics of the intrusion detection model. Furthermore, the VAE-WACGAN-based intrusion detection approach surpasses several other advanced methods, underscoring its effectiveness in tackling network security challenges.

Residual-enhanced graph convolutional networks with hypersphere mapping for anomaly detection in attributed networks

In the burgeoning field of anomaly detection within attributed networks, traditional methodologies often encounter the intricacies of network complexity, particularly in capturing nonlinearity and sparsity. This study introduces an innovative approach that synergizes the strengths of Graph Convolutional Networks (GCNs) with advanced deep residual learning and a unique residual-based attention mechanism, thereby creating a more nuanced and efficient method for anomaly detection in complex networks. The heart of our model lies in the integration of GCNs that capture complex structural relationships within the network data. This is further bolstered by deep residual learning, which is employed to model intricate nonlinear connections directly from input data. A pivotal innovation in our approach is the incorporation of a residual-based attention mechanism. This mechanism dynamically adjusts the importance of nodes based on their residual information, thereby significantly enhancing the sensitivity of the model to subtle anomalies. Furthermore, we introduce a novel hypersphere mapping technique in the latent space to distinctly separate normal and anomalous data. This mapping is the key to our model’s ability to pinpoint anomalies with greater precision. An extensive experimental setup was used to validate the efficacy of the proposed model. Using attributed social network datasets, we demonstrate that our model not only competes with but also surpasses existing state-of-the-art methods in anomaly detection. The results show the exceptional capability of our model to handle the multifaceted nature of real-world networks.

Mutation-Based Multivariate Time-Series Anomaly Generation on Latent Space with an Attention-Based Variational Recurrent Neural Network for Robust Anomaly Detection in an Industrial Control System

Anomaly detection involves identifying data that deviates from normal patterns. Two primary strategies are used: one-class classification and binary classification. In Industrial Control Systems (ICS), where anomalies can cause significant damage, timely and accurate detection is essential, often requiring analysis of time-series data. One-class classification is commonly used but tends to have a high false alarm rate. To address this, binary classification is explored, which can better differentiate between normal and anomalous data, though it struggles with class imbalance in ICS datasets. This paper proposes a mutation-based technique for generating ICS time-series anomalies. The method maps ICS time-series data into a latent space using a variational recurrent autoencoder, applies mutation operations, and reconstructs the time-series, introducing plausible anomalies that reflect multivariate correlations. Evaluations of ICS datasets show that these synthetic anomalies are visually and statistically credible. Training a binary classifier on data augmented with these anomalies effectively mitigates the class imbalance problem.

Generalized Time‐Series Analysis for In Situ Spacecraft Observations: Anomaly Detection and Data Prioritization Using Principal Components Analysis and Unsupervised Clustering

AbstractIn situ spacecraft observations are critical to our study and understanding of the various phenomena that couple mass, momentum, and energy throughout near‐Earth space and beyond. However, on‐orbit telemetry constraints can severely limit the capability of spacecraft to transmit high‐cadence data, and missions are often only able to telemeter a small percentage of their captured data at full rate. This presents a programmatic need to prioritize intervals with the highest probability of enabling the mission's science goals. Larger missions such as the Magnetospheric Multiscale mission (MMS) aim to solve this problem with a Scientist‐In‐The‐Loop (SITL), where a domain expert flags intervals of time with potentially interesting data for high‐cadence data downlink and subsequent study. Although suitable for some missions, the SITL solution is not always feasible, especially for low‐cost missions such as CubeSats and NanoSats. This manuscript presents a generalizable method for the detection of anomalous data points in spacecraft observations, enabling rapid data prioritization without substantial computational overhead or the need for additional infrastructure on the ground. Specifically, Principal Components Analysis and One‐Class Support Vector Machines are used to generate an alternative representation of the data and provide an indication, for each point, of the data's potential for scientific utility. The technique's performance and generalizability is demonstrated through application to intervals of observations, including magnetic field data and plasma moments, from the CASSIOPE e‐POP/Swarm‐Echo and MMS missions.

Adjacent Image Augmentation and Its Framework for Self-Supervised Learning in Anomaly Detection.

Anomaly detection has gained significant attention with the advancements in deep neural networks. Effective training requires both normal and anomalous data, but this often leads to a class imbalance, as anomalous data is scarce. Traditional augmentation methods struggle to maintain the correlation between anomalous patterns and their surroundings. To address this, we propose an adjacent augmentation technique that generates synthetic anomaly images, preserving object shapes while distorting contours to enhance correlation. Experimental results show that adjacent augmentation captures high-quality anomaly features, achieving superior AU-ROC and AU-PR scores compared to existing methods. Additionally, our technique produces synthetic normal images, aiding in learning detailed normal data features and reducing sensitivity to minor variations. Our framework considers all training images within a batch as positive pairs, pairing them with synthetic normal images as positive pairs and with synthetic anomaly images as negative pairs. This compensates for the lack of anomalous features and effectively distinguishes between normal and anomalous features, mitigating class imbalance. Using the ResNet50 network, our model achieved perfect AU-ROC and AU-PR scores of 100% in the bottle category of the MVTec-AD dataset. We are also investigating the relationship between anomalous pattern size and detection performance.