Anomaly Detection Task Research Articles

GAN Ensemble for Anomaly Detection

When formulated as an unsupervised learning problem, anomaly detection often requires a model to learn the distribution of normal data. Previous works modify Generative Adversarial Networks (GANs) by using encoder-decoders as generators and apply them to anomaly detection tasks. Previous studies indicate that GAN ensembles are often more stable than single GANs in image generation tasks. In this work, we propose to construct GAN ensembles for anomaly detection. In the proposed method, a group of generators interact with a group of discriminators, so every generator gets feedback from every discriminator, and vice versa. Compared to a single GAN, an ensemble of GANs can better model the distribution of normal data and thus better detect anomalies. We also make a theoretical analysis of GANs and GAN ensembles in the context of anomaly detection. The empirical study constructs ensembles based on four different types of detecting models, and the results show that the ensemble outperforms the single model for all four model types.

VAE-based Deep SVDD for anomaly detection

Anomaly detection is an essential task for different fields in the real world. The imbalanced data and lack of labels make the task challenging. Deep learning models based on autoencoder (AE) have been applied to address the above difficulties successfully. However, in these AE-based deep methods, the AE-based model’s optimization and the anomaly detector design are separated. Therefore, the latent representations in AE are less relevant for the anomaly detection task, which reduces the accuracy of anomaly detection. A deep support vector data description based on variational autoencoder (Deep SVDD-VAE) is proposed in this paper to solve this problem. In the proposed model, VAE is used to reconstruct the input instances, while a spherical discriminative boundary is learned with the latent representations simultaneously based on SVDD. Unlike existing AE-based methods, we seek the model parameters via the joint optimization of VAE and SVDD, which ensures the separability of the latent representations. Experimental results on MNIST, CIFAR-10, and GTSRB datasets show the effectiveness of Deep SVDD-VAE.

Early stage white etching crack identification using artificial neural networks

White Etching Cracks (WEC) in gearbox bearings is a major concern in the wind turbine industry, which can lead to a premature failure of the gearbox. Though many hypotheses regarding the generation of WEC have been proposed over the decades, the answer is still disputable. To trace back the failures to earlier stages before they occur, an innovative sensor-set has been utilized on a test rig to monitor the influencing factors that lead to WEC. This paperwork seeks to recognize abnormal patterns from recorded sensor data and derive statements of sensible sensor combinations in WEC early detection. A Long Short Term Memory (LSTM) network-based autoencoder is proposed for the anomaly detection (AD) task. Employing an auto-associative sequence-to-sequence predictor, a model is trained to reconstruct the normal time series data without WEC. The reconstruction error of testing time series data is evaluated for the determination of its anomaly. The results show that the specified LSTM autoencoder framework can qualitatively distinguish anomalies from collected multivariate time series data. Moreover, the anomaly score evaluated via reconstruction-error-based metrics can discriminate normal and abnormal behaviors in the study. This investigation’s results entail a significant step towards early WEC risk detection and more cost-efficient wind turbine technology if this approach can be further applied on stream data with plausible thresholds in monitoring system.

Recurrent Reconstructive Network for Sequential Anomaly Detection.

Anomaly detection identifies anomaly samples that deviate significantly from normal patterns. Usually, the number of anomaly samples is extremely small compared to the normal samples. To handle such imbalanced sample distribution, one-class classification has been widely used in identifying the anomaly by modeling the features of normal data using only normal data. Recently, recurrent autoencoder (RAE) has shown outstanding performance in the sequential anomaly detection compared to the other conventional methods. However, RAE, which has a long-term dependency problem, is optimized only to handle the fixed-length inputs. To overcome the limitations of RAE, we propose recurrent reconstructive network (RRN) as a novel RAE, with three functionalities for anomaly detection of streaming data: 1) a self-attention mechanism; 2) hidden state forcing; and 3) skip transition. The designed self-attention mechanism and the hidden state forcing between the encoder and decoder effectively manage the input sequences of varying length. The skip transition with the attention gate improves the reconstruction performance. We conduct a series of comprehensive experiments on four datasets and verify the superior performance of the proposed RRN in the sequential anomaly detection tasks.

Integrated Multiscale Appearance Features and Motion Information Prediction Network for Anomaly Detection.

The rise of video-prediction algorithms has largely promoted the development of anomaly detection in video surveillance for smart cities and public security. However, most current methods relied on single-scale information to extract appearance (spatial) features and lacked motion (temporal) continuity between video frames. This can cause a loss of partial spatiotemporal information that has great potential to predict future frames, affecting the accuracy of abnormality detection. Thus, we propose a novel prediction network to improve the performance of anomaly detection. Due to the objects of various scales in each video, we use different receptive fields to extract detailed appearance features by the hybrid dilated convolution (HDC) module. Meanwhile, the deeper bidirectional convolutional long short-term memory (DB-ConvLSTM) module can remember the motion information between consecutive frames. Furthermore, we use RGB difference loss to replace optical flow loss as temporal constraint, which greatly reduces the time for optical flow extraction. Compared with the state-of-the-art methods in the anomaly-detection task, experiments prove that our method can more accurately detect abnormalities in various video surveillance scenes.

Using Deep Autoencoders for In-vehicle Audio Anomaly Detection

Current developments on self-driving cars have increased the interest on autonomous shared taxicabs. While most self-driving technologies focus on the outside environment, there is also a need to provide in-vehicle intelligence (e.g., detect health and safety issues related with the car occupants). Set within an R&D project focused on in-vehicle cockpit intelligence, the research presented in this paper addresses an unsupervised Acoustic Anomaly Detection (AAD) task. Since data is nonexistent in this domain, we first design an in-vehicle sound event data simulator that can realistically mix background audios (recorded from car driving trips) with normal (e.g., people talking, radio on) and abnormal (e.g., people arguing, cough) event sounds, allowing the generation of three synthetic in-vehicle sound datasets. Then, we explore two main sound feature extraction methods (based on a combination of three audio features and mel frequency energy coefficients) and propose a novel Long Short-Term Memory Autoencoder (LSTM-AE) deep learning architecture for in-vehicle sound anomaly detection. Competitive results were achieved by the proposed LSTM-AE when compared with two state-of-the-art methods, namely a dense Autoencoder (AE) and a two-stage clustering.

A Multimode Anomaly Detection Method Based on OC-ELM for Aircraft Engine System

The practical industrial processes possess the characteristics of multimode, unbalanced data distribution, and complex types of abnormalities, which are challenging to the anomaly detection task of complex industrial systems. In this paper, a novel anomaly detection framework based on one-class extreme learning machine (OC-ELM) for the multimode system is presented. To tackle the multiple operation modes, a clustering algorithm is first applied to distinguish the operation modes of the system. The corresponding detection models are built under different operation modes resulting in the multiple models operated in parallel. In addition, the proposed method constructs the reasonable boundary of the complex data distribution, reflecting the equipment running in the healthy or the normal state. The anomaly detection index is obtained according to the deviation degree between the testing sample and the normal model. As a result, a global monitoring index reflecting the degradation state is obtained by combining the anomaly monitoring indices of the equipment under multiple operation modes. The proposed method is verified on a public dataset of aircraft engines, and the advantages are demonstrated by comparing with the implemented detection model without handling the information of operation modes, and the multiple principal component analysis method.

A Novel Filter-Based Anomaly Detection Framework for Hyperspectral Imagery

Hyperspectral images stand out from other remote sensing images in anomaly target detection because they contain unique distinguishing spectral information and attract great interest in applications of search and rescue. However, most of the popular techniques for hyperspectral anomaly detection tasks focus on improving accuracy with complicated algorithms and face difficulty in efficiently balancing performance and complexity. In this paper, we propose a novel anomaly detection approach using a selected band image extracted from the band selection model combined with an image filter. Singular value decomposition (SVD) is adopted for spectral dimensionality reduction. A dual-window guided filter is constructed to highlight the potential anomaly targets. To quickly calculate the abnormity degree, we design an efficient diagonal matrix operation to achieve the energy of each pixel, and a spatial regulation model is designed to enhance the subpixel target detection performance. Extensive experiments conducted on two real-world hyperspectral datasets demonstrate that, compared with the existing relevant state-of-the-art approaches, the proposed method requires less detection time and achieves higher detection accuracy.

Imbalanced satellite telemetry data anomaly detection model based on Bayesian LSTM

Anomaly detection of satellite telemetry data has always been a significant issue in the development of aeronautics and astronautics. Timely and effective anomaly detection method of satellite telemetry data is a research hotspot in academia and aerospace industry. For satellite telemetry data, we propose an anomaly detection model based on Bayesian deep learning without domain knowledge. In this model, we show the feasibility of implementing MC-dropout on the Long Short-Term Memory Network (LSTM), and establish the Bayesian LSTM. First, we perform a preliminary anomaly detection task through our model—Monte Carlo Dropout Bidirectional Long Short-term Memory Network (MCD-BiLSTM). Then, Monte Carlo (MC) Sampling Variance, Prediction Entropy and Mutual Information are taken to measure the uncertainty of output through MCD-BiLSTM. What's more, we further explore and exploit the three types of uncertainties, and utilize the variational auto-encoder (VAE) to reevaluate the high uncertainty samples to improve the anomaly detection capability. To our knowledge, it is the first time that Bayesian neural networks have been introduced into the field of satellite telemetry data anomaly detection. The experimental results on an imbalanced satellite telemetry dataset show that our proposed model can add effective regularization constraints, and obtain great robustness on imbalanced data, which performs better than popular traditional neural networks and other Bayesian neural networks.

Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection

Anomaly detection in the automated optical quality inspection is of great important for guaranteeing the surface quality of industrial products. Most related methods are based on supervised learning techniques, which require a large number of normal and anomalous samples to obtain a robust classifier. However, the diversity of potential defects and low availability of defective samples during manufacturing bring more challenges to anomaly detection. Based on the encoder-decoder-encoder paradigm, a semi-supervised anomaly detection method Dual Prototype Auto-Encoder (DPAE) is proposed in this paper. At the training stage, the dual prototype loss and reconstruction loss are introduced to encourage the latent vectors generated by the encoders to keep closer to their own prototype. Therefore, two latent vectors of the normal image tend to be closer, and large distance between the latent vectors indicates an anomaly. And we also construct the Aluminum Profile Surface Defect (APSD) dataset for the anomaly detection task. Finally, extensive experiments on four datasets show that DPAE is effective and outperforms state-of-the-art methods.

Neural memory plasticity for medical anomaly detection

In the domain of machine learning, Neural Memory Networks (NMNs) have recently achieved impressive results in a variety of application areas including visual question answering, trajectory prediction, object tracking, and language modelling. However, we observe that the attention based knowledge retrieval mechanisms used in current NMNs restrict them from achieving their full potential as the attention process retrieves information based on a set of static connection weights. This is suboptimal in a setting where there are vast differences among samples in the data domain; such as anomaly detection where there is no consistent criteria for what constitutes an anomaly. In this paper, we propose a plastic neural memory access mechanism which exploits both static and dynamic connection weights in the memory read, write and output generation procedures. We demonstrate the effectiveness and flexibility of the proposed memory model in three challenging anomaly detection tasks in the medical domain: abnormal EEG identification, MRI tumour type classification and schizophrenia risk detection in children. In all settings, the proposed approach outperforms the current state-of-the-art. Furthermore, we perform an in-depth analysis demonstrating the utility of neural plasticity for the knowledge retrieval process and provide evidence on how the proposed memory model generates sparse yet informative memory outputs.

Transfer Learning for Anomaly Detection through Localized and Unsupervised Instance Selection

Anomaly detection attempts to identify instances that deviate from expected behavior. Constructing performant anomaly detectors on real-world problems often requires some labeled data, which can be difficult and costly to obtain. However, often one considers multiple, related anomaly detection tasks. Therefore, it may be possible to transfer labeled instances from a related anomaly detection task to the problem at hand. This paper proposes a novel transfer learning algorithm for anomaly detection that selects and transfers relevant labeled instances from a source anomaly detection task to a target one. Then, it classifies target instances using a novel semi-supervised nearest-neighbors technique that considers both unlabeled target and transferred, labeled source instances. The algorithm outperforms a multitude of state-of-the-art transfer learning methods and unsupervised anomaly detection methods on a large benchmark. Furthermore, it outperforms its rivals on a real-world task of detecting anomalous water usage in retail stores.

An edge-stream computing infrastructure for real-time analysis of wearable sensors data

The fast development of IoT in general and wearable smart sensors in particular in the context of wellness and healthcare are demanding for definition of specific infrastructure supporting real time data analysis for anomaly detection, event identification, situation awareness just to mention few. The explosion in the development and adoption of these smart wearable sensors has contributed to the definition of the Internet of Medical Things (IoMT), which is revolutionizing the way healthcare is tackled worldwide. Data produced by wearable sensors continuously grow and could be spread among clinical centers, hospitals, research labs, yielding to a Big Data management problem. In this paper we propose a technological and architectural solution, based on Open Source big data technologies to perform real-time analysis of wearable sensor data streams. The proposed architecture is composed of four distinct layers: a sensing layer, a pre-processing layer (Raspberry Pi), a cluster processing layer (Kafka’s broker and Flink’s mini-cluster) and a persistence layer (Cassandra database). A performance evaluation of each layer has been carried out by considering CPU and memory usage for accomplishing a simple anomaly detection task using the REALDISP dataset.

Hyperspectral Anomaly Detection via Discriminative Feature Learning with Multiple-Dictionary Sparse Representation

Most hyperspectral anomaly detection methods directly utilize all the original spectra to recognize anomalies. However, the inherent characteristics of high spectral dimension and complex spectral correlation commonly make their detection performance unsatisfactory. Therefore, an effective feature extraction technique is necessary. To this end, this paper proposes a novel anomaly detection method via discriminative feature learning with multiple-dictionary sparse representation. Firstly, a new spectral feature selection framework based on sparse presentation is designed, which is closely guided by the anomaly detection task. Then, the representative spectra which can significantly enlarge anomaly’s deviation from background are picked out by minimizing residues between background spectrum reconstruction error and anomaly spectrum recovery error. Finally, through comprehensively considering the virtues of different groups of representative features selected from multiple dictionaries, a global multiple-view detection strategy is presented to improve the detection accuracy. The proposed method is compared with ten state-of-the-art methods including LRX, SRD, CRD, LSMAD, RSAD, BACON, BACON-target, GRX, GKRX, and PCA-GRX on three real-world hyperspectral images. Corresponding to each competitor, it has the average detection performance improvement of about 9.9 % , 7.4 % , 24.2 % , 10.1 % , 26.2 % , 20.1 % , 5.1 % , 19.3 % , 10.7 % , and 2.0 % respectively. Extensive experiments demonstrate its superior performance in effectiveness and efficiency.

Manifold learning techniques for unsupervised anomaly detection

Abstract Appropriately identifying outlier data is a critical requirement in the decision-making process of many expert and intelligent systems deployed in a variety of fields including finance, medicine, and defense. Classical outlier detection schemes typically rely on the assumption that normal/background data of interest are distributed according to an assumed statistical model and search for data that deviate from that assumption. However, it is frequently the case that performance is reduced because the underlying distribution does not follow the assumed model. Manifold learning techniques offer improved performance by learning better models of the background but can be too computationally expensive due to the need to calculate a distance measure between all data points. Here, we study a general framework that allows manifold learning techniques to be used for unsupervised anomaly detection by reducing computational expense via a uniform random sampling of a small fraction of the data. A background manifold is learned from the sample and then an out-of-sample extension is used to project unsampled data into the learned manifold space and construct an anomaly detection statistic based on the prediction error of the learned manifold. The method works well for unsupervised anomaly detection because, by definition, the ratio of anomalous to non-anomalous data points is small and the sampling will be dominated by background points. However, a variety of parameters that affect detection performance are introduced so we use here a low-dimensional toy problem to investigate their effect on the performance of four learning algorithms (kernel PCA, two versions of diffusion map, and the Parzen density estimator). We then apply the methods to the detection of watercraft in an ensemble of 22 infrared maritime scenes where we find kernel PCA to be superior and show that it outperforms a commonly employed baseline algorithm. The framework is not limited to the tested image processing example and can be used for any unsupervised anomaly detection task.

Exploration and evaluation of AR, MPCA and KL anomaly detection techniques to embankment dam piezometer data

In the U.S., the current practice of analyzing the structural integrity of embankment dams relies primarily on manual a posteriori analysis of instrument data by engineers, leaving much room for improvement through the application of advanced data analysis techniques. In this research, different types of anomaly detection techniques are examined in an effort to propose which data analytics are appropriate for various anomaly scenarios as well as piezometer locations. Moreover, both the parametric (Auto Regressive [AR] and Moving Principal Component Analysis [MPCA]) and nonparametric (Kullback–Leibler Divergence [KL]) techniques are applied in order to test if the widely-held assumptions about piezometer data, i.e., linearity between piezometer data and pool levels, as well as normally distributed piezometer data, are necessary in the anomaly detection task. In general, KL performs better than MPCA and AR, and delivers more consistent results throughout the different piezometers and anomaly scenarios. Given that KL is a nonparametric technique, the authors conclude that the prior assumptions about piezometer data do not always provide the best performance for anomaly prediction.

FRaC: a feature-modeling approach for semi-supervised and unsupervised anomaly detection

Anomaly detection involves identifying rare data instances (anomalies) that come from a different class or distribution than the majority (which are simply called "normal" instances). Given a training set of only normal data, the semi-supervised anomaly detection task is to identify anomalies in the future. Good solutions to this task have applications in fraud and intrusion detection. The unsupervised anomaly detection task is different: Given unlabeled, mostly-normal data, identify the anomalies among them. Many real-world machine learning tasks, including many fraud and intrusion detection tasks, are unsupervised because it is impractical (or impossible) to verify all of the training data. We recently presented FRaC, a new approach for semi-supervised anomaly detection. FRaC is based on using normal instances to build an ensemble of feature models, and then identifying instances that disagree with those models as anomalous. In this paper, we investigate the behavior of FRaC experimentally and explain why FRaC is so successful. We also show that FRaC is a superior approach for the unsupervised as well as the semi-supervised anomaly detection task, compared to well-known state-of-the-art anomaly detection methods, LOF and one-class support vector machines, and to an existing feature-modeling approach.

Low-Complexity Principal Component Analysis for Hyperspectral Image Compression

Principal component analysis (PCA) is an effective tool for spectral decorrelation of hyperspectral imagery, and PCAbased spectral transforms have been employed successfully in conjunction with JPEG2000 for hyperspectral image compression. However, the computational cost of determining the data-dependent PCA transform is high because of its traditional eigendecomposition implementation which requires calculation of a covariance matrix across the data. Several strategies for reducing the computation burden of PCA are explored, including both spatial and spectral subsampling in the covariance calculation as well as an iterative algorithm that circumvents determination of the covariance matrix entirely. Experimental results investigate the impacts of such low-complexity PCA on JPEG2000 compression of hyperspectral images, focusing on rate-distortion performance as well as data-analysis performance at an anomaly-detection task.

Network intrusion detection through Adaptive Sub-Eigenspace Modeling in multiagent systems

Recently, network security has become an extremely vital issue that beckons the development of accurate and efficient solutions capable of effectively defending our network systems and the valuable information journeying through them. In this article, a distributed multiagent intrusion detection system (IDS) architecture is proposed, which attempts to provide an accurate and lightweight solution to network intrusion detection by tackling issues associated with the design of a distributed multiagent system, such as poor system scalability and the requirements of excessive processing power and memory storage. The proposed IDS architecture consists of (i) the Host layer with lightweight host agents that perform anomaly detection in network connections to their respective hosts, and (ii) the Classification layer whose main functions are to perform misuse detection for the host agents, detect distributed attacks, and disseminate network security status information to the whole network. The intrusion detection task is achieved through the employment of the lightweight Adaptive Sub-Eigenspace Modeling (ASEM)-based anomaly and misuse detection schemes. Promising experimental results indicate that ASEM-based schemes outperform the KNN and LOF algorithms, with high detection rates and low false alarm rates in the anomaly detection task, and outperform several well-known supervised classification methods such as C4.5 Decision Tree, SVM, NN, KNN, Logistic, and Decision Table (DT) in the misuse detection task. To assess the performance in a real-world scenario, the Relative Assumption Model, feature extraction techniques, and common network attack generation tools are employed to generate normal and anomalous traffic in a private LAN testbed. Furthermore, the scalability performance of the proposed IDS architecture is investigated through the simulation of the proposed agent communication scheme, and satisfactory linear relationships for both degradation of system response time and agent communication generated network traffic overhead are achieved.

Hyperspectral Image Compression Using JPEG2000 and Principal Component Analysis

Principal component analysis (PCA) is deployed in JPEG2000 to provide spectral decorrelation as well as spectral dimensionality reduction. The proposed scheme is evaluated in terms of rate-distortion performance as well as in terms of information preservation in an anomaly-detection task. Additionally, the proposed scheme is compared to the common approach of JPEG2000 coupled with a wavelet transform for spectral decorrelation. Experimental results reveal that, not only does the proposed PCA-based coder yield rate-distortion and information-preservation performance superior to that of the wavelet-based coder, the best PCA performance occurs when a reduced number of PCs are retained and coded. A linear model to estimate the optimal number of PCs to use in such dimensionality reduction is proposed