Normal Training Data Research Articles

A new method for fault detection of aero-engine based on isolation forest

The research on fault detection of aero-engine is of great significance to its safe and reliable operation. In this paper, a dynamic threshold method for aero-engine fault detection based on Isolation Forest (iForest) is proposed. The proposed method can use only normal aero-engine data for training to build the fault detection model, which solves the problem that there is no large amount of fault data for training in the field of aero-engine fault detection due to the limitations of actual conditions. The method is verified by the residual data of the turbofan engine gas path system which is generated by the state variable model under three different fault states. Compared with the results of other methods, it is found that the proposed method can not only achieve high detection accuracy but also has a short running time. It is proved that the proposed method is suitable for fault detection of aero-engine.

A Novel Deep Learning Approach for Anomaly Detection of Time Series Data

Anomalies in time series, also called “discord,” are the abnormal subsequences. The occurrence of anomalies in time series may indicate that some faults or disease will occur soon. Therefore, development of novel computational approaches for anomaly detection (discord search) in time series is of great significance for state monitoring and early warning of real-time system. Previous studies show that many algorithms were successfully developed and were used for anomaly classification, e.g., health monitoring, traffic detection, and intrusion detection. However, the anomaly detection of time series was not well studied. In this paper, we proposed a long short-term memory- (LSTM-) based anomaly detection method (LSTMAD) for discord search from univariate time series data. LSTMAD learns the structural features from normal (nonanomalous) training data and then performs anomaly detection via a statistical strategy based on the prediction error for observed data. In our experimental evaluation using public ECG datasets and real-world datasets, LSTMAD detects anomalies more accurately than other existing approaches in comparison.

GAN을 사용한 재구성 오차 기반의 이상감지

Anomaly detection is a technique used to identify unusual patterns that do not conform to expected behavior such as defects or unusual situations. Various generative adversarial network (GAN) and autoencoder based methods have been suggested and shown much progress in this field. ABC[5], an existing representative supervised anomalies detection method, generates blur images for normal data, and the distribution of reconstruction errors for normal and abnormal overlaps considerably. In order to solve these problems, we proposed an improved anomaly detection method using autoencoder and GAN. The proposed method combine various loss functions of both supervised and unsupervised anomaly detection utilizing both normal and abnormal training data. We demonstrate the proposed GAN based anomaly detection by performing experiments on Fashion-MNIST, and real-world industrial dataset – metal surface defects. Compared to ABC[5], our model is superior to the previous approach in terms of average area under the ROC curve (AUROC) and distribution of the reconstruction errors for normal and anomaly data.

Feature Selection and Ensemble Learning Techniques in One-Class Classifiers: An Empirical Study of Two-Class Imbalanced Datasets

Class imbalance learning is an important research problem in data mining and machine learning. Most solutions including data levels, algorithm levels, and cost sensitive approaches are derived using multi-class classifiers, depending on the number of classes to be classified. One-class classification (OCC) techniques, in contrast, have been widely used for anomaly or outlier detection where only normal or positive class training data are available. In this study, we treat every two-class imbalanced dataset as an anomaly detection problem, which contains a larger number of data in the majority class, i.e. normal or positive class, and a very small number of data in the minority class. The research objectives of this paper are to understand the performance of OCC classifiers and examine the level of performance improvement when feature selection is considered for pre-processing the training data in the majority class and ensemble learning is employed to combine multiple OCC classifiers. Based on 55 datasets with different ranges of class imbalance ratios and one-class support vector machine, isolation forest, and local outlier factor as the representative OCC classifiers, we found that the OCC classifiers are good at high imbalance ratio datasets, outperforming the C4.5 baseline. In most cases, though, performing feature selection does not improve the performance of the OCC classifiers in most. However, many homogeneous and heterogeneous OCC classifier ensembles do outperform the single OCC classifiers, with some specific combinations of multiple OCC classifiers, both with and without feature selection, performing similar to or better than the baseline combination of SMOTE and C4.5.

ConNet: Deep Semi-Supervised Anomaly Detection Based on Sparse Positive Samples

Existing semi-supervised anomaly detection methods usually use a large amount of labeled normal data for training, which have the problem of high labeling costs. Only a few semi-supervised methods utilize unlabeled data and a few labeled anomalies to train models. However, these kinds of methods usually encounter two problems: (i) since anomalies usually have different behavior patterns or the internal mechanisms that produce anomalies are complex and diverse, a few labeled anomalies cannot cover all anomaly types; and (ii) the amount of unlabeled data in the training set is substantially greater than the amount of labeled data, which leads to that unlabeled data with contamination often dominates the training process. To solve these two problems, we propose the semi-supervised anomaly detection method named ConNet and a new loss function named concentration loss. Specifically, ConNet consists of two stages. Firstly, we obtain the prior anomaly score of unlabeled data via prior estimation module and attach the prior anomaly score to unlabeled data as the training weight. Then, an anomaly scoring network is training to assign anomaly scores to data instances, which can ensure that the anomaly scores of anomalies significantly deviate from those of normal instances. We have conducted experiments on thirteen real-world data sets and tested the performance of our method in terms of detection accuracy, utilization efficiency of labeled data, and robustness to different contamination rates. The experimental results show that the performance of our method is significantly better than those of the state-of-the-art anomaly detection methods.

A Unified Framework and Method for EEG-Based Early Epileptic Seizure Detection and Epilepsy Diagnosis

Electroencephalogram (EEG) contains important physiological information that can reflect the activity of human brain, making it useful for epileptic seizure detection and epilepsy diagnosis. However visual inspection of large amounts of EEG by human expert is time-consuming, and meanwhile there are often inconsistences in judgement between physicians. In this paper, we develop a unified framework for early epileptic seizure detection and epilepsy diagnosis, which includes two phases. In the first phase, the signal intensity is first calculated for each data point of the given EEG, enabling the well-known autoregressive moving average (ARMA) model to characterize the dynamic behavior of the EEG time series. The residual error between the predicted value of learned ARMA model and the actually observed value is used as the anomaly score to support a null hypothesis testing for making epileptic seizure decision. The epileptic seizure detection phase can provide a quick detection for anomaly EEG patterns, but the resulting suspicious segment may include epilepsy or other disordering EEG activities thus required to be identified. Therefore, in the second phase, we use pattern recognition technique to classify the suspicious EEG segments. In particular, we propose a new and practical classifier based on a pairwise of one-class SVMs for epilepsy diagnosis. The proposed classifier requires normal and epilepsy data for training, but can recognize normal, epilepsy and even other disorders that would not be trained in the training samples. This point is practical and meaningful in real clinic scenarios as the input EEG may include other brain disordering diseases besides of epilepsy. We conducted experiments on the publicly-available Bern-Barcelona and CHB-MIT EEG database, respectively, to validate the effectiveness of the proposed framework, and our method achieved classification accuracy of 93% and 94% on them. Comprehensive experimental results, outperforming the state-of-the-arts , suggest its great potentials in real applications.

Anomaly detection in a fleet of industrial assets with hierarchical statistical modeling

AbstractAnomaly detection in asset condition data is critical for reliable industrial asset operations. But statistical anomaly classifiers require certain amount of normal operations training data before acceptable accuracy can be achieved. The necessary training data are often not available in the early periods of assets operations. This problem is addressed in this paper using a hierarchical model for the asset fleet that systematically identifies similar assets, and enables collaborative learning within the clusters of similar assets. The general behavior of the similar assets are represented using higher level models, from which the parameters are sampled describing the individual asset operations. Hierarchical models enable the individuals from a population, comprising of statistically coherent subpopulations, to collaboratively learn from one another. Results obtained with the hierarchical model show a marked improvement in anomaly detection for assets having low amount of data, compared to independent modeling or having a model common to the entire fleet.

DLSTM: a new approach for anomaly detection using deep learning with delayed prediction

In this paper, we propose delayed Long Short-Term Memory (dLSTM), an anomaly detection method for time-series data. We first build a predictive model from normal (non-anomalous) training data, then perform anomaly detection based on the prediction error for observed data. However, there are multiple states in the waveforms of normal data, which may lower prediction accuracy. To deal with this problem, we utilize multiple prediction models based on LSTM for anomaly detection. In this scheme, the prediction accuracy strongly depends on the method of selecting a proper predictive model from multiple possible models. We propose a novel method to determine the proper predictive model for anomaly detection. Our approach provides multiple predicted value candidates in advance and selects the one that is closest to the measured value. We delay the model selection until the corresponding measured values are acquired. Using this concept for anomaly detection, dLSTM selects the proper predictive model to enhance prediction accuracy. In our experimental evaluation using real and artificial data, dLSTM detects anomalies more accurately than methods in comparison.

Abnormal events detection using deep neural networks: application to extreme sea surface temperature detection in the Red Sea

We present a method based on deep learning for detecting and localizing abnormal/extreme events in sea surface temperature (SST) of the Red Sea images using training samples of normal events only. The method operates in two stages; the first one involves features extraction from each patch of the SST input image using the first two convolutional layers extracted from a pretrained convolutional neural network. In the second stage, two methods are used for training the model from the normal training data. The first method uses one-class support vector machine (1-SVM) classifier that allows a fast and robust abnormal detection in the presence of outliers in the training dataset. In the second method, a Gaussian model is defined on the Mahalanobis distances between all normal training data. Experimental tests are conducted on satellite-derived SST data of the Red Sea spanning for a period of 31 years (1985–2015). Our results suggest that the Gaussian model of Mahalanobis distances outperformed 1-SVM by providing better performance in terms of sensitivity and specificity.

Primary-Auxiliary Statistical Local Kernel Principal Component Analysis and Its Application to Incipient Fault Detection of Nonlinear Industrial Processes

Statistical local kernel principal component analysis (SLKPCA) has demonstrated its success in incipient fault detection of nonlinear industrial processes by incorporating the statistical local analysis (SLA) technology. However, the basic SLKPCA method builds the statistical model only based on the normal data and neglects the utilization of the prior fault information, which is often available in many industrial cases. To take full advantage of the prior fault information, this paper proposes an enhanced SLKPCA method, called primary-auxiliary SLKPCA (PA-SLKPCA), for better incipient fault monitoring. The contribution of the proposed method includes three aspects. First, one primary-auxiliary statistical monitoring framework is designed, by which not only the normal training data are applied to develop a primary SLKPCA model, but also the prior fault data are used to build the auxiliary SLKPCA models. Second, a double-block modeling strategy is developed to construct the auxiliary SLKPCA model for each fault case, where a variable grouping strategy based on Kullback-Leibler divergence is applied to divide the process variables into the fault-relevant group and fault-independent variable group, and the sub-model is developed for each group. Third, the Bayesian inference is used to combine the statistical results of each variable group, and one weighted fusion strategy is further designed to integrate the monitoring results from the primary and auxiliary models. Lastly, two case studies including one numerical system and the simulated continuous stirred tank reactor (CSTR) system are used for method evaluation and the simulations show that the proposed method can detect the incipient faults effectively and outperform the traditional SLKPCA method.

Deep Discriminative Representation Learning for Nonlinear Process Fault Detection

Nonlinear process fault detection remains a challenge, with representation learning being a key step. In this article, a deep neural network (DNN)-based discriminative representation learning approach is proposed to achieve efficient fault detection for nonlinear plant-wide processes. An early-stage fault rarely affects several independent variables concurrently; hence, mutual information-based block division and randomized fault construction are performed to generate faulty validation data. By using the training data from the normal operation training data and the constructed validation data, a DNN with stacked autoencoders and a softmax classifier is trained to generate discriminative representations that maximize the capability of discriminating normal and abnormal statuses. Finally, on the basis of the learned deep discriminative representations, support vector data description is employed to discriminate the normal and abnormal process statuses. The proposed monitoring approach is tested on a numerical example and an industrial tail-gas treatment process, through which the efficiency is verified. Note to Practitioners —A modern process is generally characterized by a large scale and complex nonlinear correlation, and monitoring of such nonlinear plant-wide processes is imperative. Nowadays, a large amount of process data is generally available, and deep neural network-based monitoring is promising in dealing with such data on nonlinear processes. This article proposes a deep discriminative representation learning method for efficient nonlinear plant-wide process monitoring. The key idea is to first decompose a large-scale process into multiple units according to a variable relationship, and then generate faulty validation data based on randomized fault construction. Then, deep discriminative representations are learned by optimizing a stacked autoencoder-based deep neural network (DNN). This article provides guidelines for designing an efficient monitoring algorithm for plant-wide nonlinear processes in the industrial big data environment.

Spectral Kurtosis Entropy and Weighted SaE-ELM for Bogie Fault Diagnosis under Variable Conditions.

Bogies are crucial for the safe operation of rail transit systems and usually work under uncertain and variable operating conditions. However, the diagnosis of bogie faults under variable conditions has barely been discussed until now. Thus, it is valuable to develop effective methods to deal with variable conditions. Besides, considering that the normal data for training are much more than the faulty data in practice, there is another problem in that only a small amount of data is available that includes faults. Concerning these issues, this paper proposes two new algorithms: (1) A novel feature parameter named spectral kurtosis entropy (SKE) is proposed based on the protrugram. The SKE not only avoids the manual post-processing of the protrugram but also has strong robustness to the operating conditions and parameter configurations, which have been validated by a simulation experiment in this paper. In this paper, the SKE, in conjunction with variational mode decomposition (VMD), is employed for feature extraction under variable conditions. (2) A new learning algorithm named weighted self-adaptive evolutionary extreme learning machine (WSaE-ELM) is proposed. WSaE-ELM gives each sample an extra sample weight to rebalance the training data and optimizes these weights along with the parameters of hidden neurons by means of the self-adaptive differential evolution algorithm. Finally, the hybrid method based on VMD, SKE, and WSaE-ELM is verified by using the vibration signals gathered from real bogies with speed variations. It is demonstrated that the proposed method of bogie fault diagnosis outperforms the conventional methods by up to 4.42% and 6.22%, respectively, in percentages of accuracy under variable conditions.

Anomaly detection using sparse reconstruction in crowded scenes

In this paper, we propose an algorithm of anomaly detection in crowded scenes by using sparse representation over the normal bases. First, the histogram of maximal optical flow projection (HMOFP) features are extracted from a set of normal training data. Then, the online dictionary learning algorithm is used to train an optimal dictionary with proper redundancy, which is better than the dictionary simply composed by the HMOFP features of the whole training data. In order to detect the normalness of a frame, the l1-norm of the sparse reconstruction coefficients is used as the Reconstruction Coefficient Sparsity (RCS). Our algorithm is effective for both global abnormal events (GAE) and local abnormal events (LAE). We evaluate our method on three benchmark datasets-the UMN dataset, the PETS2009 dataset and the UCSD Ped1 dataset. Compared with the most popular methods, experimental results show that our algorithm achieves good results especially for the pixel-level local abnormal event localization.

Fast and Efficient Outlier Detection Method in Wireless Sensor Networks

Outlier detection is the task of classifying test data that differ in some respect from the data that are available during training. This may be seen as one-class classification, in which a model is constructed to describe normal training data. In wireless sensor networks (WSNs), the outlier detection process is a necessary step in building sensor network systems to assure data quality for perfect decision making. For this case, the task amounts to create a useful model based on kernel principal component analysis (KPCA) to recognize data as normal or outliers. Recently, KPCA has used for nonlinear case which can extract higher order statistics. KPCA mapping the data onto another feature space and using nonlinear function. On account of the attractive capability, KPCA-based methods have been extensively investigated, and it have showed excellent performance. Within this setting, we propose KPCA-based Mahalanobis kernel as a new outlier detection method using Mahalanobis distance to implicitly calculate the mapping of the data points in the feature space so that we can separate outlier points from normal pattern of data distribution. The use of KPCA-based Mahalanobis kernel on real-word data obtained from Intel Berkeley are reported showing that the proposed method performs better in finding outliers in WSNs when compared with the original reconstruction error-based variant and the one-class support vector machine detection approach. All computation are done in the original space, thus saving computing time using Mahalanobis kernel.

A novel hybrid intrusion detection method integrating anomaly detection with misuse detection

In this paper, a new hybrid intrusion detection method that hierarchically integrates a misuse detection model and an anomaly detection model in a decomposition structure is proposed. First, a misuse detection model is built based on the C4.5 decision tree algorithm and then the normal training data is decomposed into smaller subsets using the model. Next, multiple one-class SVM models are created for the decomposed subsets. As a result, each anomaly detection model does not only use the known attack information indirectly, but also builds the profiles of normal behavior very precisely. The proposed hybrid intrusion detection method was evaluated by conducting experiments with the NSL-KDD data set, which is a modified version of well-known KDD Cup 99 data set. The experimental results demonstrate that the proposed method is better than the conventional methods in terms of the detection rate for both unknown and known attacks while it maintains a low false positive rate. In addition, the proposed method significantly reduces the high time complexity of the training and testing processes. Experimentally, the training and testing time of the anomaly detection model is shown to be only 50% and 60%, respectively, of the time required for the conventional models.

ASSESSING UNCERTAINTIES IN ACCURACY OF LANDUSE CLASSIFICATION USING REMOTE SENSING IMAGES

Abstract. Multispectral remote sensing images are widely used for landuse/landcover (LULC) classification. Performance of such classification practices is normally evaluated through the confusion matrix which summarizes the producer’s and user’s accuracies and the overall accuracy. However, the confusion matrix is based on the classification results of a set of multi-class training data. As a result, the classification accuracies are heavily dependent on the representativeness of the training data set and it is imperative for practitioners to assess the uncertainties of LULC classification in order for a full understanding of the classification results. In addition, the Gaussian-based maximum likelihood classifier (GMLC) is widely applied in many practices of LULC classification. The GMLC assumes the classification features jointly form a multivariate normal distribution, whereas as, in reality, many features of individual landcover classes have been found to be non-Gaussian. Direct application of GMLC will certainly affect the classification results. In a pilot study conducted in Taipei and its vicinity, we tackled these two problems by firstly transforming the original training data set to a corresponding data set which forms a multivariate normal distribution before conducting LULC classification using GMLC. We then applied the bootstrap resampling technique to generate a large set of multi-class resampled training data from the multivariate normal training data set. LULC classification was then implemented for each resampled training data set using the GMLC. Finally, the uncertainties of LULC classification accuracies were assessed by evaluating the means and standard deviations of the producer’s and user’s accuracies of individual LULC classes which were derived from a set of confusion matrices. Results of this study demonstrate that Gaussian-transformation of the original training data achieved better classification accuracies and the bootstrap resampling technique is a very helpful tool for assessing uncertainties of LULC classification. The uncertainties in classification accuracies were also found to be affected by the sizes of class-specific training samples.

次元削減とクラスタリングによる宇宙機テレメトリ監視法

Development of intelligent system monitoring and fault detection techniques for spacecraft is of a great interest in the space engineering. In this paper, we propose a “data-driven” anomaly detection framework for spacecraft telemetry data using dimensionality reduction and clustering techniques. In this framework, we first apply dimensionality reduction or/and clustering algorithms to a normal training data set, so that we obtain statistical models representing the normal behavior of spacecraft. After the training, we monitor test data sets and detect anomaly if any, by using the obtained models. This framework is so comprehensive that a variety of clustering, dimensionality reduction and their hybrid algorithms can be used with it. In the experiment, we tested several algorithms on the past artificial satellite data, and found that a hybrid method called VQPCA is more suitable for modeling high-dimensional and multi-modal telemetry than others.

Intrusion Detection using Continuous Time Bayesian Networks

Intrusion detection systems (IDSs) fall into two high-level categories: network-based systems (NIDS) that monitor network behaviors, and host-based systems (HIDS) that monitor system calls. In this work, we present a general technique for both systems. We use anomaly detection, which identifies patterns not conforming to a historic norm. In both types of systems, the rates of change vary dramatically over time (due to burstiness) and over components (due to service difference). To efficiently model such systems, we use continuous time Bayesian networks (CTBNs) and avoid specifying a fixed update interval common to discrete-time models. We build generative models from the normal training data, and abnormal behaviors are flagged based on their likelihood under this norm. For NIDS, we construct a hierarchical CTBN model for the network packet traces and use Rao-Blackwellized particle filtering to learn the parameters. We illustrate the power of our method through experiments on detecting real worms and identifying hosts on two publicly available network traces, the MAWI dataset and the LBNL dataset. For HIDS, we develop a novel learning method to deal with the finite resolution of system log file time stamps, without losing the benefits of our continuous time model. We demonstrate the method by detecting intrusions in the DARPA 1998 BSM dataset.

A Network Anomaly Detection Method Based on Transduction Scheme

Network anomaly detection has been an active and difficult research topic in the field of intrusion detection for many years. Up to now, high false alarm rate, requirement of high quality data for modeling the normal patterns and the deterioration of detection rate because of some noisy data in the training set still make it not perform as well as expected in practice. This paper presents a novel network anomaly detection method based on improved TCM-KNN (transductive confidence machines for K-nearest neighbors) machine learning algorithm, which can effectively detect anomalies using normal data for training. A series of experiments on well known KDD Cup 1999 dataset demonstrate that it has lower false positive rate, especially higher confidence under the condition of ensuring high detection rate than the traditional anomaly detection methods. In addition, even provided with training dataset contaminated by noisy data, the proposed method still holds good detection performance. Furthermore, it can be optimized without obvious loss of detection performance by adopting small dataset for training and employing feature selection aiming at avoiding the curse of dimensionality.

Host-based intrusion detection using dynamic and static behavioral models

Intrusion detection has emerged as an important approach to network security. In this paper, we adopt an anomaly detection approach by detecting possible intrusions based on program or user profiles built from normal usage data. In particular, program profiles based on Unix system calls and user profiles based on Unix shell commands are modeled using two different types of behavioral models for data mining. The dynamic modeling approach is based on hidden Markov models (HMM) and the principle of maximum likelihood, while the static modeling approach is based on event occurrence frequency distributions and the principle of minimum cross entropy. The novelty detection approach is adopted to estimate the model parameters using normal training data only, as opposed to the classification approach which has to use both normal and intrusion data for training. To determine whether or not a certain behavior is similar enough to the normal model and hence should be classified as normal, we use a scheme that can be justified from the perspective of hypothesis testing. Our experimental results show that the dynamic modeling approach is better than the static modeling approach for the system call datasets, while the dynamic modeling approach is worse for the shell command datasets. Moreover, the static modeling approach is similar in performance to instance-based learning reported previously by others for the same shell command database but with much higher computational and storage requirements than our method.