Unsupervised Novelty Detection Research Articles

Unsupervised machine learning model for predicting anomalies in subsurface safety valves and application in offshore wells during oil production

Predicting oil well behavior regarding the integrity of its equipment during production and anticipating behavioral changes and anomalies are among the main challenges in oil production. In this context, this study focuses on the development of predictive models for real-time monitoring of well behavior using sensor data from production wells. An unsupervised Novelty and Outlier Detection model has been introduced with a specific focus on predicting instances of unexpected subsurface safety valve closures in subsea wells. This model effectively classifies anomalies observed in these systems by leveraging real-world pressure and temperature data sourced from published literature. The methodology involves the implementation of a floating window for assembling training and test sets. Additionally, a comprehensive investigation is conducted into the impact of hyperparameters and the model’s threshold value (cp threshold). The results highlight the effectiveness of the developed model, observed through the accuracy achieved around 99.9% in predicting spurious closure events of the Downhole Safety Valve. On the same dataset, previous works reported 99.9% accuracy by using long short-term memory (LSTM) autoencoder, 87.1% by using random forest, and 60% with the Decision Tree method. Looking at F1-SCORE values, the developed model performs the best, followed by the LSTM model, both of which are significantly superior to the Decision Tree and random forest models. Furthermore, the model’s applicability is validated through testing in ultradeep water subsea wells within the pre-salt area of the Santos Basin. The significance lies in the potential for this research to enhance anomaly prediction in offshore wells, consequently reducing the costly interventions due to equipment malfunctions. Timely detection and corrective actions, facilitated by the model, can mitigate production loss and safeguard well integrity, addressing critical concerns in the oil and gas industry.

Unsupervised machine and deep learning methods for structural damage detection: A comparative study

AbstractWhile many structural damage detection methods have been developed in recent decades, few data‐driven methods in unsupervised learning mode have been developed to solve the practical difficulties in data acquisition for civil infrastructures in different scenarios. To address such a challenge, this article proposes a number of improved unsupervised novelty detection methods and conducts extensive comparative studies on a laboratory scale steel bridge to examine their performances of damage detection. The key concept behind unsupervised novelty detection in this article is that only normal data from undamaged/baseline structural scenarios are required to train statistical models with these methods. Then, these trained models are used to identify abnormal testing data from damaged scenarios. To detect structural damage in the form of loosening bolts in the steel bridge, four machine‐learning methods (i.e., K‐nearest neighbors method, Gaussian mixture models, one‐class support vector machines, density peaks‐based fast clustering method) and one deep learning method using a deep auto‐encoder are selected. Meanwhile, some modifications and improvements are made to enable these methods to detect structural damage in unsupervised novelty detection mode. In their comparative studies, the advantages and disadvantages of these methods are analyzed based on their results of structural damage detection.

A novelty detection approach to effectively predict conversion from mild cognitive impairment to Alzheimer’s disease

Accurately recognising patients with progressive mild cognitive impairment (pMCI) who will develop Alzheimer’s disease (AD) in subsequent years is very important, as early identification of those patients will enable interventions to potentially reduce the number of those transitioning from MCI to AD. Most studies in this area have concentrated on high-dimensional neuroimaging data with supervised binary/multi-class classification algorithms. However, neuroimaging data is more costly to obtain than non-imaging, and healthcare datasets are normally imbalanced which may reduce classification performance and reliability. To address these challenges, we proposed a new strategy that employs unsupervised novelty detection (ND) techniques to predict pMCI from the AD neuroimaging initiative non-imaging data. ND algorithms, including the k-nearest neighbours (kNN), k-means, Gaussian mixture model (GMM), isolation forest (IF) and extreme learning machine (ELM), were employed and compared with supervised binary support vector machine (SVM) and random forest (RF). We introduced optimisation with nested cross-validation and focused on maximising the adjusted F measure to ensure maximum generalisation of the proposed system by minimising false negative rates. Our extensive experimental results show that ND algorithms (0.727 ± 0.029 kNN, 0.7179 ± 0.0523 GMM, 0.7276 ± 0.0281 ELM) obtained comparable performance to supervised binary SVM (0.7359 ± 0.0451) with 20% stable MCI misclassification tolerance and were significantly better than RF (0.4771 ± 0.0167). Moreover, we found that the non-invasive, readily obtainable, and cost-effective cognitive and functional assessment was the most efficient predictor for predicting the pMCI within 2 years with ND techniques. Importantly, we presented an accessible and cost-effective approach to pMCI prediction, which does not require labelled data.

Autoencoder and Mahalanobis distance for novelty detection in structural health monitoring data of an offshore wind turbine.

Structural Health Monitoring (SHM) has seen an explosion in data gathering in the last few years. This is illustrated in the offshore wind industry through an increase in the amount of placed offshore wind turbines (OWT), a higher rate of SHM instrumented OWTs and an increase in the sampling rate. The growing data gathering has led to the interest of big data techniques in the SHM industry. This paper introduces a new more robust unsupervised novelty detection pipeline combining an autoencoder and the Mahalanobis distance and comparing this combination to both methods separately. This reduces the high dimensional input to a one dimensional novelty index for detecting anomalies in the OWT. Additionally the research considers the challenges that the downtime of non operationally essential sensors poses, a method is introduced to guarantee high model availability without losing the benefit of high fidelity.

Structural health monitoring by a novel probabilistic machine learning method based on extreme value theory and mixture quantile modeling

This article proposes a novel probabilistic machine learning method based on unsupervised novelty detection for health monitoring of civil structures. The core of this method is based on extreme value theory (EVT) and mixture quantile modeling. Accordingly, a mixture quantile value by combining non-parametric and parametric quantile estimators is proposed as a new decision-making or novelty score. The non-parametric estimator relies on an empirical quantile function, while the parametric estimator stems from modeling a generalized extreme value distribution. Generally, the proposed method is composed of some main steps; that is, calculating distances between feature samples, sorting the distances in ascending order, changing their signs for providing negated quantities, selecting some negated maximum distances or extreme values in an iterative algorithm by using a goodness-of-fit test based on Kullback-Leibler information, and computing a mixture quantile. Furthermore, an EVT-based approach is proposed to estimate an alarming threshold. The major innovation in this article is to develop a novel probabilistic novelty detector with a new score for decision-making. The advantages of the proposed method contain preparing discriminative novelty scores for damage detection, dealing with the major challenge of environmental and/or operational variability, estimating a reliable threshold, and implementing both the procedures of decision-making and threshold estimation within a single framework. Dynamic and statistical features of two full-scale bridge structures are utilized to validate the proposed method along with comparative studies. Results demonstrate that the method is a reliable and influential tool for health monitoring of civil structures under varying various environmental and/or operational conditions.

An Unsupervised Tunnel Damage Identification Method Based on Convolutional Variational Auto-Encoder and Wavelet Packet Analysis.

Finding a low-cost and highly efficient method for identifying subway tunnel damage can greatly reduce catastrophic accidents. At present, tunnel health monitoring is mainly based on the observation of apparent diseases and vibration monitoring, which is combined with a manual inspection to perceive the tunnel health status. However, these methods have disadvantages such as high cost, short working time, and low identification efficiency. Thus, in this study, a tunnel damage identification algorithm based on the vibration response of in-service train and WPE-CVAE is proposed, which can automatically identify tunnel damage and give the damage location. The method is an unsupervised novelty detection that requires only sufficient normal data on healthy structure for training. This study introduces the theory and implementation process of this method in detail. Through laboratory model tests, the damage of the void behind the tunnel wall is designed to verify the performance of the algorithm. In the test case, the proposed method achieves the damage identification performance with a 96.25% recall rate, 86.75% hit rate, and 91.5% accuracy. Furthermore, compared with the other unsupervised methods, the method performance and noise immunity are better than others, so it has a certain practical value.

Toward a general unsupervised novelty detection framework in structural health monitoring

AbstractThis study proposes an unsupervised, online structural health monitoring framework robust to the sensor configuration, that is, the number and placement of sensors. The proposed methodology leverages generative adversarial networks (GANs). The GAN's discriminator network is the novelty detector, while its generator provides additional data to tune the detection threshold. GAN models are trained with the fast Fourier transform of structural accelerations as input, avoiding the need for any structure‐specific feature extraction. Dense, convolutional (convolutional neural network), and long short‐term memory (LSTM) units are evaluated as discriminators under different GAN training loss patterns, that is, the differences between discriminator and generator training losses. Results show that the LSTM‐based discriminators and the suggested threshold tuning technique to be robust even in loss patterns with overfitted discriminators, a probable outcome of limited training sets. The framework is evaluated on two benchmark datasets. With only 100 s of training data, it achieved 95% novelty detection accuracy, distinguishing between different damage classes and identifying their resurgence under varying sensor configurations. Finally, the majority‐vote‐ensemble of discriminator‐generator pairs at different training epochs is introduced to reduce false alarms, improve novelty detection accuracy and stability.

Applicability of machine learning approaches for structural damage detection of offshore wind jacket structures based on low resolution data

Structural damage in offshore wind jacket support structures are relatively unlikely due to the precautions taken in design but it could imply dramatic consequences if undetected. This work explores the possibilities of damage detection when using low resolution data, which are available with lower costs compared to dedicated high-resolution structural health monitoring. Machine learning approaches showed to be generally feasible for detecting a structural damage based on SCADA data collected in a simulation environment. Focus is here given to investigate model uncertainties, to assess the applicability of machine learning approaches for reality. Two jacket models are utilised representing the as-designed and the as-installed system, respectively. Extensive semi-coupled simulations representing different operating load cases are conducted to generate a database of low-resolution signals serving the machine learning training and testing. The analysis shows the challenges of classification approaches, i.e. supervised learning aiming to separate healthy and damage status, in coping with the uncertainty in system dynamics. Contrarily, an unsupervised novelty detection approach shows promising results when trained with data from both, the as-designed and the as-installed system. The findings highlight the importance of investigating model uncertainties and careful selection of training data.

Automated diagnosis of heart valve degradation using novelty detection algorithms and machine learning.

The blood flow through the major vessels holds great diagnostic potential for the identification of cardiovascular complications and is therefore routinely assessed with current diagnostic modalities. Heart valves are subject to high hydrodynamic loads which render them prone to premature degradation. Failing native aortic valves are routinely replaced with bioprosthetic heart valves. This type of prosthesis is limited by a durability that is often less than the patient’s life expectancy. Frequent assessment of valvular function can therefore help to ensure good long-term outcomes and to plan reinterventions. In this article, we describe how unsupervised novelty detection algorithms can be used to automate the interpretation of blood flow data to improve outcomes through early detection of adverse cardiovascular events without requiring repeated check-ups in a clinical environment. The proposed method was tested in an in-vitro flow loop which allowed simulating a failing aortic valve in a laboratory setting. Aortic regurgitation of increasing severity was deliberately introduced with tube-shaped inserts, preventing complete valve closure during diastole. Blood flow recordings from a flow meter at the location of the ascending aorta were analyzed with the algorithms introduced in this article and a diagnostic index was defined that reflects the severity of valvular degradation. The results indicate that the proposed methodology offers a high sensitivity towards pathological changes of valvular function and that it is capable of automatically identifying valvular degradation. Such methods may be a step towards computer-assisted diagnostics and telemedicine that provide the clinician with novel tools to improve patient care.

Deep embeddings for novelty detection in myopathy

We address the challenge of finding anomalies in ultrasound images via deep learning, specifically applying this to screening for myopathies and finding rare presentations of myopathic disease. Among myopathic diseases, this study focuses on the use case of myositis given the spectrum of muscle involvement seen in these inflammatory muscle diseases, as well as the potential for treatment. For this study, we have developed a fully annotated dataset (called “Myositis3K”) which includes 3586 images of eighty-nine individuals (35 control and 54 with myositis) acquired with informed consent. We approach this challenge as one of performing unsupervised novelty detection (ND), and use tools leveraging deep embeddings combined with several novelty scoring methods. We evaluated these various ND algorithms and compared their performance against human clinician performance, against other methods including supervised binary classification approaches, and against unsupervised novelty detection approaches using generative methods. Our best performing approach resulted in a (ROC) AUC (and 95% CI error margin) of 0.7192 (0.0164), which is a promising baseline for developing future clinical tools for unsupervised prescreening of myopathies.

Unsupervised Novelty Detection Using Deep Autoencoders with Density Based Clustering

Novelty detection is a classification problem to identify abnormal patterns; therefore, it is an important task for applications such as fraud detection, fault diagnosis and disease detection. However, when there is no label that indicates normal and abnormal data, it will need expensive domain and professional knowledge, so an unsupervised novelty detection approach will be used. On the other hand, nowadays, using novelty detection on high dimensional data is a big challenge and previous research suggests approaches based on principal component analysis (PCA) and an autoencoder in order to reduce dimensionality. In this paper, we propose deep autoencoders with density based clustering (DAE-DBC); this approach calculates compressed data and error threshold from deep autoencoder model, sending the results to a density based cluster. Points that are not involved in any groups are not considered a novelty; the grouping points will be defined as a novelty group depending on the ratio of the points exceeding the error threshold. We have conducted the experiment by substituting components to show that the components of the proposed method together are more effective. As a result of the experiment, the DAE-DBC approach is more efficient; its area under the curve (AUC) is shown to be 13.5 percent higher than state-of-the-art algorithms and other versions of the proposed method that we have demonstrated.

Concept drift robust adaptive novelty detection for data streams

Abstract In this paper we study the performance of two original adaptive unsupervised novelty detection methods (NDMs) on data with concept drift. Newly, the concept drift is considered as a challenging data imbalance that should be ignored by the NDMs, and only system changes and outliers represent novelty. The field of application for such NDMs is broad. For example, the method can be used as a supportive method for real-time system fault detection, for onset detection of events in biomedical signals, in monitoring of nonlinearly controlled processes, for event driven automated trading, etc. The two newly studied methods are the error and learning based novelty detection (ELBND) and the learning entropy (LE) based detection. These methods use both the error and weight increments of a (supervised) learning model. Here, we study these methods with normalized least-mean squares (NLMS) adaptive filter, and while the NDMs were studied on various real life tasks, newly, we carry out the study on two types of data streams with concept drift to analyze the general ability for unsupervised novelty detection. The two data streams, one with system changes, second with outliers, represent different novelty scenarios to demonstrate the performance of the proposed NDMs with concept drifts in data. Both tested NDMs work as a feature extractor. Thus, a classification framework is used for the evaluation of the obtained features and NDM benchmarking, where two other NDMs, one based on the adaptive model plain error, second using the sample entropy (SE), are used as the reference for the comparison to the proposed methods. The results show that both newly studied NDMs are superior to the merely use of the plain error of adaptive model and also to the sample entropy based detection while they are robust against the concept drift occurrence.

Unsupervised novelty detection–based structural damage localization using a density peaks-based fast clustering algorithm

Within machine learning, several structural damage detection and localization methods based on clustering and novelty detection methods have been proposed in the recent years in order to monitor mechanical and civil structures. In order to train a machine learning model, an unsupervised mode is preferred because it only requires sufficient normal data from the intact states of a structure for training, and the testing abnormal data from various damage states are generally quite rare. With an unsupervised training mode, the capability of detecting structural damage mainly depends on the identification of abnormal data from the testing data. This identification process is termed unsupervised novelty detection. The premise of unsupervised novelty detection is that a large volume of a normal data set is available first to train a normal model that is established by machine learning algorithms. Then, the trained normal model can be used to identify abnormal data from future testing data. In this article, a new structural damage detection and localization method is proposed using a density peaks-based fast clustering algorithm. In order to realize damage detection, the original density peaks-based fast clustering algorithm is modified to an unsupervised machine learning method by adding training and testing processes. Furthermore, to improve the performance of the proposed method, the Gaussian kernel function of radius is introduced to calculate the local density of data points, and a new damage-sensitive feature using a continuous wavelet transform is also proposed. Damage-sensitive features are extracted from the measured data through sensors installed on a laboratory-scale steel structure. Extensive experimental studies are carried out under various structural damage scenarios in order to validate the performance of the proposed method. The proposed density peaks-based fast clustering method shows satisfactory performance with regard to damage localization under various damage scenarios as compared to a traditional approach.

A unifying methodology for the evaluation of neural network models on novelty detection tasks

An important issue in data analysis and pattern classification is the detection of anomalous observations and its influence on the classifier's performance. In this paper, we introduce a novel methodology to systematically compare the performance of neural network (NN) methods applied to novelty detection problems. Initially, we describe the most common NN-based novelty detection techniques. Then we generalize to the supervised case, a recently proposed unsupervised novelty detection method for computing reliable decision thresholds. We illustrate how to use the proposed methodology to evaluate the performances of supervised and unsupervised NN-based novelty detectors on a real-world benchmarking data set, assessing their sensitivity to training parameters, such as data scaling, number of neurons, training epochs and size of the training set.

Automatic Identification of Gasoline – Biofuel Blend Type in an Internal Combustion Four-stroke Engine based on Unsupervised Novelty Detection and Active Learning

In this study, unsupervised novelty detection was applied to determine automatically both the type of fuel and the blending ratio of the fuel mixture based on analysis vibrational signals during operation of a four-stroke combustion engine fueled with gasoline and gasoline blends with ethanol and methanol. Three types of fuel were utilized: unleaded gasoline as base fuel compared to the mixture of two alcohols, ethanol and methanol in admixture with gasoline, as a percentage ratio of around 10%, 20% and 30%, respectively. The engine tests were performed at 1000, 1300, 1600 and 1900rpm. The collections of measurements were conducted by a triaxial accelerometer. Features in time and frequency were calculated from the signals received. The classification of the type of fuel and fuel blend ratio was achieved using an active learning method based on incremental application of One Class Principal Component Analysis (OCPCA) for novelty detection.

On the regularization path of the support vector domain description

The internet and a growing number of increasingly sophisticated measuring devices make vast amounts of data available in many applications. However, the dimensionality is often high, and the time available for manual labelling scarce. Methods for unsupervised novelty detection are a great step towards meeting these challenges, and the support vector domain description has already shown its worth in this field. The method has recently received more attention, since it has been shown that the regularization path is piece-wise linear, and can be calculated efficiently. The presented work restates the new findings in a manner which permits the calculation with O ( n · n B ) complexity in each iteration step instead of O n 2 + n B 3 , where n is the number of data points and n B is the number of boundary points. This is achieved by updating and downdating the system matrix to avoid redundant calculations. We believe this will further promote the use of this method.

Unsupervised novelty detection using Gabor filters for defect segmentation in textures

Gabor wavelets are applied to develop an unsupervised novelty method for defect detection and segmentation that is fully automatic and free of any adjustable parameter. The algorithm combines the Gabor analysis of the sample image with a statistical analysis of the wavelet coefficients corresponding to each detail. The statistical distribution of the coefficients corresponding to the defect-free background texture is calculated from the coefficient's distribution of the sample under inspection. Once the background texture features are estimated, a threshold is automatically fixed and applied to all the details, whose information is merged into a single binary output image in which the defect appears segmented from the background. The method is applicable to random, nonperiodic, and periodic textures. Since all the information to inspect a sample is obtained from the sample itself, the method is proof against heterogeneities between different samples of the material, in-plane positioning errors, scale variations, and lack of homogeneous illumination. Experimental results are presented. Some results are compared with other unsupervised methods designed for defect segmentation in periodic textures.