Novelty Detection Model Research Articles

Scalable solution for agricultural soil organic carbon measurements using laser-induced breakdown spectroscopy

Effective verification of soil organic carbon (SOC) improvement interventions through soil carbon sequestration (SCS) requires robust methodologies to measure, report, and verify changes in soil carbon (C) levels. Furthermore, soil C must be monitored over time to ensure that sequestered C is not being re-emitted, thus ensuring the permanence of C removals. The traditional methods for soil C measurement are time-consuming, labor-intensive, and energy-intensive, increasing analysis costs. In this article, we verify the use of a commercially available laser-induced breakdown spectroscopy analyzer, the LaserAg-Quantum, coupled with the recursive feature addition, the gradient-boosted decision trees regression model, and the novelty detection model to predict C in soils. The developed method shows promising performance with an average limit of quantification of 0.75% of C and a precision of 4.10%. Accuracy metrics, including R2, mean absolute error, and root mean square error, yielded values of 0.81, 0.27%, and 0.37% for the validation dataset. Additionally, around 10% of validation samples after the novelty detection model exhibited relative error greater than 30%. Finally, our findings demonstrate the potential of the LaserAg-Quantum process to support measuring SOC in agricultural soils on a large scale.

Mitigation of unmeasured environmental and operational variability in long-term bridge health monitoring by unsupervised statistical learning

Environmental and operational variations present a significant challenge in the long-term health monitoring of bridge structures due to their potential to cause serious confounding influences and wrong decisions. These influences may lead to false alarms of damage during normal operation of a civil structure or, conversely, mask actual damage, resulting in a failure to correctly signal a warning. Consequently, a critical task in structural health monitoring, particularly for bridges, is to mitigate the negative impact of environmental variability. An initial approach is to measure the most influential environmental and operational factors and then apply statistical tools to eliminate their effects. However, this approach may not be effective under all conditions. Therefore, the primary objective of this paper is to propose an innovative unsupervised statistical learning method that addresses the challenges posed by environmental and operational variability by considering unmeasured factors. The proposed method involves three steps: data clustering, unsupervised feature selection, and novelty detection. The initial step utilizes k-means clustering to categorize features (modal frequencies) into five types of clusters, reflecting the influence of five environmental and operational factors on bridge structures; that is, temperature, humidity/rain/snow, wind speed/direction, traffic, and damage. These clustered features are then processed through an unsupervised feature selection algorithm based on reconstruction independent component analysis to extract new features. These reduced features are subsequently employed in a novelty detection model, developed using the Mahalanobis-squared distance, to analyze the environmental and operational effects. Real dynamic data of a steel arch bridge is utilized to verify the proposed method. Results demonstrate that this method can effectively handle the demanding issue stemming from unmeasured environmental and operational variability conditions.

Distance-based novelty detection model for identifying individuals at risk of developing Alzheimer's disease.

Novelty detection (ND, also known as one-class classification) is a machine learning technique used to identify patterns that are typical of the majority class and can discriminate deviations as novelties. In the context of Alzheimer's disease (AD), ND could be employed to detect abnormal or atypical behavior that may indicate early signs of cognitive decline or the presence of the disease. To date, few research studies have used ND to discriminate the risk of developing AD and mild cognitive impairment (MCI) from healthy controls (HC). In this work, two distinct cohorts with highly heterogeneous data, derived from the Australian Imaging Biomarkers and Lifestyle (AIBL) Flagship Study of Ageing project and the Fujian Medical University Union Hospital (FMUUH) China, were employed. An innovative framework with built-in easily interpretable ND models constructed solely on HC data was introduced along with proposing a strategy of distance to boundary (DtB) to detect MCI and AD. Subsequently, a web-based graphical user interface (GUI) that incorporates the proposed framework was developed for non-technical stakeholders. Our experimental results indicate that the best overall performance of detecting AD individuals in AIBL and FMUUH datasets was obtained by using the Mixture of Gaussian-based ND algorithm applied to single modality, with an AUC of 0.8757 and 0.9443, a sensitivity of 96.79% and 89.09%, and a specificity of 89.63% and 90.92%, respectively. The GUI offers an interactive platform to aid stakeholders in making diagnoses of MCI and AD, enabling streamlined decision-making processes. More importantly, the proposed DtB strategy could visually and quantitatively identify individuals at risk of developing AD.

Unsupervised machine learning model for predicting anomalies in subsurface safety valves and application in offshore wells during oil production

Predicting oil well behavior regarding the integrity of its equipment during production and anticipating behavioral changes and anomalies are among the main challenges in oil production. In this context, this study focuses on the development of predictive models for real-time monitoring of well behavior using sensor data from production wells. An unsupervised Novelty and Outlier Detection model has been introduced with a specific focus on predicting instances of unexpected subsurface safety valve closures in subsea wells. This model effectively classifies anomalies observed in these systems by leveraging real-world pressure and temperature data sourced from published literature. The methodology involves the implementation of a floating window for assembling training and test sets. Additionally, a comprehensive investigation is conducted into the impact of hyperparameters and the model’s threshold value (cp threshold). The results highlight the effectiveness of the developed model, observed through the accuracy achieved around 99.9% in predicting spurious closure events of the Downhole Safety Valve. On the same dataset, previous works reported 99.9% accuracy by using long short-term memory (LSTM) autoencoder, 87.1% by using random forest, and 60% with the Decision Tree method. Looking at F1-SCORE values, the developed model performs the best, followed by the LSTM model, both of which are significantly superior to the Decision Tree and random forest models. Furthermore, the model’s applicability is validated through testing in ultradeep water subsea wells within the pre-salt area of the Santos Basin. The significance lies in the potential for this research to enhance anomaly prediction in offshore wells, consequently reducing the costly interventions due to equipment malfunctions. Timely detection and corrective actions, facilitated by the model, can mitigate production loss and safeguard well integrity, addressing critical concerns in the oil and gas industry.

Uncover: Toward Interpretable Models for Detecting New Star Cluster Members.

In this design study, we present Uncover, an interactive tool aimed at astronomers to find previously unidentified member stars in stellar clusters. We contribute data and task abstraction in the domain of astronomy and provide an approach for the non-trivial challenge of finding a suitable hyper-parameter set for highly flexible novelty detection models. We achieve this by substituting the tedious manual trial and error process, which usually results in finding a small subset of passable models with a five-step workflow approach. We utilize ranges of a priori defined, interpretable summary statistics models have to adhere to. Our goal is to enable astronomers to use their domain expertise to quantify model goodness effectively. We attempt to change the current culture of blindly accepting a machine learning model to one where astronomers build and modify a model based on their expertise. We evaluate the tools' usability and usefulness in a series of interviews with domain experts.

Anomaly Detection for Hydraulic Power Units—A Case Study

This article aims to present the real-world implementation of an anomaly detection system of a hydraulic power unit. Implementation involved the Internet of Things approach. A detailed description of the system architecture is provided. The complete path from sensors through PLC and the edge computer to the cloud is presented. Some technical information about hydraulic power units is also given. This article involves the description of several model-at-scale deployment techniques. In addition, the approach to the synthesis of anomaly and novelty detection models was described. Anomaly detection of data acquired from the hydraulic power unit was carried out using two approaches, statistical and black-box, involving the One Class SVM model. The costs of cloud resources and services that were generated in the project are presented. Since the article describes a commercial implementation, the results have been presented as far as the formal and business conditions allow.

Modelling novelty detection in the thalamocortical loop.

In complex natural environments, sensory systems are constantly exposed to a large stream of inputs. Novel or rare stimuli, which are often associated with behaviorally important events, are typically processed differently than the steady sensory background, which has less relevance. Neural signatures of such differential processing, commonly referred to as novelty detection, have been identified on the level of EEG recordings as mismatch negativity (MMN) and on the level of single neurons as stimulus-specific adaptation (SSA). Here, we propose a multi-scale recurrent network with synaptic depression to explain how novelty detection can arise in the whisker-related part of the somatosensory thalamocortical loop. The "minimalistic" architecture and dynamics of the model presume that neurons in cortical layer 6 adapt, via synaptic depression, specifically to a frequently presented stimulus, resulting in reduced population activity in the corresponding cortical column when compared with the population activity evoked by a rare stimulus. This difference in population activity is then projected from the cortex to the thalamus and amplified through the interaction between neurons of the primary and reticular nuclei of the thalamus, resulting in rhythmic oscillations. These differentially activated thalamic oscillations are forwarded to cortical layer 4 as a late secondary response that is specific to rare stimuli that violate a particular stimulus pattern. Model results show a strong analogy between this late single neuron activity and EEG-based mismatch negativity in terms of their common sensitivity to presentation context and timescales of response latency, as observed experimentally. Our results indicate that adaptation in L6 can establish the thalamocortical dynamics that produce signatures of SSA and MMN and suggest a mechanistic model of novelty detection that could generalize to other sensory modalities.

Asymmetric HMMs for Online Ball-Bearing Health Assessments

The degradation of critical components inside large industrial assets, such as ball-bearings, has a negative impact on production facilities, reducing the availability of assets due to an unexpectedly high failure rate. Machine learning-based monitoring systems can estimate the remaining useful life (RUL) of ball bearings, reducing the downtime by early failure detection. However, traditional approaches for predictive systems require run-to-failure (RTF) data as training data, which in real scenarios can be scarce and expensive to obtain as the expected useful life could be measured in years. Therefore, to overcome the need of RTF, we propose a new methodology based on online novelty detection and asymmetrical hidden Markov models (As-HMMs) to work out the health assessment. This new methodology does not require previous RTF data and can adapt to natural degradation of mechanical components over time in data-stream and online environments. As the system is designed to work online within the electrical cabinet of machines, it has to be deployed using embedded electronics. Therefore, a performance analysis of As-HMM is presented to detect the strengths and critical points of the algorithm. To validate our approach, we use real life ball-bearing data sets and compare our methodology with other methodologies where no RTF data are needed and check the advantages in RUL prediction and health monitoring. As a result, we showcase a complete end-to-end solution from the sensor to actionable insights regarding RUL estimation toward maintenance application in real industrial environments.

Clinical application of the "sellar barrier's concept" for predicting intraoperative CSF leak in endoscopic endonasal surgery for pituitary adenomas with a machine learning analysis.

BackgroundRecently, it was defined that the sellar barrier entity could be identified as a predictor of cerebrospinal fluid (CSF) intraoperative leakage. The aim of this study is to validate the application of the sellar barrier concept for predicting intraoperative CSF leak in endoscopic endonasal surgery for pituitary adenomas with a machine learning approach.MethodsWe conducted a prospective cohort study, from June 2019 to September 2020: data from 155 patients with pituitary subdiaphragmatic adenoma operated through endoscopic approach at the Division of Neurosurgery, Università degli Studi di Napoli “Federico II,” were included. Preoperative magnetic resonance images (MRI) and intraoperative findings were analyzed. After processing patient data, the experiment was conducted as a novelty detection problem, splitting outliers (i.e., patients with intraoperative fistula, n = 11/155) and inliers into separate datasets, the latter further separated into training (n = 115/144) and inlier test (n = 29/144) datasets. The machine learning analysis was performed using different novelty detection algorithms [isolation forest, local outlier factor, one-class support vector machine (oSVM)], whose performance was assessed separately and as an ensemble on the inlier and outlier test sets.ResultsAccording to the type of sellar barrier, patients were classified into two groups, i.e., strong and weak barrier; a third category of mixed barrier was defined when a case was neither weak nor strong. Significant differences between the three datasets were found for Knosp classification score (p = 0.0015), MRI barrier: strong (p = 1.405 × 10−6), MRI barrier: weak (p = 4.487 × 10−8), intraoperative barrier: strong (p = 2.788 × 10−7), and intraoperative barrier: weak (p = 2.191 × 10−10). We recorded 11 cases of intraoperative leakage that occurred in the majority of patients presenting a weak sellar barrier (p = 4.487 × 10−8) at preoperative MRI. Accuracy, sensitivity, and specificity for outlier detection were 0.70, 0.64, and 0.72 for IF; 0.85, 0.45, and 1.00 for LOF; 0.83, 0.64, and 0.90 for oSVM; and 0.83, 0.55, and 0.93 for the ensemble, respectively.ConclusionsThere is a true correlation between the type of sellar barrier at MRI and its in vivo features as observed during endoscopic endonasal surgery. The novelty detection models highlighted differences between patients who developed an intraoperative CSF leak and those who did not.

A practical and synchronized data acquisition network architecture for industrial robot predictive maintenance in manufacturing assembly lines

This manuscript presents a methodology and a practical implementation of a network architecture for industrial robot data acquisition and predictive maintenance . We propose a non-intrusive and scalable robot signal extraction architecture, easily applicable in real manufacturing assembly lines. The novelty of the paper lies in the fact that it is the first proposal of a network architecture which is specially designed to address the predictive maintenance of industrial robots in real production environments. All the infrastructure needed for the implementation of the architecture is comprised of traditional well-known industrial assets. We synchronize the data acquisition with the execution of robot routines using common Programmable Logic Controllers (PLC) to obtain comparable data batches. A network architecture that acquires comparable and structured data over time, is a crucial step to advance towards an effective predictive maintenance of these complex systems, in terms of effectively detecting time dependent degradation. We implement the architecture in a real automotive manufacturing assembly line and show the potential of the solution to detect robot joint failures in real world scenarios. The architecture is therefore specially interesting for industrial practitioners and maintenance personnel. Finally, we test the feasibility of using one-class novelty detection models for robot health status degradation assessment using data of a real robot failure. To the best of our knowledge, this is the first contribution that uses robot torque signals of a real production line failure to train one-class models. • A non-intrusive and scalable robot data acquisition architecture is proposed for real manufacturing assembly lines. • The network acquires synchronized comparable robot operational data over time. • The architecture is able to detect a significant deviation in a faulty ABB IRB 6400r robot data. • One-class novelty detection models detect deviations in real operational conditions.

Effective strategies to attract crowdfunding investment based on the novelty of business ideas

Whether the novelty of an idea is a factor that directly influences crowdfunding success remains an area of ambiguity. We hypothesize that target funder diversification is effective with incremental ideas. However, focused business proposals are better suited to assert radical ideas. We also hypothesize the impact of two different strategic actions that founders can take during fundraising campaigns, agile information update and communication, on crowdfunding success. A deep-learning-based novelty detection model combined with statistical analysis is used to empirically test 7406 crowdfunding projects crawled from online platform. Our results support our hypotheses and reveal that information updates from startup founders show non-linear quadratic relationships with fundraising performance, whereas two-sided communication helps stimulate investors. We also revealed that novelty level can influence strategic choice, indicating that a project with a higher novelty should have a focused target. Our finding suggests a solution to the conflicting conclusions in previous studies on the direct impact of novelty level and target diversification, by explaining the process of novelty-dependent behavioral strategies based on signaling theory.

Fusion of Novelty Detectors Using Deep and Local Invariant Visual Features for Inspection Task

In this study, a novel framework using multiple novelty detection filters is developed to learn a model of the normality of a robot’s visual perception, which is called multichannel novelty detection. Subsequently, the acquired model was used to highlight dissimilar perceptions when the robot explored an environment. The main purpose of fusing multiple novelty filters is that each novelty filter performs well in detecting specific types of novelties; therefore, a new framework is proposed that demonstrates a new way to combine multiple different purposed novelty detection filters together in order to yield an overall more robust novelty status on the visual features. To develop a multichannel novelty detection system, expectation- and appearance-based novelty detection models were used in this study. To become experts in detecting different types of novelty using these models, different features from the input image were extracted as inputs for the models. The expectation-based novelty detection model uses the MobileNetV2 deep network to extract the deep features of the input image, which is subsequently used to learn a sequential and temporal model of normality to detect novelty. By contrast, the appearance-based novelty detection model uses speeded up robust features (SURF), which provide more region-focused features within the input image, to identify whether a specific region of the image is novel. The proposed multichannel novelty detection system is a completely online and real-time approach that is very important for mobile robotics applications. The proposed framework was tested in three novel environments, and it was reported that the proposed multichannel novelty detection system performs better than expectation-based and appearance-based novelty filters separately. Statistically, in the three novel environments, the Matthews correlation coefficients are reported to be 0.94, 0.97, and 0.93, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> scores are reported to be 0.95, 0.97, and 0.93, respectively, which proves that and can be concluded as almost perfect statistically.

Adversarially Robust One-Class Novelty Detection.

One-class novelty detectors are trained with examples of a particular class and are tasked with identifying whether a query example belongs to the same known class. Most recent advances adopt a deep auto-encoder style architecture to compute novelty scores for detecting novel class data. Deep networks have shown to be vulnerable to adversarial attacks, yet little focus is devoted to studying the adversarial robustness of deep novelty detectors. In this article, we first show that existing novelty detectors are susceptible to adversarial examples. We further demonstrate that commonly-used defense approaches for classification tasks have limited effectiveness in one-class novelty detection. Hence, we need a defense specifically designed for novelty detection. To this end, we propose a defense strategy that manipulates the latent space of novelty detectors to improve the robustness against adversarial examples. The proposed method, referred to as Principal Latent Space (PrincipaLS), learns the incrementally-trained cascade principal components in the latent space to robustify novelty detectors. PrincipaLS can purify latent space against adversarial examples and constrain latent space to exclusively model the known class distribution. We conduct extensive experiments on eight attacks, five datasets and seven novelty detectors, showing that PrincipaLS consistently enhances the adversarial robustness of novelty detection models.

Comparison of Novelty Detection Methods for Detection of Various Rotary Machinery Faults.

Condition monitoring is an indispensable element related to the operation of rotating machinery. In this article, the monitoring system for the parallel gearbox was proposed. The novelty detection approach is used to develop the condition assessment support system, which requires data collection for a healthy structure. The measured signals were processed to extract quantitative indicators sensitive to the type of damage occurring in this type of structure. The indicator’s values were used for the development of four different novelty detection algorithms. Presented novelty detection models operate on three principles: feature space distance, probability distribution, and input reconstruction. One of the distance-based models is adaptive, adjusting to new data flowing in the form of a stream. The authors test the developed algorithms on experimental and simulation data with a similar distribution, using the training set consisting mainly of samples generated by the simulator. Presented in the article results demonstrate the effectiveness of the trained models on both data sets.

Towards a data-driven adaptive anomaly detection system for human activity

• An adaptive system is proposed for abnormality detection in human activities. • The data-driven system adapts to changes in human behavioural routine. • The approach is based on data ageing and data dissimilarity forgetting factors. • The forgetting factor features allow the system to discard old behavioural patterns. Research in the field of ambient intelligence allows for the utilisation of different computational models for human activity recognition and abnormality detection to promote independent living and to improve the quality of life for the increasing ageing population. The existing monitoring systems are not adaptive to the overly changing human behavioural routine leading to a high rate of false predictions. An adaptive system pipeline is proposed in this paper for adapting to changes in human behaviour based on data ageing and data dissimilarity forgetting factors. The forgetting factor feature allows adaptation of the model to the current routines of an individual while forgetting outdated behavioural patterns. The data ageing forgetting factor discard old behavioural routine based on the age of the activity data while in the data dissimilarity approach, this is achieved by measuring the similarity of the activity data. Behaviour modelling is achieved using an ensemble of novelty detection models termed as Consensus Novelty Detection Ensemble consisting of One-Class Support Vector Machine, Local Outlier Factor, Robust Covariance Estimation and Isolation Forest. The proposed approach is data-driven and environment-invariant, making it feasible for deployment in heterogeneous environments. A comparative analysis carried out with other abnormality detection models for human activities across two datasets shows that the proposed approach achieved better results.

Ensemble-learning based neural networks for novelty detection in multi-class systems

In most real-world systems or processes, determining the complete set of classes during the training phase is generally impossible. There is a high chance that novelties or abnormal data can appear in future phases which might severely affect the performance of the machine learning system. Novelty detection is of great importance in many critical systems and domains, such as business intelligence, process monitoring, information security, clinical decision support etc. Most of the available methods for novelty detection use a one-class classification (OCC) criterion, i.e. treating multiple known classes as a single ”Normal” class, whose aim is to distinguish data samples between “Normal” and “Not Normal” classes. In this paper, the problem of novelty detection in multi-class systems is addressed through ensemble based learning of neural networks (EBNN), capable of both detecting novelties and classifying the known normal samples in future datasets. Moreover, the model is analogous to the semi-supervised learning system as it is trained using only the available normal classes. Evaluation of the proposed model (EBNN) on UCI machine learning datasets showed that the model not only outperforms other models in detecting novelties but also has a better multi-class classification accuracy for known normal classes. The proposed model implements a novel activation function in its framework and differs from the commonly available novelty detection models in three aspects. First, the model is much simpler to implement and does not need any initial assumptions about the model. Second, the model does not require any novel or abnormal data during training phase (semi-supervised learning). Third, it can be used as a two in one system to detect novelties and at the same time to classify data based on known classes.

Fault Detection and Identification Methodology Under an Incremental Learning Framework Applied to Industrial Machinery

An industrial machinery condition monitoring methodology based on ensemble novelty detection and evolving classification is proposed in this study. The methodology contributes to solve current challenges dealing with classical electromechanical system monitoring approaches applied in industrial frameworks, that is, the presence of unknown events, the limitation to the nominal healthy condition as starting knowledge, and the incorporation of new patterns to the available knowledge. The proposed methodology is divided into four main stages: 1) a dedicated feature calculation and reduction over available physical magnitudes to increase novelty detection and fault classification capabilities; 2) a novelty detection based on the ensemble of one-class support vector machines to identify not previously considered events; 3) a diagnosis by means of eClass evolving classifiers for patterns recognition; and 4) re-training to include new patterns to the novelty detection and fault identification models. The effectiveness of the proposed fault detection and identification methodology has been compared with classical approaches, and verified by experimental results obtained from an automotive end-of-line test machine.

LGND: a new method for multi-class novelty detection

Multi-class novelty detection is a crucial yet challenging aspect for recognition systems. Several methods have been presented, which either concatenate multiple classes into a large artificial super-class, or combine several independent classifiers of each known class, or utilize the results of multi-class classifiers. However, these methods ignore the correlation within each class, or cannot be elegantly formulated in a joint model. To overcome these limitations, we propose a new local and global novelty detection model (LGND). Different from the previous approaches, LGND incorporates the local correlation with the global regularization in a unified framework. This new optimization model boils down to a convex quadratic programming with guaranteed global optimum solution. Furthermore, comprehensive discussions, including the relationship between locality and globality, the discussion on the parameters in LGND and the connections to multi-class classification, are also presented. LGND opens up a new way for multi-class novelty detection from both local and global perspectives. Experimental results on Corel5k, Caltech-256, and five UCI data sets confirm that LGND outperforms or is at least comparable to state-of-the-art methods.

Network-based approach to detect novelty of scholarly literature

We present a method to detect the novelty of a research paper. Because novelty in scholarly literature also examines the larger research community, a network-based approach for extracting features is proposed. Two graphs are introduced, a macro-level graph, where authors and documents are used as nodes, and a micro-level graph, where keywords, topics, and words are used as nodes. After constructing the seed graph, papers are incrementally added while changes in the graph are recorded as the feature set of a paper. An autoencoder neural network is then used as the novelty detection model. The experimental results show that the commonly used text feature representations, TF-IDF and one-class SVM, are not suitable for detecting the novelty of a research paper. Among the constructed graphs, keyword-level graph features exhibit the best performance using regression analysis as the metric. We also combine the macro-level graph, micro-level graph, and all features and find that the combination of keywords, topics, and word features perform the best using regression and citation count analysis. Other factors that could affect the citation counts, impact, and audience, are also discussed.

Advanced detection of HVAC faults using unsupervised SVM novelty detection and Gaussian process models

A novel fault detection algorithm based on machine learning is introduced in this paper, that is applied to the detection of faults in heater, ventilation and air conditioning (HVAC) systems. The algorithm is based on the use of a set of nonlinear regressors intended to estimate the response of the HVAC to the external variables. The regression algorithm is the well known Gaussian process regression, which, through a Gaussian modeling of the parameter priors and the conditional likelihood of the observations, is able to produce a probabilistic model of the prediction. We use the prediction error and its estimated variance as an input to a support vector machine novelty detector that, in an unsupervised way, is able to detect the faults of the HVAC. This algorithm improves the standard novelty detection, as it can be seen in the experiments.