Novelty Detection Algorithm Research Articles

A Classification Algorithm for Time-domain Novelties in Preparation for LSST Alerts. Application to Variable Stars and Transients Detected with DECam in the Galactic Bulge

Abstract With the advent of the Legacy Survey of Space and Time, time-domain astronomy will be faced with an unprecedented volume and rate of data. Real-time processing of variables and transients detected by such large-scale surveys is critical to identifying the more unusual events and allocating scarce follow-up resources efficiently. We develop an algorithm to identify these novel events within a given population of variable sources. We determine the distributions of magnitude changes (dm) over time intervals (dt) for a given passband f, , and use these distributions to compute the likelihood of a test source being consistent with the population or being an outlier. We demonstrate our algorithm by applying it to the DECam multiband time-series data of more than 2000 variable stars identified by Saha et al. in the Galactic Bulge that are largely dominated by long-period variables and pulsating stars. Our algorithm discovers 18 outlier sources in the sample, including a microlensing event, a dwarf nova, and two chromospherically active RS CVn stars, as well as sources in the blue horizontal branch region of the color–magnitude diagram without any known counterparts. We compare the performance of our algorithm for novelty detection with the multivariate Kernel Density Estimator and Isolation Forest on the simulated PLAsTiCC data set. We find that our algorithm yields comparable results despite its simplicity. Our method provides an efficient way for flagging the most unusual events in a real-time alert-broker system.

Diagnosis of Faulty Wind Turbine Bearings Using Tower Vibration Measurements

Condition monitoring of gear-based mechanical systems in non-stationary operation conditions is in general very challenging. This issue is particularly important for wind energy technology because most of the modern wind turbines are geared and gearbox damages account for at least the 20% of their unavailability time. In this work, a new method for the diagnosis of drive-train bearings damages is proposed: the general idea is that vibrations are measured at the tower instead of at the gearbox. This implies that measurements can be performed without impacting the wind turbine operation. The test case considered in this work is a wind farm owned by the Renvico company, featuring six wind turbines with 2 MW of rated power each. A measurement campaign has been conducted in winter 2019 and vibration measurements have been acquired at five wind turbines in the farm. The rationale for this choice is that, when the measurements have been acquired, three wind turbines were healthy, one wind turbine had recently recovered from a planetary bearing fault, and one wind turbine was undergoing a high speed shaft bearing fault. The healthy wind turbines are selected as references and the damaged and recovered are selected as targets: vibration measurements are processed through a multivariate Novelty Detection algorithm in the feature space, with the objective of distinguishing the target wind turbines with respect to the reference ones. The application of this algorithm is justified by univariate statistical tests on the selected time-domain features and by a visual inspection of the data set via Principal Component Analysis. Finally, a novelty index based on the Mahalanobis distance is used to detect the anomalous conditions at the damaged wind turbine. The main result of the study is that the statistical novelty of the damaged wind turbine data set arises clearly, and this supports that the proposed measurement and processing methods are promising for wind turbine condition monitoring.

Robust support vector data description for novelty detection with contaminated data

Support vector data description (SVDD) is a widely used novelty detection algorithm. It provides excellent predictions even in the absence of negative samples and retains the mathematical elegance of Support Vector Machines. The decision boundary can be very flexible due to the incorporation of kernel functions. However, SVDD can suffer a lot from contaminated data containing, for example, outliers or mislabeled observations. Although several weighting schemes have been proposed to find a more reliable description of the target data, the calculation of the weight are themselves affected by the outliers and does not provide much insight in the data. The masked outliers fail to receive lower weight values. The Stahel–Donoho (SD) outlyingness from multivariate statistics is a very robust measure to expose the outliers. To avoid the masking effect, we propose to assign weight to each observation based on the SD outlyingness in an arbitrary kernel space. A robust SVDD is defined down-weighting the samples with large outlyingness. The experimental results demonstrate superiority of the proposed method in terms of AUC for contaminated data.

A comparative evaluation of novelty detection algorithms for discrete sequences

The identification of anomalies in temporal data is a core component of numerous research areas such as intrusion detection, fault prevention, genomics and fraud detection. This article provides an experimental comparison of candidate methods for the novelty detection problem applied to discrete sequences. The objective of this study is to identify which state-of-the-art methods are efficient and appropriate candidates for a given use case. These recommendations rely on extensive novelty detection experiments based on a variety of public datasets in addition to novel industrial datasets. We also perform thorough scalability and memory usage tests resulting in new supplementary insights of the methods’ performance, key selection criteria to solve problems relying on large volumes of data and to meet the expectations of applications subject to strict response time constraints.

Dynamic dispatching system using a deep denoising autoencoder for semiconductor manufacturing

Deep denoising autoencoders (DDAE), which are variants of the autoencoder, have shown outstanding performance in various machine learning tasks. In this study, we propose using a DDAE to address a dispatching rule selection problem that represents a major problem in semiconductor manufacturing. Recently, the significance of dispatching systems for storage allocation has become more apparent because operational issues lead to transfer inefficiency, resulting in production losses. Further, recent approaches have overlooked the possibility of a class imbalance problem in predicting the best dispatching rule. The main purpose of this study is to examine DDAE-based predictive control of the storage dispatching systems to reduce idle machines and production losses. We conducted an experimental evaluation to compare the predictive performance of DDAE with those of five other novelty detection algorithms. Finally, we compared our adaptive approach with the optimization and existing heuristic approaches to demonstrate the effectiveness and efficiency of the proposed method. The experimental results demonstrated that the proposed method outperformed the existing methods in terms of machine utilizations and throughputs.

Automated diagnosis of heart valve degradation using novelty detection algorithms and machine learning.

The blood flow through the major vessels holds great diagnostic potential for the identification of cardiovascular complications and is therefore routinely assessed with current diagnostic modalities. Heart valves are subject to high hydrodynamic loads which render them prone to premature degradation. Failing native aortic valves are routinely replaced with bioprosthetic heart valves. This type of prosthesis is limited by a durability that is often less than the patient’s life expectancy. Frequent assessment of valvular function can therefore help to ensure good long-term outcomes and to plan reinterventions. In this article, we describe how unsupervised novelty detection algorithms can be used to automate the interpretation of blood flow data to improve outcomes through early detection of adverse cardiovascular events without requiring repeated check-ups in a clinical environment. The proposed method was tested in an in-vitro flow loop which allowed simulating a failing aortic valve in a laboratory setting. Aortic regurgitation of increasing severity was deliberately introduced with tube-shaped inserts, preventing complete valve closure during diastole. Blood flow recordings from a flow meter at the location of the ascending aorta were analyzed with the algorithms introduced in this article and a diagnostic index was defined that reflects the severity of valvular degradation. The results indicate that the proposed methodology offers a high sensitivity towards pathological changes of valvular function and that it is capable of automatically identifying valvular degradation. Such methods may be a step towards computer-assisted diagnostics and telemedicine that provide the clinician with novel tools to improve patient care.

Machine learning in industrial measurement technology for detection of known and unknown faults of equipment and sensors

Abstract This paper focuses on the application of novelty detection in combination with supervised fault classification for industrial condition monitoring. Its goal is to provide a guideline for engineers on how to apply novelty detection for outlier detection, monitoring of supervised classification and detection of unknown faults without the need of a data scientist. All guidelines are demonstrated by means of a publicly available condition monitoring dataset. In each application case the results achieved with different common novelty detection algorithms are compared, advantages and disadvantages of the respective algorithms are shown. To increase applicability of the suggested approach visualization of results is emphasized and all algorithms have been included in a publicly available data analysis software toolbox with graphical user interface.

A post-processing method for detecting unknown intent of dialogue system via pre-trained deep neural network classifier

With the maturity and popularity of dialogue systems, detecting user’s unknown intent in dialogue systems has become an important task. It is also one of the most challenging tasks since we can hardly get examples, prior knowledge or the exact numbers of unknown intents. In this paper, we propose SofterMax and deep novelty detection (SMDN), a simple yet effective post-processing method for detecting unknown intent in dialogue systems based on pre-trained deep neural network classifiers. Our method can be flexibly applied on top of any classifiers trained in deep neural networks without changing the model architecture. We calibrate the confidence of the softmax outputs to compute the calibrated confidence score (i.e., SofterMax) and use it to calculate the decision boundary for unknown intent detection. Furthermore, we feed the feature representations learned by the deep neural networks into traditional novelty detection algorithm to detect unknown intents from different perspectives. Finally, we combine the methods above to perform the joint prediction. Our method classifies examples that differ from known intents as unknown and does not require any examples or prior knowledge of it. We have conducted extensive experiments on three benchmark dialogue datasets. The results show that our method can yield significant improvements compared with the state-of-the-art baselines11The code will be available at https://github.com/tnlin/SMDN..

A Consensus Novelty Detection Ensemble Approach for Anomaly Detection in Activities of Daily Living

A new approach to creating an ensemble of novelty detection algorithms is proposed in this paper. The novelty detection process identifies new or unknown data by detecting if a test data differs significantly from the data available during training. It is applicable for anomaly detection in a situation where there is sufficiently large training data representing the normal class and little or no training data for the anomalous (abnormal) class. Abnormality in Activities of Daily Living (ADL) is identified as any significant deviation from an individual’s usual behavioural routine. Novelty detection is relevant to ADL anomaly detection since abnormalities in ADL are rare and data representing the anomalous cases are not readily available. The proposed Consensus Novelty Detection Ensemble approach is based on the concept of internal and external consensus. The internal consensus is an internal voting scheme within each model in the ensemble while the external consensus is an external voting scheme among the ensemble models. The weight of each model is estimated based on its performance and a score, called “Normality Score”. Computed score is used in classifying the data as abnormal (anomalous) based on certain threshold crossing, normal otherwise. Experimental evaluation is conducted to detect abnormalities in ADL data obtained from CASAS repository as well as experimental dataset collected for this research. The obtained results show that the proposed approach is able to identify anomalous instances. The proposed approach offers more flexibility compared with the existing approaches by allowing the Normality Score threshold to be adjusted without retraining the models.

Novelty detection improves performance of reinforcement learners in fluctuating, partially observable environments

Evolved and engineered organisms must adapt to fluctuating environments that are often only partially observed. We show that adaptation to a second environment can be significantly harder after adapting to a first, completely unrelated environment, even when using second-order learning algorithms and a constant learning rate. In effect, there is a lack of fading memory in the organism’s performance. However, organisms can adapt well to the second environment by incorporating a simple novelty detection algorithm that signals when the environment has changed and reinitializing the parameters that define their behavior if so. We propose that it may be fruitful to look for signs of this novelty detection in biological organisms, and to engineer novelty detection algorithms into artificial organisms.

An adaptive algorithm for anomaly and novelty detection in evolving data streams

In the era of big data, considerable research focus is being put on designing efficient algorithms capable of learning and extracting high-level knowledge from ubiquitous data streams in an online fashion. While, most existing algorithms assume that data samples are drawn from a stationary distribution, several complex environments deal with data streams that are subject to change over time. Taking this aspect into consideration is an important step towards building truly aware and intelligent systems. In this paper, we propose GNG-A, an adaptive method for incremental unsupervised learning from evolving data streams experiencing various types of change. The proposed method maintains a continuously updated network (graph) of neurons by extending the Growing Neural Gas algorithm with three complementary mechanisms, allowing it to closely track both gradual and sudden changes in the data distribution. First, an adaptation mechanism handles local changes where the distribution is only non-stationary in some regions of the feature space. Second, an adaptive forgetting mechanism identifies and removes neurons that become irrelevant due to the evolving nature of the stream. Finally, a probabilistic evolution mechanism creates new neurons when there is a need to represent data in new regions of the feature space. The proposed method is demonstrated for anomaly and novelty detection in non-stationary environments. Results show that the method handles different data distributions and efficiently reacts to various types of change.

Automated novelty detection in the WISE survey with one-class support vector machines

Wide-angle photometric surveys of previously uncharted sky areas or wavelength regimes will always bring in unexpected sources whose existence and properties cannot be easily predicted from earlier observations: novelties or even anomalies. Such objects can be efficiently sought for with novelty detection algorithms. Here we present an application of such a method, called one-class support vector machines (OCSVM), to search for anomalous patterns among sources preselected from the mid-infrared AllWISE catalogue covering the whole sky. To create a model of expected data we train the algorithm on a set of objects with spectroscopic identifications from the SDSS DR13 database, present also in AllWISE. OCSVM detects as anomalous those sources whose patterns - WISE photometric measurements in this case - are inconsistent with the model. Among the detected anomalies we find artefacts, such as objects with spurious photometry due to blending, but most importantly also real sources of genuine astrophysical interest. Among the latter, OCSVM has identified a sample of heavily reddened AGN/quasar candidates distributed uniformly over the sky and in a large part absent from other WISE-based AGN catalogues. It also allowed us to find a specific group of sources of mixed types, mostly stars and compact galaxies. By combining the semi-supervised OCSVM algorithm with standard classification methods it will be possible to improve the latter by accounting for sources which are not present in the training sample but are otherwise well-represented in the target set. Anomaly detection adds flexibility to automated source separation procedures and helps verify the reliability and representativeness of the training samples. It should be thus considered as an essential step in supervised classification schemes to ensure completeness and purity of produced catalogues.

사용자 행위 모델과 이상치 탐지 알고리즘을 활용한 내부자 이상행위 탐지

사용자 행위 모델과 이상치 탐지 알고리즘을 활용한 내부자 이상행위 탐지

Improved epileptic seizure detection combining dynamic feature normalization with EEG novelty detection.

Continuous electroencephalographic monitoring of critically ill patients is an established procedure in intensive care units. Seizure detection algorithms, such as support vector machines (SVM), play a prominent role in this procedure. To correct for inter-human differences in EEG characteristics, as well as for intra-human EEG variability over time, dynamic EEG feature normalization is essential. Recently, the median decaying memory (MDM) approach was determined to be the best method of normalization. MDM uses a sliding baseline buffer of EEG epochs to calculate feature normalization constants. However, while this method does include non-seizure EEG epochs, it also includes EEG activity that can have a detrimental effect on the normalization and subsequent seizure detection performance. In this study, EEG data that is to be incorporated into the baseline buffer are automatically selected based on a novelty detection algorithm (Novelty-MDM). Performance of an SVM-based seizure detection framework is evaluated in 17 long-term ICU registrations using the area under the sensitivity–specificity ROC curve. This evaluation compares three different EEG normalization methods, namely a fixed baseline buffer (FB), the median decaying memory (MDM) approach, and our novelty median decaying memory (Novelty-MDM) method. It is demonstrated that MDM did not improve overall performance compared to FB (p < 0.27), partly because seizure like episodes were included in the baseline. More importantly, Novelty-MDM significantly outperforms both FB (p = 0.015) and MDM (p = 0.0065).Electronic supplementary materialThe online version of this article (doi:10.1007/s11517-016-1479-8) contains supplementary material, which is available to authorized users.

The Effect of Personalization on Smartphone-Based Fall Detectors.

The risk of falling is high among different groups of people, such as older people, individuals with Parkinson's disease or patients in neuro-rehabilitation units. Developing robust fall detectors is important for acting promptly in case of a fall. Therefore, in this study we propose to personalize smartphone-based detectors to boost their performance as compared to a non-personalized system. Four algorithms were investigated using a public dataset: three novelty detection algorithms—Nearest Neighbor (NN), Local Outlier Factor (LOF) and One-Class Support Vector Machine (OneClass-SVM)—and a traditional supervised algorithm, Support Vector Machine (SVM). The effect of personalization was studied for each subject by considering two different training conditions: data coming only from that subject or data coming from the remaining subjects. The area under the receiver operating characteristic curve (AUC) was selected as the primary figure of merit. The results show that there is a general trend towards the increase in performance by personalizing the detector, but the effect depends on the individual being considered. A personalized NN can reach the performance of a non-personalized SVM (average AUC of 0.9861 and 0.9795, respectively), which is remarkable since NN only uses activities of daily living for training.

Evaluation of Multiclass Novelty Detection Algorithms for Data Streams

Data stream mining is an emergent research area that investigates knowledge extraction from large amounts of continuously generated data, produced by non-stationary distribution. Novelty detection, the ability to identify new or previously unknown situations, is a useful ability for learning systems, especially when dealing with data streams, where concepts may appear, disappear, or evolve over time. There are several studies currently investigating the application of novelty detection techniques in data streams. However, there is no consensus regarding how to evaluate the performance of these techniques. In this study, we propose a new evaluation methodology for multiclass novelty detection in data streams able to deal with: i) unsupervised learning, which generates novelty patterns without an association with the true classes, where one class may be composed of a novelty set, ii) confusion matrix that increases over time, iii) confusion matrix with a column representing unknown examples, i.e., those not explained by the model, and iv) representation of the evaluation measures over time. We propose a new methodology to associate the novelty patterns detected by the algorithm, in an unsupervised fashion, with the true classes. Finally, we evaluate the performance of the proposed methodology through the use of known novelty detection algorithms with artificial and real data sets.

MINAS: multiclass learning algorithm for novelty detection in data streams

Data stream mining is an emergent research area that aims at extracting knowledge from large amounts of continuously generated data. Novelty detection (ND) is a classification task that assesses if one or a set of examples differ significantly from the previously seen examples. This is an important task for data stream, as new concepts may appear, disappear or evolve over time. Most of the works found in the ND literature presents it as a binary classification task. In several data stream real life problems, ND must be treated as a multiclass task, in which, the known concept is composed by one or more classes and different new classes may appear. This work proposes MINAS, an algorithm for ND in data streams. MINAS deals with ND as a multiclass task. In the initial training phase, MINAS builds a decision model based on a labeled data set. In the online phase, new examples are classified using this model, or marked as unknown. Groups of unknown examples can be used later to create valid novelty patterns (NP), which are added to the current model. The decision model is updated as new data come over the stream in order to reflect changes in the known classes and allow the addition of NP. This work also presents a set of experiments carried out comparing MINAS and the main novelty detection algorithms found in the literature, using artificial and real data sets. The experimental results show the potential of the proposed algorithm.

Detection of structural damage using novelty detection algorithm under variational environmental and operational conditions

Novelty detection is a widely used algorithm in different fields of study due to its capabilities to recognize any kind of abnormalities in a specific process in order to ensure better working in normal conditions. In the context of Structural Health Monitoring (SHM), this method is utilized as damage detection technique because the presence of defects can be considered as abnormal to the structure. Nevertheless, the performance of such a method could be jeopardized if the structure is operating in harsh environmental and operational conditions (EOCs). In this paper, novelty detection statistical technique is used to investigate the detection of damages under various EOCs. Experiments were conducted with different scenarios: damage sizes and shapes. EOCs effects were simulated by adding stochastic noise to the collected experimental data. Different levels of noise were studied to determine the accuracy and the performance of the proposed method.

An integrated system for voice command recognition and emergency detection based on audio signals

The recent reports on population ageing in the most advanced countries are driving governments and the scientific community to focus on technologies for providing assistance to people in their own homes. Particular attention has been devoted to solutions based on acoustic signals since they provide a convenient way to monitor people activities and they enable hands-free human–machine interfaces. In this context, this paper presents a complete solution for voice command recognition and emergency detection based on audio signals entirely integrated in a low-consuming embedded platform. The system combines an active operation mode were distress calls are captured and a vocal interface is enabled for controlling the home automation subsystem, and a pro-active mode, were a novelty detection algorithm detects abnormal acoustic events to alert the user of a possible emergency. In the first operation mode, a Voice Activity Detector captures voice segments of the audio signal, and a speech recogniser detects commands and distress calls. In the pro-active mode, an acoustic novelty detector is employed in order to be able to deal with unknown sounds, thus not requiring an explicit modelling of emergency sounds. In addition, the system integrates a VoIP infrastructure so that emergencies can be communicated to relatives or care centres. The monitoring unit is equipped with multiple microphones and it is connected to the home local area network to communicate with the home automation subsystem. The algorithms have been implemented in a low-consuming embedded platform based on a ARM Cortex-A8 CPU. The effectiveness of the adopted algorithms has been tested on two different databases: ITAAL and A3Novelty. The obtained results show that the adopted solutions are suitable for speech and audio event monitoring in a realistic scenario.

Automatic detection and quantitative assessment of peculiar galaxy pairs in Sloan Digital Sky Survey

We applied computational tools for automatic detection of peculiar galaxy pairs. We first detected in SDSS DR7 ~400,000 galaxy images with i magnitude <18 that had more than one point spread function, and then applied a machine learning algorithm that detected ~26,000 galaxy images that had morphology similar to the morphology of galaxy mergers. That dataset was mined using a novelty detection algorithm, producing a short list of 500 most peculiar galaxies as quantitatively determined by the algorithm. Manual examination of these galaxies showed that while most of the galaxy pairs in the list were not necessarily peculiar, numerous unusual galaxy pairs were detected. In this paper we describe the protocol and computational tools used for the detection of peculiar mergers, and provide examples of peculiar galaxy pairs that were detected.