Novelty Detection Techniques Research Articles

A novelty detection approach to effectively predict conversion from mild cognitive impairment to Alzheimer’s disease

Accurately recognising patients with progressive mild cognitive impairment (pMCI) who will develop Alzheimer’s disease (AD) in subsequent years is very important, as early identification of those patients will enable interventions to potentially reduce the number of those transitioning from MCI to AD. Most studies in this area have concentrated on high-dimensional neuroimaging data with supervised binary/multi-class classification algorithms. However, neuroimaging data is more costly to obtain than non-imaging, and healthcare datasets are normally imbalanced which may reduce classification performance and reliability. To address these challenges, we proposed a new strategy that employs unsupervised novelty detection (ND) techniques to predict pMCI from the AD neuroimaging initiative non-imaging data. ND algorithms, including the k-nearest neighbours (kNN), k-means, Gaussian mixture model (GMM), isolation forest (IF) and extreme learning machine (ELM), were employed and compared with supervised binary support vector machine (SVM) and random forest (RF). We introduced optimisation with nested cross-validation and focused on maximising the adjusted F measure to ensure maximum generalisation of the proposed system by minimising false negative rates. Our extensive experimental results show that ND algorithms (0.727 ± 0.029 kNN, 0.7179 ± 0.0523 GMM, 0.7276 ± 0.0281 ELM) obtained comparable performance to supervised binary SVM (0.7359 ± 0.0451) with 20% stable MCI misclassification tolerance and were significantly better than RF (0.4771 ± 0.0167). Moreover, we found that the non-invasive, readily obtainable, and cost-effective cognitive and functional assessment was the most efficient predictor for predicting the pMCI within 2 years with ND techniques. Importantly, we presented an accessible and cost-effective approach to pMCI prediction, which does not require labelled data.

CREDIT CARD FRAUD DETECTION USING ISOLATION FOREST AND LOCAL OUTLIER FACTOR

Nowadays, card fraud is increasing due to the prevalence of modern technology. Thus, Automatic systems to detect and prevent against card fraud are a significant tool in the financial industries battle against card crime. Machine learning and novelty detection techniques approaches are taken in Credit Card Fraud Detection field, since they are effective technologies and methodologies and easy to apply at the same time. It is used in order to reduce fraud activities. The main aim of this project was to create a program that detects and Identifies potentially fraudulent credit card transactions from a given data set, and evaluate its performance to be compared with other classifiers with evaluation metric. The program was trained with the given data set, based on the some of the most popular Machine Learning and Deep Learning Classification algorithms which are; Random Forest, Isolation Forest and Neural Networks Algorithm Load Balancing and Feature Selection were maintained throughout the project, and it was implemented using Python programming language. However, there were no autonomous system that will be able to categorically define a transaction as fraud. The objective was to highlight those transactions that have a high probability of being fraudulent based on some criteria, known or otherwise learnt. Key Words: isolation forest and local outlier factor, Pandas, Json, Matplotlib and seaborn.

Light-Duty Vehicle Trip Classification Using One-Class Novelty Detection and Exhaustive Feature Extraction

Travel mode classification within travel survey data sets, especially light-duty vehicle (LDV) trips, is foundational, though nontrivial, to emerging mobility systems, travel behavior analysis, and fuel consumption estimation. Current travel mode detection approaches require well-sampled and balanced data sets with ground-truth travel mode labels. The detection approaches are rarely applied and validated on large-scale, real-world data sets, which may not satisfy the data set requirements. This article proposes an LDV trip detection model as a supplement to current travel mode detection methods for the case when the training set is highly (and/or completely) unbalanced, to the extent that classical machine learning approaches become difficult or impossible to deploy. The proposed model uses a novelty detection technique—one-class support vector machines—and a novel exhaustive feature extraction technique on continuous time series data [i.e., global positioning system (GPS) speed profiles] for single-mode trip trajectories. Training and validation of the model are conducted on a large-scale, real-world data set. The proposed method accurately identifies LDV trips from a broad set of multimodal trips by leveraging a wealth of pre-existing in-vehicle GPS travel data. Additional sensitivity analysis sheds light on the optimal training size and feature selection, which will benefit applications limited by highly imbalanced data. This article also discusses performance comparison with regular machine learning approaches, the model's robustness, and the potential to extend the proposed model to multimodal trip prediction.

Monitoring Optical Variability in Complex Inland Waters Using Satellite Remote Sensing Data

Optical classification for water bodies was carried out based on satellite remote sensing data, which avoided the limitation of having a limited amount of in situ measured spectral data. Unsupervised cluster analysis was performed on 53,815 reflectance spectra extracted at 500-m intervals based on the same season or quasi-same season Landsat 8 SR data using the algorithm of fuzzy c-means. Lakes and reservoirs in the study area were comprehensively identified as three optical types representing different limnological features. The shape and amplitude characteristics of the reflectance spectra for the three optical water types indicated that one corresponds to the clearest water, one corresponds to turbid water, and the other is moderate clear water. The novelty detection technique was further used to label the match-ups of the in situ data set collected during 2006 to 2019 in 12 field surveys based on mathematical rules of the three optical water types. The results confirmed that each optical water type was associated with different bio-optical properties, and the total suspended matter of the clearest, moderate clear and turbid water types were 14.99 mg/L, 41.06 mg/L and 83.81 mg/L, respectively. Overall, the clearest, moderate clear and turbid waters in the study area accounted for 49.3%, 36.7% and 14.0%, respectively. The spatial distribution of optical water types in the study area was seamlessly mapped. Results showed that the bio-optical conditions of the water distributed across the southeast region were roughly homogeneous, but in most of other regions and within some water bodies, they showed a patchy distribution and heterogeneity. This study is useful for monitoring water quality and provides a useful foundation to develop or tuning algorithms to retrieve water quality parameters.

CNN‐based novelty detection for terrestrial and extra‐terrestrial autonomous exploration

Novelty detection is concerned with detecting features that do not belong to any known class or are not well represented by existing models. Ergo, in autonomous navigation novelty detection determines whether an input camera frame contains certain entities of high interest which do not correspond to a known category. One of the key requirements for the future space exploration missions is the reduction of the information to be transferred back to Earth. Thus, novelty detection techniques have been developed to select the subset of acquired images with significant measurements that justify utilisation of the limited bandwidth from the available information link. Such methods are based on the identification of salient regions, which are then evaluated against a set of trained classifiers. We explore a novelty detection approach, based on the reasoning properties of Neural Networks, which follow the same guidelines while also being trainable in an end-to-end manner. This characteristic allows for the intertwined optimisation of the individual components leading to a closer estimation of a global solution. Our experiments reveal that the proposed novelty detection system achieves better performance, as compared to hand-crafted techniques, when the learning and testing examples refer to similar environments.

Novelty detection in continuously changing environments

Self-improving system integration (SISSY) aims at mastering the challenges of system organisation decisions for subsystems with highly dynamic behaviours. This is achieved by increased decision freedom of these subsystems, i.e., by moving the corresponding design decisions from design-time to run-time and from engineers to the systems themselves. As a result, a more robust and efficient constellation of the overall system organisation is maintained even under the presence of continuous change and disturbed conditions. Appropriate integration decisions require a solid foundation of assessing the current conditions and their dynamics using identifying triggers for a necessary evaluation of their own integration status. With this article, we present mCANDIES, a modular novelty detection technique especially suited for SISSY systems that fulfils exactly these identification tasks. mCANDIES tackles an anomaly or novelty detection problem in a modular way by combining the advantages of different detection approaches by exploiting locality in different regions of the input space — depending on, e.g. the density or location of expected observations. This is augmented with information about changes in the underlying model of the observations, i.e., mCANDIES identifies concept drifts. The corresponding information can then be used to trigger online learning behaviour. We analyse the behaviour of mCANDIES on artificial and real-world data sets and compare it to other techniques from the state of the art.

Trajectory-based gait pattern shift detection for assistive robotics applications

In the medical field, rehabilitation of the lower limbs is a slow and continuous process, where healthcare professionals have to follow each patient over long periods of time. In conventional rehabilitation, the progression of a patient is assessed by a professional, who analyzes visual tracking data. This assessment is dependent on each professional’s own experience. In this paper, we propose an approach to analyze tracking data, captured by our robotic walker’s gait tracking system, to detect shifts in the gait pattern over time automatically. For this purpose, the system takes in gait tracking data and segments it into gait cycles (heel strike to heel strike). Then, our approach handles each gait cycle considering it as a group of gait parameters that define trajectories in that time frame. From each gait parameter within the cycle, spatiotemporal features are extracted and similarity rates are computed using autoencoders. These spatiotemporal features and similarity rates are fused in a feature space which is fed to a one-class support vector machine that constructs a model of the observable gait cycle. Each posterior observed gait cycle is checked for shifts, using a set of proposed novelty detection techniques. Experimental tests using a dataset captured using our robotic walker platform revealed a promising performance when detecting if a gait cycle is ‘reference’ or ‘novel’ when compared to a previously trained model of a unique ‘reference’ gait pattern.

Localised gear anomaly detection without historical data for reference density estimation

Abstract Performing condition monitoring on rotating machines which operate under fluctuating conditions remains a challenge, with robust diagnostic techniques being required for the condition inference task. Some of the developed diagnostic techniques rely on the availability of historical fault data, which is impractical or even impossible to obtain in many circumstances and therefore novelty detection techniques such as discrepancy analysis are used. However, discrepancy analysis assumes that the condition of the machine is the same throughout the signal in the model optimisation process i.e. no localised damage is present, which can pose problems if the training data unwittingly contain a component with localised damage. In this paper, an automatic procedure is proposed for diagnosing localised gear damage in the presence of fluctuating operating conditions, with no historical data being required to model the data used in the condition inference process. The continuous wavelet transform, principal component analysis and information theory are used to obtain divergence data of the gear under consideration. The divergence data are used with Bayesian data analysis techniques to automatically infer the presence of localised anomalies due to localised gear damage. The proposed technique is validated in two experimental investigations, with promising results being obtained.

An open set recognition methodology utilising discrepancy analysis for gear diagnostics under varying operating conditions

Abstract Historical fault data are often difficult and expensive to acquire, which can prohibit the application of supervised learning techniques in the condition-based maintenance field. Hence, novelty detection techniques such as discrepancy analysis are useful because only healthy historical data are required. However, even if discrepancy analysis is implemented on a machine, some historical fault data will become available during the operational lifetime of the machine and could be utilised to improve the efficiency of the condition inference process. An open set recognition methodology relying on discrepancy analysis is proposed that is capable of detecting novelties when only healthy historical data are available. It is also capable of inferring the condition of the machine if historical fault data are available and it is further able to detect novelties in regions not well supported by the historical fault data. In the condition recognition procedure, Gaussian mixture models are used with Bayes’ rule and a decision rule to infer the condition of the machine in an open set recognition framework, where it is emphasised that it is beneficial to use the complete datasets (i.e. data acquired throughout the life of the component) for optimising the models. The benefit of the open set recognition model is that it is easy to incorporate new historical data in the framework as the data become available. In this work, practical aspects of the condition inference process such as the importance of good decision boundaries are highlighted and discussed as well. The methodology is validated on a synthetic dataset and experimental datasets acquired under varying operating conditions and it is also compared to a conventional classification process where discrepancy analysis is not used. The results indicate the potential of using the proposed methodology for rotating machine diagnostics under varying operating conditions.

Towards an Abnormal Bridge Location Identification Method Based on Novelty Detection Technique

The bridge structure abnormality recognition is one of the key steps of its health assessment. Using novelty detection technique based on BP neural network, the method to identify and locate abnormal bridge status was presented. It uses non-training-data in the original sample data to generate novelty indicator and determines threshold. If the difference between detection status indicator and normal value is larger than the threshold, the structure status is determined changed. The method adapts stepwise partition identification method. The method firstly determines damage position within a small range and then analyzes sensor data in detail, so as to locate specific position. The measured data on T beam model verifies the method can accurately carry out status identification and locate cracking position under cracking load conditions.

Evaluation of Multiclass Novelty Detection Algorithms for Data Streams

Data stream mining is an emergent research area that investigates knowledge extraction from large amounts of continuously generated data, produced by non-stationary distribution. Novelty detection, the ability to identify new or previously unknown situations, is a useful ability for learning systems, especially when dealing with data streams, where concepts may appear, disappear, or evolve over time. There are several studies currently investigating the application of novelty detection techniques in data streams. However, there is no consensus regarding how to evaluate the performance of these techniques. In this study, we propose a new evaluation methodology for multiclass novelty detection in data streams able to deal with: i) unsupervised learning, which generates novelty patterns without an association with the true classes, where one class may be composed of a novelty set, ii) confusion matrix that increases over time, iii) confusion matrix with a column representing unknown examples, i.e., those not explained by the model, and iv) representation of the evaluation measures over time. We propose a new methodology to associate the novelty patterns detected by the algorithm, in an unsupervised fashion, with the true classes. Finally, we evaluate the performance of the proposed methodology through the use of known novelty detection algorithms with artificial and real data sets.

Evaluating the reliability level of virtual metrology results for flexible process control: a novelty detection-based approach

The purpose of virtual metrology (VM) in semiconductor manufacturing is to support process monitoring and quality control by predicting the metrological values of every wafer without an actual metrology process, based on process sensor data collected during the operation. Most VM-based quality control schemes assume that the VM predictions are always accurate, which in fact may not be true due to some unexpected variations that can occur during the process. In this paper, therefore, we propose a means of evaluating the reliability level of VM prediction results based on novelty detection techniques, which would allow flexible utilization of the VM results. Our models generate a high-reliability score for a wafer's VM prediction only when its process sensor values are found to be consistent with those of the majority of wafers that are used in model building; otherwise, a low-reliability score is returned. Thus, process engineers can selectively utilize VM results based on their reliability level. Experimental results show that our reliability generation models are effective; the VM results for wafers with a high level of reliability were found to be much more accurate than those with a low level.

On damage diagnosis for a wind turbine blade using pattern recognition

With the increased interest in implementation of wind turbine power plants in remote areas, structural health monitoring (SHM) will be one of the key cards in the efficient establishment of wind turbines in the energy arena. Detection of blade damage at an early stage is a critical problem, as blade failure can lead to a catastrophic outcome for the entire wind turbine system. Experimental measurements from vibration analysis were extracted from a 9 m CX-100 blade by researchers at Los Alamos National Laboratory (LANL) throughout a full-scale fatigue test conducted at the National Renewable Energy Laboratory (NREL) and National Wind Technology Center (NWTC). The blade was harmonically excited at its first natural frequency using a Universal Resonant EXcitation (UREX) system. In the current study, machine learning algorithms based on Artificial Neural Networks (ANNs), including an Auto-Associative Neural Network (AANN) based on a standard ANN form and a novel approach to auto-association with Radial Basis Functions (RBFs) networks are used, which are optimised for fast and efficient runs. This paper introduces such pattern recognition methods into the wind energy field and attempts to address the effectiveness of such methods by combining vibration response data with novelty detection techniques.

Novelty detection in airport baggage conveyor gear-motors using Synchro-squeezing transform and Self-organizing maps


 
 
 A powerful continuous wavelet transform based signal processing tool named Synchro-squeezing transform (SST) has recently emerged in the context of non-stationary signal processing. Founded upon the premise of time-frequency (TF) reassignment, its basic objective is to provide a sharper representation of signals in the TF plane. Additionally, it can also extract the individual components of a non- stationary multi-component signal, which makes it attractive for rotating machinery signals. This work utilizes the decomposing power of SST transform to extract useful components from gear-motor signals in relevant sub-bands, followed by the application of standard rotating machinery condition indicators. For timely detection of faults in airport baggage conveyor gear-motors, a novelty detection technique based on the recently developed concepts of self-organizing maps (SOM) is applied on the condition indicators. This approach promises improved anomaly detection pow
 er than that can be achieved by applying condition indicators and SOM directly to the inherently complex raw-data. Data collected from the airport baggage conveyor gear-motors provides the test bed to demonstrate the efficacy of the proposed approach.
 
 

Maritime abnormality detection using Gaussian processes

Novelty, or abnormality, detection aims to identify patterns within data streams that do not conform to expected behaviour. This paper introduces novelty detection techniques using a combination of Gaussian processes, extreme value theory and divergence measurement to identify anomalous behaviour in both streaming and batch data. The approach is tested on both synthetic and real data, showing itself to be effective in our primary application of maritime vessel track analysis.

A novelty detection machine and its application to bank failure prediction

Novelty detection has been well-studied for many years and has found a wide range of applications, but correctly identifying the outliers is still a hard problem because of the diverse variation and the small quantity of such outliers. We address the problem using several distinct characteristics of the outliers and the normal patterns. First, normal patterns are usually grouped together, forming clusters in the high density regions of the data space. Second, outliers are characteristically very different from the normal patterns, and hence tend to be located far away from the normal patterns in the data space. Third, the number of outliers is generally very small in a given dataset. Based on these observations, we can envisage that the appropriate decision boundary segregating the outliers and the normal patterns usually lies in some low density regions of the data space. This is referred to as cluster assumption. The resultant optimization problem to learn the decision function can be solved using the mixed integer programming approach. Following that, we present a cutting plane algorithm together with a multiple kernel learning technique to solve the convex relaxation of the optimization problem. Specifically, we make use of the scarcity of the outliers to find a violating solution to the cutting plane algorithm. Experimental results with several benchmark datasets show that our proposed novelty detection method outperforms existing hyperplane and density estimation-based novelty detection techniques. We subsequently apply our method to the prediction of banking failures to identify potential bank failures or high risk banks through the traits of financial distress.

Vibration‐ and acoustic‐emissions based novelty detection of fretted bearings

PurposeThe purpose of this paper is to examine the use of a new feature reduction technique with novelty detection on vibration and acoustic‐emission sensors monitoring bearings mounted in the test benches of automotive manufacturers.Design/methodology/approachSignals from standard accelerometers and acoustic‐emission sensors were gathered from bearings operating under steady conditions on an accessory‐drive test bench. The bearings under test were subject to a variety of faults including fretting. These signals were processed and reduced to standard feature vectors, the dimensionality of which was reduced using a new principal‐component‐like technique optimized for novelty detection. The reduced data were analyzed with a novelty detection technique called the Support Vector Data Descriptor.FindingsThe classification results from these sensors, after being reduced with the proposed feature reduction technique, are substantially improved over those achievable with only standard novelty detection; nearly zero‐percent classification error was achieved.Research limitations/implicationsThe feature reduction technique depends, in part, on the availability of the fault type in question – potentially violating the normal novelty detection assumption of limited abnormal data. This may require the manufacturer to gather real or simulated fault data prior to running tests.Practical implicationsIncipient faults may be detectable at a much earlier stage in a manufacturer's component failure analysis. Test engineers may use this technique to reliably automate the fault detection process and enable improved root‐cause analysis through the earlier identification of faults.Originality/valueThe application of the feature reduction technique will provide manufacturers and researchers with a new means of improving fault detection in machinery components.

Intrusion Detection in Unstructured Contexts Using On-line Clustering and Novelty Detection

The characterization of processes behavior is usually considered whenperforming intrusion detection. Several works characterize specific aspects of systemsand attempt to detect novelties in that context, associating observed anomalies to at-tack events. Such approach is limited or even useless when the observed context isunstructured, i.e. when the monitor generates text-based log files or a variable numberof application attributes. In order to overcome such drawback, this paper considersthe use of single-pass clustering techniques to quantize unstructured data and generatetime series, using algorithms with low computational complexity, applicable in a real-world scenario. Afterward, novelty detection techniques are employed on such seriesto distinguish behavior anomalies, which are associated with intrusions. We evaluatedthe approach using a system characterization dataset and confirmed that it aggregatescontext information to represent the behavior of applications as time series, wherenovelty detection can be successfully performed.

Identifying technological opportunities using the novelty detection technique: a case of laser technology in semiconductor manufacturing

While identification of technological opportunities has received considerable attention, previous studies have some weaknesses in terms of subjectivity when finding the opportunities in practical terms. This paper proposes a systematic framework to identify technological opportunities, focusing on objective evidences which are specific and practical to be used in a business environment. To do this, we used patents as a source and employed a novelty detection technique whose primary object is detecting the novel pattern. To begin with, the patents are collected from the United States Patent and Trademark Office (USPTO) database. These patents are then pre-processed into a structured keyword vector that can represent the characteristics of each patent. These keyword vectors are then used to analyse the new and emerging pattern, using the novelty detection technique. As the final step, the results are analysed to identify the technological opportunities. A case study on laser technology in lithography is presented to show the proposed framework.

Damage alarming for bridge expansion joints using novelty detection technique based on long-term monitoring data

Damage alarming and safety evaluation using long-term monitoring data is an area of significant research activity for long-span bridges. In order to extend the research in this field, the damage alarming technique for bridge expansion joints based on long-term monitoring data was developed. The effects of environmental factors on the expansion joint displacement were analyzed. Multiple linear regression models were obtained to describe the correlation between displacements and the dominant environmental factors. The damage alarming index was defined based on the multiple regression models. At last, the X-bar control chart was utilized to detect the abnormal change of the displacements. Analysis results reveal that temperature and traffic condition are the dominant environmental factors to influence the displacement. When the confidence level of X-bar control chart is set to be 0.003, the false-positive indications of damage can be avoided. The damage sensitivity analysis shows that the proper X-bar control chart can detect 0.1 cm damage-induced change of the expansion joint displacement. It is reasonably believed that the proposed technique is robust against false-positive indication of damage and suitable to alarm the possible future damage of the expansion joints.