Online Novelty Detection Research Articles

Motor online novelty detection scheme based on one-class hyperdisk

Online novelty detection is of great importance in the series production of motors. This study developed an online novelty detection scheme for motors based on a one-class hyperdisk (OCHD) model. In the OCHD approach, the decision boundary is estimated using a hyperdisk (HD), which is derived from the training sample set. The HD model addresses the underestimation issue commonly associated with convex-hull-based methods by providing a more accurate estimation of the class region. Furthermore, an optimal separating hyperplane is constructed at the nearest point on the HD by solving a quadratically constrained quadratic program problem. Statistical features refined by the Laplacian score are employed in the proposed novelty detection scheme. This study introduces an online novelty detection scheme for assessing motor quality in actual series production. The test results from the offline experiment demonstrate the superiority of the OCHD method. Datasets collected at the end of the production line were evaluated using the proposed novelty detection scheme. The inspection results for motor components confirm that the proposed method effectively identifies faulty motors during the series production process.

Asymmetric HMMs for Online Ball-Bearing Health Assessments

The degradation of critical components inside large industrial assets, such as ball-bearings, has a negative impact on production facilities, reducing the availability of assets due to an unexpectedly high failure rate. Machine learning-based monitoring systems can estimate the remaining useful life (RUL) of ball bearings, reducing the downtime by early failure detection. However, traditional approaches for predictive systems require run-to-failure (RTF) data as training data, which in real scenarios can be scarce and expensive to obtain as the expected useful life could be measured in years. Therefore, to overcome the need of RTF, we propose a new methodology based on online novelty detection and asymmetrical hidden Markov models (As-HMMs) to work out the health assessment. This new methodology does not require previous RTF data and can adapt to natural degradation of mechanical components over time in data-stream and online environments. As the system is designed to work online within the electrical cabinet of machines, it has to be deployed using embedded electronics. Therefore, a performance analysis of As-HMM is presented to detect the strengths and critical points of the algorithm. To validate our approach, we use real life ball-bearing data sets and compare our methodology with other methodologies where no RTF data are needed and check the advantages in RUL prediction and health monitoring. As a result, we showcase a complete end-to-end solution from the sensor to actionable insights regarding RUL estimation toward maintenance application in real industrial environments.

Experiments in online expectation-based novelty-detection using 3D shape and colour perceptions for mobile robot inspection

Novelty detection is a very useful function for detecting abnormal data in any application. An expectation-based novelty-detection approach has been introduced that learns the dynamic relationship model among normal data in order to predict the next expected data. Most novelty-detection systems use an offline approach with a fixed structure, a system type that has limitations when the data count in the environment is unknown. A new expectation-based novelty-detection system features an online recurrent neural network approach that learns the data by inserting new nodes or deleting unused nodes from its structure. Generally, to detect novelties, a global novelty threshold is defined to filter out all input data as novel whenever the prediction error of the network exceeds the threshold. However, because a neural network cannot learn to predict all classes of input data perfectly, using a global novelty threshold leads to the misclassification of the insufficiently learned normal data as novel. To overcome this problem, the novelty-detection system has been improved to learn local novelty thresholds alongside its learning to predict expectations. The proposed algorithm is applied to an online novelty detection using colour and depth data obtained from a Kinect sensor on a mobile robot. The performance of the expected novelty detector and its limitations during experiments are analysed and shown. Furthermore, colour and depth data as inputs into the novelty filter are separately analysed and their contributions on the overall novelty detection highlighted. In conclusion, the performance of the novelty filter could further be improved by applying a better feature-selection technique to extract more interesting features from high-dimensional input data.

Incremental null Foley‐Sammon transform

The Foley-Sammon transform (FST) is one of the most well-known dimensionality reduction and feature extraction methods. However, the classical FST cannot be used directly in the small sample size problem where the within-class scatter matrix is singular. Null-space based FST (NFST) provides a good solution to this problem. Proposed is a fast incremental NFST (INFST). INFST extracts new information brought by newly-added samples and integrates it with the existing model by an efficient updating scheme. INFST can achieve the aims of online classification and novelty detection. Experiments on real-world data demonstrate the effectiveness of INFST.

Probabilistic Novelty Detection With Support Vector Machines

Novelty detection, or one-class classification, is of particular use in the analysis of high-integrity systems, in which examples of failure are rare in comparison with the number of examples of stable behaviour, such that a conventional multi-class classification approach cannot be taken. Support Vector Machines (SVMs) are a popular means of performing novelty detection, and it is conventional practice to use a train-validate-test approach, often involving cross-validation, to train the one-class SVM, and then select appropriate values for its parameters. An alternative method, used with multi-class SVMs, is to calibrate the SVM output into conditional class probabilities. A probabilistic approach offers many advantages over the conventional method, including the facility to select automatically a probabilistic novelty threshold. The contributions of this paper are (i) the development of a probabilistic calibration technique for one-class SVMs, such that on-line novelty detection may be performed in a probabilistic manner; and (ii) the demonstration of the advantages of the proposed method (in comparison to the conventional one-class SVM methodology) using case studies, in which one-class probabilistic SVMs are used to perform condition monitoring of a high-integrity industrial combustion plant, and in detecting deterioration in patient physiological condition during patient vital-sign monitoring.

From self-assessment to frustration, a small step toward autonomy in robotic navigation.

Autonomy and self-improvement capabilities are still challenging in the fields of robotics and machine learning. Allowing a robot to autonomously navigate in wide and unknown environments not only requires a repertoire of robust strategies to cope with miscellaneous situations, but also needs mechanisms of self-assessment for guiding learning and for monitoring strategies. Monitoring strategies requires feedbacks on the behavior's quality, from a given fitness system in order to take correct decisions. In this work, we focus on how a second-order controller can be used to (1) manage behaviors according to the situation and (2) seek for human interactions to improve skills. Following an incremental and constructivist approach, we present a generic neural architecture, based on an on-line novelty detection algorithm that may be able to self-evaluate any sensory-motor strategies. This architecture learns contingencies between sensations and actions, giving the expected sensation from the previous perception. Prediction error, coming from surprising events, provides a measure of the quality of the underlying sensory-motor contingencies. We show how a simple second-order controller (emotional system) based on the prediction progress allows the system to regulate its behavior to solve complex navigation tasks and also succeeds in asking for help if it detects dead-lock situations. We propose that this model could be a key structure toward self-assessment and autonomy. We made several experiments that can account for such properties for two different strategies (road following and place cells based navigation) in different situations.

Anytime online novelty and change detection for mobile robots

AbstractIn many mobile robot applications, the high cost of damaging the robot or its environment makes even rare failures unacceptable. To mitigate this risk, a robot must be able to detect potentially hazardous situations before it experiences a major failure. This problem therefore becomes one of novelty and change detection: how a robot can identify when perception inputs differ from prior inputs seen during training or previous operation in the same area. With this ability, the system can either avoid novel locations to minimize risk or stop and enlist human help via supervisory control or teleoperation. We present an anytime novelty detection algorithm that deals with noisy and redundant high‐dimensional feature spaces that are common in robotics by utilizing prior class information within the training set. This approach is also well suited for online use when a constantly adjusting environmental model is beneficial. Furthermore, we address the problem of change detection in an environment of repeated operation by framing it as a location‐specific version of novelty detection and present an online scene segmentation algorithm that improves accuracy across diverse environments. We validate these approaches through extensive experiments onboard two outdoor mobile robot platforms, show that our approaches are robust to noisy sensor data and moderate registration errors, and argue how such abilities could be key in increasing the real‐world applications and impact of mobile robotics. © 2011 Wiley Periodicals, Inc.

Adaptive One-Class Support Vector Machine

In this correspondence, we derive an online adaptive one-class support vector machine. The machine structure is updated via growing and pruning mechanisms and the weights are updated using structural risk minimization principles underlying support vector machines. Our approach leads to very compact machines compared to other online kernel methods whose size, unless truncated, grows almost linearly with the number of observed patterns. The proposed method is online in the sense that every pattern is only presented once to the machine and there is no need to store past samples and adaptive in the sense that it can forget past input patterns and adapt to the new characteristics of the incoming data. Thus, the characterizing properties of our algorithm are compactness, adaptiveness and real-time processing capabilities, making it especially well-suited to solve online novelty detection problems. Regarding algorithm performance, we have carried out experiments in a time series segmentation problem, obtaining favorable results in both accuracy and model complexity with respect to two existing state-of-the-art methods.

Novelty detection with application to data streams

This paper presents and evaluates an approach to novelty detection that addresses it as the problem of identifying novel concepts in a continuous learning scenario, as an extension to a single-class classification problem. OLINDDA, an OnLIne Novelty and Drift Detection Algorithm that implements this approach, uses efficient standard clustering algorithms to continuously generate candidate clusters among examples that were not explained by the current known concepts. Clusters complying with a validation criterion that takes cohesiveness and representativeness into account are initially identified as concepts. By merging similar concepts, OLINDDA may enhance the representation of some concepts as it advances toward its final goal of describing novel emerging concepts in an unsupervised way. The proposed approach is experimentally evaluated by the use of several measures taken throughout the learning process. Results show that it is capable of identifying novel concepts that are pure and correspond to real classes, disregarding unrepresentative clusters and outliers.

On-line novelty detection by recursive dynamic principal component analysis and gas sensor arrays under drift conditions

Leakage detection is a common chemical-sensing application. Leakage detection by thresholds on a single sensor signal suffers from important drawbacks when sensors show drift effects or when they are affected by other long-term cross sensitivities. In this paper, we present an adaptive method based on a recursive dynamic principal component analysis (RDPCA) algorithm that models the relationships between the sensors in the array and their past history. In normal conditions, a certain variance distribution characterizes sensor signals, however, in the presence of a new source of variance the PCA decomposition changes drastically. In order to prevent the influence of sensor drift, the model is adaptive, and it is calculated in a recursive manner with minimum computational effort. The behavior of this technique is studied with synthetic and real signals arising by oil vapor leakages in an air compressor. Results clearly demonstrate the efficiency of the proposed method

Automated Exploration and Inspection: Comparing Two Visual Novelty Detectors

Mobile robot applications that involve exploration and inspection of dynamic environments benefit, and often even are dependant on reliable novelty detection algorithms. In this paper we compare and discuss the performance and functionality of two different on-line novelty detection algorithms, one based on incremental Principal Component Analysis and the other on a Grow-When-Required artificial neural network. A series of experiments using visual input obtained by a mobile robot interacting with laboratory and real-world environments demonstrate and measure advantages and disadvantages of each approach.

On-line novelty detection for autonomous mobile robots

The use of mobile robots for inspection tasks is an attractive idea. A robot can travel through environments that humans cannot, and can be trained to identify sensor perceptions that signify potential or actual problems without requiring human intervention. However, in many cases, the appearance of a problem can vary widely, and ensuring that the robot does not miss any possible appearance of the problem (false negatives) is virtually impossible using conventional methods. This paper presents an alternative methodology using novelty detection. A neural network is trained to ignore normal perceptions that do not suggest any problems, so that anything that the robot has not sensed before is highlighted as a possible fault. This makes the incidence of false negatives less likely. We propose a novelty filter that can operate on-line, so that each new input is evaluated for novelty with respect to the data seen so far. The novelty filter learns to ignore inputs that have been sensed previously, or where similar inputs have been perceived. We demonstrate the use of the novelty filter on a series of simple inspection tasks using a mobile robot. The robot highlights those parts of an environment that are novel in some way, that is they are not part of the model acquired during exploration of a different environment. We show the effectiveness of the method using inputs from both sonar sensors and a monochrome camera.

Online Learning with Kernels

Kernel-based algorithms such as support vector machines have achieved considerable success in various problems in batch setting, where all of the training data is available in advance. Support vector machines combine the so-called kernel trick with the large margin idea. There has been little use of these methods in an online setting suitable for real-time applications. In this paper, we consider online learning in a reproducing kernel Hilbert space. By considering classical stochastic gradient descent within a feature space and the use of some straightforward tricks, we develop simple and computationally efficient algorithms for a wide range of problems such as classification, regression, and novelty detection. In addition to allowing the exploitation of the kernel trick in an online setting, we examine the value of large margins for classification in the online setting with a drifting target. We derive worst-case loss bounds, and moreover, we show the convergence of the hypothesis to the minimizer of the regularized risk functional. We present some experimental results that support the theory as well as illustrating the power of the new algorithms for online novelty detection.

On-line novelty detection for artefact identification in automatic anaesthesia record keeping

We report the design of a kernel-based on-line novelty detector (ADDaM — Automatic Dynamic Data Mapper) and its use in the detection of artefacts in an automatic anaesthesia record keeper (AARK). ADDaM produces a partitioned history of any ordered data stream and constructs a probability distribution function (PDF) from that history using Gaussian kernels. Two forms of PDF are possible: a s tatic PDF where the prior probability of each kernel is determined by the number of observations it represents and a temporal PDF where more recent observations have a higher prior probability. Testing against the current PDF assesses the novelty of the next point entering the stream. The performance of this method for artefact detection in heart rate data was compared to Kalman, ARIMA and moving mean filters using receiver operator characteristic (ROC) curves. Performance was measured using the area under the curves (AUC), and the false positive rate (FPR) and positive predictive value (PPV) calculated at the optimal cost-point on the curves. The results obtained were: ADDaM (Static PDF) AUC 0.92, FPR 0.12, PPV 0.12 and ADDaM (Temporal PDF) AUC 0.97, FPR 0.12, PPV 0.15. Both ADDaM-based methods out performed all other on-line methods tested.

Novelty detection using products of simple experts—a potential architecture for embedded systems

The ‘Product of Experts’ architecture (concentrating on binary-stochastic elements) is described in the context of its suitability for implementation as mixed-mode hardware, with algorithmic modifications that render the training procedure hardware-implementable. Results show that the PoE is capable of modelling non-linear, multi-dimensional data drawn from both artificial and real sources. The capability of the PoE to perform on-line novelty detection is described and demonstrated on both artificial and real data.