One-class Classification Algorithms Research Articles

Machinery Fault Signal Detection with Deep One-Class Classification

Fault detection of machinery systems is a fundamental prerequisite to implementing condition-based maintenance, which is the most eminent manufacturing equipment system management strategy. To build the fault detection model, one-class classification algorithms have been used, which construct the decision boundary only using normal class. For more accurate one-class classification, signal data have been used recently because the signal data directly reflect the condition of the machinery system. To analyze the machinery condition effectively with the signal data, features of signals should be extracted, and then, the one-class classifier is constructed with the features. However, features separately extracted from one-class classification might not be optimized for the fault detection tasks, and thus, it leads to unsatisfactory performance. To address this problem, deep one-class classification methods can be used because the neural network structures can generate the features specialized to fault detection tasks through the end-to-end learning manner. In this study, we conducted a comprehensive experimental study with various fault signal datasets. The experimental results demonstrated that the deep support vector data description model, which is one of the most prominent deep one-class classification methods, outperforms its competitors and traditional methods.

Foundations and practice of binary process discovery

Most contemporary process discovery methods take as inputs only positive examples of process executions, and so they are one-class classification algorithms. However, we have found negative examples to also be available in industry, hence we build on earlier work that treats process discovery as a binary classification problem. This approach opens the door to many well-established methods and metrics from machine learning, in particular to improve the distinction between what should and should not be allowed by the output model. Concretely, we (1) present a verified formalisation of process discovery as a binary classification problem; (2) provide cases with negative examples from industry, including real-life logs; (3) propose the Rejection Miner binary classification procedure, applicable to any process notation that has a suitable syntactic composition operator; (4) implement two concrete binary miners, one outputting Declare patterns, the other Dynamic Condition Response (DCR) graphs; and (5) apply these miners to real world and synthetic logs obtained from our industry partners and the process discovery contest, showing increased output model quality in terms of accuracy and model size.

Patient deterioration detection using one-class classification via cluster period estimation subtask

Deterioration is the significant degradation of the physical state prior to death. Detecting the deterioration of patients could provide an early warning to their families in instances of homecare, to clinicians treating hospitalized patients and to nurses by clients of retirement homes. Traditional supervised machine learning is not helpful for this purpose because the deterioration has individual differences for each patient, and the model cannot access the information about deterioration from healthy patients. This paper applies one-class classification (OCC) to detect deterioration changes. OCC can provide an early warning because the model can learn from only normal conditions. In particular, a one-class time-series classification (OCTSC) algorithm has been developed by combining K-means clustering with sliding windows and a linear regression subtask. The core idea is to detect the change in the signal period related to heart/breathing rate. For this purpose, clustering is applied to sliding windows, and the period is estimated using linear regression for the time index of arbitrary cluster. The deterioration change is detected by unseen scores computed as the error of linear regression subtask for cluster index.

Investigating the effectiveness of one-class and binary classification for fraud detection

Research into machine learning methods for fraud detection is of paramount importance, largely due to the substantial financial implications associated with fraudulent activities. Our investigation is centered around the Credit Card Fraud Dataset and the Medicare Part D dataset, both of which are highly imbalanced. The Credit Card Fraud Detection Dataset is large data and contains actual transactional content, which makes it an ideal benchmark for credit card fraud detection. The Medicare Part D dataset is big data, providing researchers the opportunity to examine national trends and patterns related to prescription drug usage and expenditures. This paper presents a detailed comparison of One-Class Classification (OCC) and binary classification algorithms, utilizing eight distinct classifiers. OCC is a more appealing option, since collecting a second label for binary classification can be very expensive and not possible to obtain within a reasonable time frame. We evaluate our models based on two key metrics: the Area Under the Precision-Recall Curve (AUPRC)) and the Area Under the Receiver Operating Characteristic Curve (AUC). Our results show that binary classification consistently outperforms OCC in detecting fraud within both datasets. In addition, we found that CatBoost is the most performant among the classifiers tested. Moreover, we contribute novel results by being the first to publish a performance comparison of OCC and binary classification specifically for fraud detection in the Credit Card Fraud and Medicare Part D datasets.

Hierarchical One-Class Model With Subnetwork for Representation Learning and Outlier Detection.

The multilayer one-class classification (OCC) frameworks have gained great traction in research on anomaly and outlier detection. However, most multilayer OCC algorithms suffer from loosely connected feature coding, affecting the ability of generated latent space to properly generate a highly discriminative representation between object classes. To alleviate this deficiency, two novel OCC frameworks, namely: 1) OCC structure using the subnetwork neural network (OC-SNN) and 2) maximum correntropy-based OC-SNN (MCOC-SNN), are proposed in this article. The novelties of this article are as follows: 1) the subnetwork is used to build the discriminative latent space; 2) the proposed models are one-step learning networks, instead of stacking feature learning blocks and final classification layer to recognize the input pattern; 3) unlike existing works which utilize mean square error (MSE) to learn low-dimensional features, the MCOC-SNN uses maximum correntropy criterion (MCC) for discriminative feature encoding; and 4) a brand-new OCC dataset, called CO-Mask, is built for this research. Experimental results on the visual classification domain with a varying number of training samples from 6131 to 513 061 demonstrate that the proposed OC-SNN and MCOC-SNN achieve superior performance compared to the existing multilayer OCC models. For reproducibility, the source codes are available at https://github.com/W1AE/OCC.

Learning Off-Road Terrain Traversability With Self-Supervisions Only

Estimating the traversability of terrain should be reliable and accurate in diverse conditions for autonomous driving in off-road environments. However, learning-based approaches often yield unreliable results when confronted with unfamiliar contexts, and it is challenging to obtain manual annotations frequently for new circumstances. In this paper, we introduce a method for learning traversability from images that utilizes only self-supervision and no manual labels, enabling it to easily learn traversability in new circumstances. To this end, we first generate self-supervised traversability labels from past driving trajectories by labeling regions traversed by the vehicle as highly traversable. Using the self-supervised labels, we then train a neural network that identifies terrains that are safe to traverse from an image using a one-class classification algorithm. Additionally, we supplement the limitations of self-supervised labels by incorporating methods of self-supervised learning of visual representations. To conduct a comprehensive evaluation, we collect data in a variety of driving environments and perceptual conditions and show that our method produces reliable estimations in various environments. In addition, the experimental results validate that our method outperforms other self-supervised traversability estimation methods and achieves comparable performances with supervised learning methods trained on manually labeled data.

On the evaluation of outlier detection and one-class classification: a comparative study of algorithms, model selection, and ensembles

It has been shown that unsupervised outlier detection methods can be adapted to the one-class classification problem (Janssens and Postma, in: Proceedings of the 18th annual Belgian-Dutch on machine learning, pp 56–64, 2009; Janssens et al. in: Proceedings of the 2009 ICMLA international conference on machine learning and applications, IEEE Computer Society, pp 147–153, 2009. https://doi.org/10.1109/ICMLA.2009.16). In this paper, we focus on the comparison of one-class classification algorithms with such adapted unsupervised outlier detection methods, improving on previous comparison studies in several important aspects. We study a number of one-class classification and unsupervised outlier detection methods in a rigorous experimental setup, comparing them on a large number of datasets with different characteristics, using different performance measures. In contrast to previous comparison studies, where the models (algorithms, parameters) are selected by using examples from both classes (outlier and inlier), here we also study and compare different approaches for model selection in the absence of examples from the outlier class, which is more realistic for practical applications since labeled outliers are rarely available. Our results showed that, overall, SVDD and GMM are top-performers, regardless of whether the ground truth is used for parameter selection or not. However, in specific application scenarios, other methods exhibited better performance. Combining one-class classifiers into ensembles showed better performance than individual methods in terms of accuracy, as long as the ensemble members are properly selected.

Clustering ensemble-based novelty score for outlier detection

Recently, One-class classification algorithms have been successfully used for outlier detection problems in several industrial fields. However, in case of that the target class has complex structures, single outlier detection model with one-class classifier often poorly performs because it cannot appropriately reflect intrinsic data structures. To address this limitation, we propose a clustering ensemble-based novelty score algorithm. The proposed algorithm calculates novelty score from the mixture of multiple clustering solutions generated by both random subspace and random-K ensemble approaches. Then, final ensemble novelty score is defined by summarizing multiple novelty scores obtained from individual clustering results. Because these multiple novelty scores are computed from many possible characteristics of target class information, the proposed ensemble novelty score can appropriately reflect the inherent structures of target class. Experiments were conducted on various benchmark datasets to compared with existing methods and investigate the properties of the proposed algorithm. The experimental results confirm that the proposed algorithm outperforms existing one-class classification methods in various cases.

IMCOD: Incremental multi-class outlier detection model in data streams

Multi-class outlier detection from multi-class data streams is a crucial yet challenging task for many real-life applications and services. Various detection methods are available e.g., derivation from one-class classification algorithms, prior knowledge of the underlying data distribution, method-specific hyperparameters tuning for different data sets. However, almost all of these methods are designed to work in the batch learning mode. Therefore, it is difficult to apply the existing methods to multi-class data stream applications. To overcome these limitations, this study proposes a new incremental multi-class outlier detection model (iMCOD). Unlike previous approaches, iMCOD incorporates an incremental support vector machine (iSVM) with an incremental local outlier factor (iLOF) in a unified framework. The incoming query sample was classified using iSVM, and the iLOF score was computed using the newly developed class-based nearest neighbor (NN) concept. Employing the class-based median of the NN average deviation metric, we determined whether the query sample is an outlier, and the iSVM was updated accordingly for the next sample. The iMCOD can detect outliers from multi-class data streams without the need for hyperparameter tuning. Extensive experimental and statistical analyses on 15 real-world data sets demonstrated that iMCOD outperformed 13 competitors.

OCFSP: self-supervised one-class classification approach using feature-slide prediction subtask for feature data

One-class classification (OCC) is a machine learning problem where training data has only one class. Recently, self-supervised OCC algorithms have been increasing attention. These algorithms train the model for pretext tasks and use the model error for OCC. However, these tasks are specialized for images, and applying them to feature data is not practical or appropriate for such a purpose. The motivation of this study is to apply self-supervised OCC to feature data. For this purpose, this paper proposes an OCC approach using feature-slide prediction (FSP) subtask for feature data (OCFSP). The main originality is the FSP subtask, which is the first classification subtask for feature data. In particular, the proposed method creates a self-labeled dataset by generating additional feature vectors with the feature slide of original vectors and self-annotating these vectors as the number of the slides. Such a dataset is applied to train a multi-class classifier to predict the number of feature slides. Since this classification model learns data from only one class, the FSP accuracy for a seen class is higher relative to unseen classes. Accordingly, OCC could be made using the accuracy of FSP. The proposed methods are experimented with using the imbalanced-learn, covtype, and kddcup datasets. OCFSP shows fair accuracy where few training data is given. In addition, classification subtask for feature data shows a relatively fast testing speed, unlike image data. Therefore, the bottleneck of the self-supervised approach is considered the memory size, which is the main difference between image and feature data. Source code is uploaded at https://github.com/ToshiHayashi/OCFSP

A One-Class Classification method based on Expanded Non-Convex Hulls

This paper presents an intuitive, robust and efficient One-Class Classification algorithm. The method developed is called OCENCH (One-class Classification via Expanded Non-Convex Hulls) and bases its operation on the construction of subdivisible and expandable non-convex hulls to represent the target class. The method begins by reducing the dimensionality of the data to two-dimensional spaces using random projections. After that, an iterative process based on Delaunay triangulations is applied to these spaces to obtain simple polygons that characterizes the non-convex shape of the normal class data. In addition, the method subdivides the non-convex hulls to represent separate regions in space if necessary. The method has been evaluated and compared to several main algorithms of the field using real data sets. In contrast to other methods, OCENCH can deal with non-convex and disjointed shapes. Finally, its execution can be carried out in a parallel way, which is interesting to reduce the execution time.

One-Class Classification by Ensembles of Random Planes (OCCERPs).

One-class classification (OCC) deals with the classification problem in which the training data have data points belonging only to the target class. In this paper, we present a one-class classification algorithm, One-Class Classification by Ensembles of Random Plane (OCCERP), that uses random planes to address OCC problems. OCCERP creates many random planes. There is a pivot point in each random plane. A data point is projected in a random plane and a distance from a pivot point is used to compute the outlier score of the data point. Outlier scores of a point computed using many random planes are combined to get the final outlier score of the point. An extensive comparison of the OCCERP algorithm with state-of-the-art OCC algorithms on several datasets was conducted to show the effectiveness of the proposed approach. The effect of the ensemble size on the performance of the OCCERP algorithm is also studied.

Learning target class eigen subspace (LTC-ES) via eigen knowledge grid

In one-class classification (OCC) tasks, only the target class (class-of-interest (CoI)) samples are well defined during training, whereas the other class samples are totally absent. In OCC algorithms, the high dimensional data adds computational overhead apart from its intrinsic property of curse of dimensionality. For target class learning, conventional dimensionality reduction (DR) techniques are not suitable due to negligence of the unique statistical properties of CoI samples. In this context, the present research proposes a novel target class guided DR technique to extract the eigen knowledge grid that contains the most promising eigenvectors of variance-covariance matrix of CoI samples. In this process the lower and higher eigenvalued eigenvectors are rejected via statistical analysis because the high variance may split the target class itself, whereas the lower variance do not contribute significant information. Furthermore, the identified eigen knowledge grid is utilized to transform high dimensional samples to the lower dimensional eigen subspace. The proposed approach is named as learning target class eigen subspace (LTS-ES) that ensures strong separation of the target class from other classes. To show the effectiveness of transformed lower dimensional eigen subspace, one class support vector machine (OCSVM) has been experimented on wide variety of benchmark datasets in presence of: original feature space, transformed features obtained via eigenvectors of approximately 80%-90% cumulative variance, transformed features obtained via knowledge grid and transformed features obtained via eigenvectors of approximately 50% cumulative variance. Finally, a new performance measure parameter called stability factor is introduced to validate the robustness of the proposed approach.

Density-Based Geometric One-Class Classifier Combined with Genetic Algorithm

One of the most prospective issues in recent machine learning research is one-class classification (OCC), which considers datasets composed of only one class and outlier. It is more reasonable than the traditional multiclass classification in dealing with problematic datasets or special cases. Generally, classification accuracy and interpretability for users are considered to have a trade-off in OCC methods. A classifier based on hyperrectangle (H-RTGL) can alleviate such a trade-off and uses H-RTGL formulated by the conjunction of geometric rules (called an interval). This interval can form a basis for interpretability since it can be easily understood by the user. However, the existing H-RTGL-based OCC classifiers have the following limitations: (i) they cannot reflect the density of the target class, (ii) the density is considered using a primitive interval generation method, and (iii) there exists no systematic procedure for determining the hyperparameter of the H-RTGL-based OCC classifier, which influences its classification performance. Therefore, we suggest a one-class hyperrectangle descriptor based on density 1 − H R D d with a more elaborate interval generation method, including parametric and nonparametric approaches. Specifically, we design a genetic algorithm that comprises a chromosome structure and genetic operators for systematic generation of 1 − H R D d through optimization of the hyperparameter. Our study is validated through a numerical experiment using several actual datasets with different sizes and features, and the result is compared to the existing OCC algorithms along with other H-RTGL-based classifiers.

Soft one-class extreme learning machine for turboshaft engine fault detection

One-class extreme learning machine (OC-ELM) is a very common one-class classification algorithm. In the process of constructing the OC-ELM, a part of training samples will be removed by the algorithm, which leads to inconsistency between its constraint condition and decision function, and is also known as the problem of hard margin flaw. In order to solve this problem, a soft one-class extreme learning machine (SOC-ELM) is proposed by assigning an appropriate margin to each training sample. Experimental results on benchmark data sets show that SOC-ELM has a strong classification performance. When the SOC-ELM is used for fault detection of a turboshaft engine, it can achieve good detection effectiveness and has strong robustness.

The impact of sensing parameters on data management and anomaly detection in structural health monitoring

The massive and autonomous structural health monitoring (SHM) of bridges is a problem that is of growing interest due to its importance and topicality. However, a considerable amount of data must be elaborated and managed in such an application. This paper proposes a set of machine learning (ML) tools to detect anomalies in a bridge from vibrational measurements using the minimum amount of data. The proposed framework starts from the fundamental frequencies extracted through operational modal analysis (OMA) and clustering, followed by a density-based time-domain tracking algorithm. The fundamental frequencies extracted are then fed to one-class classification (OCC) algorithms that perform anomaly detection. Then, to reduce the amount of data, we analyze the effect of the number of sensors, the number of bits per sample, the observation time, and the measurement noise on damage detection performance. As a case study, the Z-24 bridge is considered because of the extensive database of accelerometric measurements in both standard and damaged conditions. A comparison of OCC algorithms, such as principal component analysis (PCA), kernel principal component analysis (KPCA), Gaussian mixture model (GMM) and one-class classifier neural network (OCCNN)^2 is performed, and their robustness to data shrinking is evaluated. In many cases, OCCNN^2 increases the performance with respect to classical anomaly detection techniques in terms of accuracy.

One-Class Classification Models for the Authentication of Analgesic Tablet Reference Medicine Using Differential Scanning Calorimetry and Visible-Near Infrared Spectroscopy

The aim of this study was to develop a fast, simple and accurate analytical method for the classification of reference tablet analgesic drugs containing dipyrone, orphenadrine and caffeine using differential scanning calorimetry (DSC) and visible-near infrared spectroscopy (VNIRS) associated with one-class chemometric classification algorithm. The training set is based on reference medicine with 15 samples as target class. Three different brands with 10 samples each and five reference medicine samples, obtaining 35 samples, were used as the test set. Chemometric models based on principal component analysis (PCA) and data-driven soft independent modelling of class analogy (DD-SIMCA) were used to obtain the results. Two DD-SIMCA models obtained 100% sensitivity, specificity, and accuracy using DSC and VNIRS, both with a significance level of 0.01. This method using one-class classification as a chemometric tool proved to be a good alternative for quality control of pharmaceutical samples.

A Leaf Disease Detection Mechanism Based on L1-Norm Minimization Extreme Learning Machine

The disease-free growth of a plant is highly influential for both environment and human life, as numerous microorganisms/viruses/fungus may affect the growth and agricultural production of a plant. Early detection and treatment thus becomes necessary and must be treated on time. The existing vision techniques either involve image segmentation or feature classification/regression applied over aerial images. This results in an increase in time and cost consumption due to various challenges, such as generalization ability and learning cost. Therefore, a feature-based disease detection approach with minimal learning time and generalization ability could be fairly befitting such as an extreme learning machine (ELM). In this letter, we demonstrate an algorithm, L1-ELM, after employing Kuan filtering for preprocessing and different feature computations. At the evaluation stage, the experimentation performed over benchmark plant datasets confirms that L1-ELM outperforms all existing one-class classification algorithms, preserving optimal learning and better generalization.

Unsupervised online anomaly detection in Software Defined Network environments

Software Defined Networking (SDN) simplifies network management and significantly reduces operational costs. SDN removes the control plane from forwarding devices (e.g., routers and switches) and centralizes this plane in a controller, enabling the management of the network forwarding decisions by programming the control plane with a high-level language. However, its centralized architecture may be compromised by flooding attacks, such as Distributed Denial of Service (DDoS) and portscan. Facing this challenge, we propose an Intrusion Detection System (IDS) based on online clustering to detect attacks in an evolving SDN network taking advantage of the entropy of source and destination IP addresses and ports. Our proposal is focused on avoiding the demand for labeling and previous knowledge to provide a practical and accurate method to address real-life online scenarios. Moreover, our proposal paves the way for a comprehensive analysis by projecting the cluster’s structure over the feature space, providing insights on intensity, seasonality, and attack type. Our experiments were carried out with the DenStream algorithm in several databases attacked by DDoS and portscan with different intensities, durations, and overlapping patterns. When comparing DenStream performance to Half-Space-Trees, an accurate online one-class classification algorithm for anomaly detection, it was possible to expose the capacity of our unsupervised proposal, overcoming the one-class solution, and reaching f-measure rates above 99.60%.

Learning to Classify DWDM Optical Channels from Tiny and Imbalanced Data.

Applying machine learning algorithms for assessing the transmission quality in optical networks is associated with substantial challenges. Datasets that could provide training instances tend to be small and heavily imbalanced. This requires applying imbalanced compensation techniques when using binary classification algorithms, but it also makes one-class classification, learning only from instances of the majority class, a noteworthy alternative. This work examines the utility of both these approaches using a real dataset from a Dense Wavelength Division Multiplexing network operator, gathered through the network control plane. The dataset is indeed of a very small size and contains very few examples of “bad” paths that do not deliver the required level of transmission quality. Two binary classification algorithms, random forest and extreme gradient boosting, are used in combination with two imbalance handling methods, instance weighting and synthetic minority class instance generation. Their predictive performance is compared with that of four one-class classification algorithms: One-class SVM, one-class naive Bayes classifier, isolation forest, and maximum entropy modeling. The one-class approach turns out to be clearly superior, particularly with respect to the level of classification precision, making it possible to obtain more practically useful models.