One-class Classification Problem Research Articles

Rail Suspension System Fault Detection using Deep Semi-Supervised Feature Extraction with One-class data

In this paper we propose a novel semi-supervised fault detection methodology for a vehicle suspension system with one-class multi-sensor data. Supervised data-driven methods have been applied in fault detection successfully in recent studies. However, it is difficult and expensive to collect data under faulty condition for supervised learning while data collection under normal condition is much easier and cheaper. Fault detection under such situation is a one-class classification problem that requires classification models to identify the positive class when the negative class is either absent or poorly sampled. The efficiency of classifiers is constrained by defining the normal class boundary only with the knowledge of positive class as well as the problem of biased or incorrect feature extracted from the positive class. In proposed method, A deep semi-supervised learning method integrated with physical-based domain knowledge is applied for feature extraction. The suspension system for a full car model is modeled using a simulation tool, SIMPACK to generate the synthetic multi-sensor data. Our results show the effectiveness of the proposed in fault detection and diagnostics with one-class data.

Relationship between Variants of One-Class Nearest Neighbors and Creating Their Accurate Ensembles

In one-class classification problems, only the data for the target class is available, whereas the data for the non-target class may be completely absent. In this paper, we study one-class nearest neighbor (OCNN) classifiers and their different variants. We present a theoretical analysis to show the relationships among different variants of OCNN that may use different neighbors or thresholds to identify unseen examples of the non-target class. We also present a method based on inter-quartile range for optimizing parameters used in OCNN in the absence of non-target data during training. Then, we propose two ensemble approaches based on random subspace and random projection methods to create accurate OCNN ensembles. We tested the proposed methods on 15 benchmark and real world domain-specific datasets and show that random-projection ensembles of OCNN perform best.

A novel method for binarization of scene text images and its application in text identification

The aim of this article is twofold. First, we propose an effective methodology for binarization of scene images. For our present study, we use the publicly available ICDAR 2011 Born Digital Data set. We introduce a new concept of variance map of a gray-level image for detection of text boundary in an image. Based on this boundary information, the image is binarized by means of adaptive thresholding. This binarization procedure produces a number of connected components. Next, these connected components are examined in order to identify possible text components. In this context, a number of shape-based features that distinguish between text and non-text components are proposed. We consider text component identification as an one-class classification problem, i.e., the ground truth information for only the text class is available for the ICDAR 2011 Born Digital Data set. Then, the ground truth text components are used to obtain a certain statistical distribution of the shape-based features. Here, we observe that all the features may not follow a single family of distributions. Therefore, we construct a joint distribution by using multivariate Gaussian copula which allows a coupling of different marginal distributions. As our experiments suggest, the copula-based method is superior to multivariate Gaussian distribution in describing the feature distribution. Finally, a text connected component of an unknown class is subjected to the trained statistical model, and by performing a hypothesis test we successfully identify a possible text component. For a comparative study, we consider a number of state-of-the-art methods. Our proposed approach significantly outperforms most of these methods in terms of recall, precision and F-measure in both the binarization and text identification tasks.

Infinite Bayesian one-class support vector machine based on Dirichlet process mixture clustering

In the problem of one-class classification, one-class classifier (OCC) tries to identify objects of a specific class, called the target class, among all objects, by learning from a training set containing only the objects of target class. However, some traditional OCCs lose their effectiveness when the distribution of target class is a multimodal distribution, and the performance is sensitive to the model parameters. To solve this problem and enhance classification performance, a novel OCC infinite Bayesian OCSVM (IB-OCSVM) is proposed in this study. In IB-OCSVM, we partition the input data into several clusters with the Dirichlet process mixture (DPM), and learn a modified OCSVM in each cluster. Specifically, the clustering procedure and the modified OCSVM are jointly learned in a unified Bayesian frame to guarantee the consistency of clustering and linear separability in each cluster. The parameters can be inferred simply and effectively via Gibbs sampling technique. Meanwhile, we modify the traditional OCSVM by replacing the kernel transformation with a new feature transformation based on the result of clustering via DPM. In the feature space of the modified OCSVM, all transformed samples preserve the same clustering characteristic as the original samples, thus guaranteeing the combination of clustering and classification in our method. Experimental results based on benchmark datasets and synthetic aperture radar real data demonstrate that the proposed method has enhanced classification performance and is more robust to the model parameters than some conventional methods.

One-class remote sensing classification: one-class vs. binary classifiers

ABSTRACTMany applications of remote sensing only require the classification of a single land type. This is known as the one-class classification problem and it can be performed using either binary classifiers, by treating all other classes as the negative class, or one-class classifiers which only consider the class of interest. The key difference between these two approaches is in their training data and the amount of effort needed to produce it. Binary classifiers require an exhaustively labelled training data set while one-class classifiers are trained using samples of just the class of interest. Given ample and complete training data, binary classifiers generally outperform one-class classifiers. However, what is not clear is which approach is more accurate when given the same amount of labelled training data. That is, for a fixed labelling effort, is it better to use a binary or one-class classifier. This is the question we consider in this article. We compare several binary classifiers, including backpropagation neural networks, support vector machines, and maximum likelihood classifiers, with two one-class classifiers, one-class SVM, and presence and background learning (PBL), on the problem of one-class classification in high-resolution remote sensing imagery. We show that, given a fixed labelling budget, PBL consistently outperforms the other methods. This advantage stems from the fact that PBL is a positive-unlabelled method in which large amounts of readily available unlabelled data is incorporated into the training phase, allowing the classifier to model the negative class more effectively.

Fault detection in operating helicopter drivetrain components based on support vector data description

The objective of the paper is to develop a vibration-based automated procedure dealing with early detection of mechanical degradation of helicopter drive train components using Health and Usage Monitoring Systems (HUMS) data. An anomaly-detection method devoted to the quantification of the degree of deviation of the mechanical state of a component from its nominal condition is developed. This method is based on an Anomaly Score (AS) formed by a combination of a set of statistical features correlated with specific damages, also known as Condition Indicators (CI), thus the operational variability is implicitly included in the model through the CI correlation. The problem of fault detection is then recast as a one-class classification problem in the space spanned by a set of CI, with the aim of a global differentiation between normal and anomalous observations, respectively related to healthy and supposedly faulty components. In this paper, a procedure based on an efficient one-class classification method that does not require any assumption on the data distribution, is used. The core of such an approach is the Support Vector Data Description (SVDD), that allows an efficient data description without the need of a significant amount of statistical data. Several analyses have been carried out in order to validate the proposed procedure, using flight vibration data collected from a H135, formerly known as EC135, servicing helicopter, for which micro-pitting damage on a gear was detected by HUMS and assessed through visual inspection. The capability of the proposed approach of providing better trade-off between false alarm rates and missed detection rates with respect to individual CI and to the AS obtained assuming jointly-Gaussian-distributed CI has been also analysed.

New fast feature selection methods based on multiple support vector data description

Feature selection can sort out useful features to obtain good performance when dealing with high-dimensional data. Feature selection methods based on support vector data description (SVDD) have been proposed for one-class classification problems: SVDD-radius-recursive feature elimination (SVDD-RRFE) and SVDD-dual-objective-recursive feature elimination (SVDD-DRFE). However, both SVDD-RRFE and SVDD-DRFE use only one-class samples even given a multi-class classification task, and suffer from high computational complexity. To remedy it, this paper extends both SVDD-RRFE and SVDD-DRFE to binary and multi-class classification problems using multiple SVDD models, and proposes fast feature ranking schemes for them in the case of the linear kernel. Experimental results on toy, UCI and microarray datasets show the efficiency and the feasibility of the proposed methods.

Semi-supervised subclass support vector data description for image and video classification

In this paper, the Semi-Supervised Subclass Support Vector Data Description is presented, a method that operates in both the supervised and the semi-supervised One-class classification case. The proposed method consists a novel extension of the standard SVDD method, by introducing two additional terms its optimization problem. These two terms correspond to expressing global and local geometric data information respectively, during the classifier optimization process. Global geometric data information is employed by minimizing the global target class variance, assuming that subclasses may have been formed within as well. In addition, by exploiting the semi-supervised learning smoothness assumption, local neighborhood information between all available (labeled and unlabeled) data is preserved, even in the supervised learning case. We show that the adoption of both terms results in a regularized feature space, where low variance directions have been emphasized, while local geometric data information have been preserved. The proposed method has been evaluated in classification problems related to face recognition, human action recognition and generic One-class classification problems, comparing favorably against related One-class classification methods in both the semi-supervised and the supervised learning cases.

One-class support higher order tensor machine classifier

One-class classification problems have been widely encountered in the fields that the negative class patterns are difficult to be collected, and the one-class support vector machine is one of the popular algorithms for solving them. However, one-class support vector machine is a vector-based learning algorithm, and it cannot work directly when the input pattern is a tensor. This paper proposes a tensor-based maximum margin classifier for one-class classification problems, and develops a One-Class Support Higher Order Tensor Machine (HO-OCSTM) which can separate most of the target patterns from the origin with the maximum margin in the higher order tensor space. HO-OCSTM directly employs the higher order tensors as the input patterns, and it is more proper for small sample study. Moreover, the direct use of tensor representation has the advantage of retaining the structural information of data, which helps improve the generalization ability of the proposed algorithm. We implement HO-OCSTM by the alternating projection method and solve a convex quadratic programming similar to the standard one-class support vector machine algorithm at each iteration. The experimental results have shown the high recognition accuracy of the proposed method.

One-Class Classification Framework Based on Shrinkage Methods

Statistical machine learning, such as kernel methods, have been widely used to discover hidden regularities and patterns in data. In particular, one-class classification algorithms gained a lot of interest in a large number of applications where the only available data designate a unique class, as in industrial processes. In this paper, we propose a sparse framework for one-class classification problems, by investigating the hypersphere enclosing the samples in a Reproducing Kernel Hilbert Space (RKHS). The center of this hypersphere is approximated using a sparse solution, by selecting an appropriate set of relevant samples. For this purpose, we investigate well-known shrinkage methods, namely Least Angle Regression, Least Absolute Shrinkage and Selection Operator, and Elastic Net. We revisit these methods and adapt their algorithms for estimating the sparse center in the RKHS. The proposed framework is extended to include the truncated Mahalanobis distance, which is necessary when dealing with heterogenous input variables. We also provide some theoretical results on the projection error and on the error of the first kind. The proposed algorithms are compared with well-known one-class classification approaches, with experiments conducted on simulated and real datasets.

An innovative one-class least squares support vector machine model based on continuous cognition

One-class classification is a basic problem in machine learning. Unlike the existing typical one-class classifiers designed from the angle of probability or geometric, this paper attempts to study this problem from the bionics point of view. Using the continuous cognition characteristic as the starting point, we propose a new framework of one-class classifier, named multiple regression model (OC-MR), which can be seen as a natural extension of multiple regression for one-class classification problem. This paper applies least squares support vector machine (LSSVM) as an example to show themodeling process of the proposed method and the corresponding one-class classifier is named one-class least squares support vector machine (OC-LSSVM). Various simulation and real-life datasets are used to test the performance of the proposed OC-LSSVM. The existing popular one-class classification methods including Parzen kernel density estimation, support vector data description and Gaussian mixture model are also applied in order to achieve a comprehensive comparison. The results show that OC-LSSVM has achieved the best performance in most of the simulation and real-life datasets due to its good robustness, which highlights the efficacy of OC-LSSVM.

Online personal risk detection based on behavioural and physiological patterns

We define personal risk detection as the timely identification of when someone is in the midst of a dangerous situation, for example, a health crisis or a car accident, events that may jeopardize a person’s physical integrity. We work under the hypothesis that a risk-prone situation produces sudden and significant deviations in standard physiological and behavioural user patterns. These changes can be captured by a group of sensors, such as the accelerometer, gyroscope, and heart rate. We introduce a dataset, called PRIDE, which provides a baseline for the development and the fair comparison of personal risk detection mechanisms. PRIDE contains information on 18 test subjects; for each subject, it includes partial information about the user’s behavioural and physiological patterns, as captured by Microsoft Band©. PRIDE test subject records include sensor readings of not only when a subject is carrying out ordinary daily life activities, but also when exposed to a stressful scenario, thereby simulating a dangerous or abnormal situation. We show how to use PRIDE to develop a personal risk detection mechanism; to accomplish this, we have tackled risk detection as a one-class classification problem. We have trained several classifiers based only on the daily behaviour of test subjects. Further, we tested the accuracy of the classifiers to detect anomalies that were not included in the training process of the classifiers. We used a number of one-class classifiers, namely: SVM, Parzen, and two versions of Parzen based on k-means. While there is still room for improvement, our results are encouraging: they support our hypothesis that abnormal behaviour can be automatically detected.

Selective ensemble of SVDDs with Renyi entropy based diversity measure

In this paper, a novel selective ensemble strategy for support vector data description (SVDD) using the Renyi entropy based diversity measure is proposed to deal with the problem of one-class classification. In order to obtain compact classification boundary, the radius of ensemble is defined as the inner product of the vector of combination weights and the vector of the radii of SVDDs. To make the center of ensemble achieve the optimal position, the Renyi entropy of the kernelized distances between the images of samples and the center of ensemble in the high-dimensional feature space is defined as the diversity measure. Moreover, to fulfill the selective ensemble, an ℓ1-norm based regularization term is introduced into the objective function of the proposed ensemble. The optimal combination weights can be iteratively obtained by the half-quadratic optimization technique. Experimental results on two synthetic data sets and twenty benchmark data sets demonstrate that the proposed selective ensemble method is superior to the single SVDD and the other four related ensemble approaches.

Bagging-TPMiner: a classifier ensemble for masquerader detection based on typical objects

The goal of a masquerade detection system is to determine whether a given computer activity does not correspond to a target user, thereby inferring that a masquerader has stolen the computer session of a user. Masquerade detection should be addressed as a one-class classification problem, where only user information is available for classifier construction. This might be mandatory when it is difficult to account for all types of attack patterns or collect enough evidence thereof. In this paper, we introduce a masquerader detection method, named Bagging-TPMiner, a one-class classifier ensemble. As the name suggests, Bagging-TPMiner bootstraps the training dataset of genuine user behavior in order to find typical objects. In the classification phase, it renders a new sample of computer behavior to be a masquerade if that behavior is distinct from the typical objects. Critically, unlike existing clustering techniques, Bagging-TPMiner gives similar attention to both types of regions, dense and sparse, thus capturing the (hidden) structure of ordinary user behavior. We have successfully tested Bagging-TPMiner on WUIL, a repository of datasets for masquerader detection that contain more faithful masquerade attempts. Our experimental results show that Bagging-TPMiner improves classification accuracy when compared to other classifiers and that it is significantly better at identifying bursts of attacks, called persistent attacks, or at continuously updating from prior mistakes.

One-class classification based on the convex hull for bearing fault detection

Originating from a nearest point problem, a novel method called one-class classification based on the convex hull (OCCCH) is proposed for one-class classification problems. The basic goal of OCCCH is to find the nearest point to the origin from the reduced convex hull of training samples. A generalized Gilbert algorithm is proposed to solve the nearest point problem. It is a geometric algorithm with high computational efficiency. OCCCH has two different forms, i.e., OCCCH-1 and OCCCH-2. The relationships among OCCCH-1, OCCCH-2 and one-class support vector machine (OCSVM) are investigated theoretically. The classification accuracy and the computational efficiency of the three methods are compared through the experiments conducted on several benchmark datasets. Experimental results show that OCCCH (including OCCCH-1 and OCCCH-2) using the generalized Gilbert algorithm performs more efficiently than OCSVM using the well-known sequential minimal optimization (SMO) algorithm; at the same time, OCCCH-2 can always obtain comparable classification accuracies to OCSVM. Finally, these methods are applied to the monitoring model constructions for bearing fault detection. Compared with OCCCH-2 and OCSVM, OCCCH-1 can significantly decrease the false alarm ratio while detecting the bearing fault successfully.

One-Class Support Tensor Machine

In fault diagnosis, face recognition, network anomaly detection, text classification and many other fields, we often encounter one-class classification problems. The traditional vector-based one-class classification algorithms represented by One-Class Support Vector Machine (OCSVM) have limitations when tensor is considered as input data. This work addresses one-class classification problem with tensor-based maximal margin classification paradigm. To this end, we formulate the One-Class Support Tensor Machine (OCSTM), which separates most samples of interested class from the origin in the tensor space, with maximal margin. The benefits of the proposed algorithm are twofold. First, the use of direct tensor representation helps to retain the data topology more efficiently. The second benefit is that tensor representation can greatly reduce the number of parameters. It helps overcome the overfitting problem caused mostly by vector-based algorithms and especially suits for high dimensional and small sample size problem. To solve the corresponding optimization problem in OCSTM, the alternating projection method is implemented, for it is simplified by solving a typical OCSVM optimization problem at each iteration. The efficiency of the proposed method is illustrated on both vector and tensor datasets. The experimental results indicate the validity of the new method.

Hyper-Rectangles를 이용한 단일 분류기 설계

Recently, the importance of one-class classification problem is more increasing. However, most of existing algorithms have the limitation on providing the information that effects on the prediction of the target value. Motivated by this remark, in this paper, we suggest an efficient one-class classifier using hyper-rectangles (H-RTGLs) that can be produced from intervals including observations. Specifically, we generate intervals for each feature and integrate them. For generating intervals, we consider two approaches : (i) interval merging and (ii) clustering. We evaluate the performance of the suggested methods by computing classification accuracy using area under the roc curve and compare them with other one-class classification algorithms using four datasets from UCI repository. Since H-RTGLs constructed for a given data set enable classification factors to be visible, we can discern which features effect on the classification result and extract patterns that a data set originally has.

Entropic One-Class Classifiers.

The one-class classification problem is a well-known research endeavor in pattern recognition. The problem is also known under different names, such as outlier and novelty/anomaly detection. The core of the problem consists in modeling and recognizing patterns belonging only to a so-called target class. All other patterns are termed nontarget, and therefore, they should be recognized as such. In this paper, we propose a novel one-class classification system that is based on an interplay of different techniques. Primarily, we follow a dissimilarity representation-based approach; we embed the input data into the dissimilarity space (DS) by means of an appropriate parametric dissimilarity measure. This step allows us to process virtually any type of data. The dissimilarity vectors are then represented by weighted Euclidean graphs, which we use to determine the entropy of the data distribution in the DS and at the same time to derive effective decision regions that are modeled as clusters of vertices. Since the dissimilarity measure for the input data is parametric, we optimize its parameters by means of a global optimization scheme, which considers both mesoscopic and structural characteristics of the data represented through the graphs. The proposed one-class classifier is designed to provide both hard (Boolean) and soft decisions about the recognition of test patterns, allowing an accurate description of the classification process. We evaluate the performance of the system on different benchmarking data sets, containing either feature-based or structured patterns. Experimental results demonstrate the effectiveness of the proposed technique.

BeeOWA: A novel approach based on ABC algorithm and induced OWA operators for constructing one-class classifier ensembles

In recent years, classifier ensembles have received increasing attention in the machine learning and pattern recognition communities. However, constructing classifier ensembles for one-class classification problems has still remained as a challenging research topic. To pursue this line of research, we need to address issues on how to generate a set of diverse one-class classifiers that are individually accurate and how to combine the outputs of them in an effective way. In this paper, we present BeeOWA, a novel approach to construct highly accurate one-class classifier ensembles. It uses a novel binary artificial bee colony algorithm, called BeePruner, to prune an initial one-class classifier ensemble and find a near-optimal sub-ensemble of base classifiers in a reasonable computational time. To evaluate the fitness of an ensemble solution, BeePruner uses two different measures: an exponential consistency measure and a non-pairwise diversity measure based on the Kappa inter-rater agreement. After one-class classifier pruning, BeeOWA uses a novel exponential induced OWA (ordered weighted averaging) operator, called EIOWA, to combine the outputs of base classifiers in the sub-ensemble. The results of experiments carried out on a number of benchmark datasets show that BeeOWA can outperform several state-of-the-art approaches, both in terms of classification performance and statistical significance.

Deep learning with support vector data description

One of the most critical problems for machine learning methods is overfitting. The overfitting problem is a phenomenon in which the accuracy of the model on unseen data is poor whereas the training accuracy is nearly perfect. This problem is particularly severe in complex models that have a large set of parameters. In this paper, we propose a deep learning neural network model that adopts the support vector data description (SVDD). The SVDD is a variant of the support vector machine, which has high generalization performance by acquiring a maximal margin in one-class classification problems. The proposed model strives to obtain the representational power of deep learning. Generalization performance is maintained using the SVDD. The experimental results showed that the proposed model can learn multiclass data without severe overfitting problems.