One-class Classification Method Research Articles

Hyperparameter selection of one-class support vector machine by self-adaptive data shifting

With flexible data description ability, one-class Support Vector Machine (OCSVM) is one of the most popular and widely-used methods for one-class classification (OCC). Nevertheless, the performance of OCSVM strongly relies on its hyperparameter selection, which is still a challenging open problem due to the absence of outlier data. This paper proposes a fully automatic OCSVM hyperparameter selection method, which requires no tuning of additional hyperparameter, based on a novel self-adaptive “data shifting” mechanism: Firstly, by efficient edge pattern detection (EPD) and “negatively” shifting edge patterns along the negative direction of estimated data density gradient, a constrained number of high-quality pseudo outliers are self-adaptively generated at more desirable locations, which readily avoids two major difficulties in previous outlier generation methods. Secondly, to avoid time-consuming cross-validation and enhance robustness to noise in the given training data, a pseudo target set is generated for model validation by “positively” shifting each given target datum along the positive direction of data density gradient. Experiments on synthetic and benchmark datasets demonstrate the effectiveness of the proposed method.

Semi-supervised subclass support vector data description for image and video classification

In this paper, the Semi-Supervised Subclass Support Vector Data Description is presented, a method that operates in both the supervised and the semi-supervised One-class classification case. The proposed method consists a novel extension of the standard SVDD method, by introducing two additional terms its optimization problem. These two terms correspond to expressing global and local geometric data information respectively, during the classifier optimization process. Global geometric data information is employed by minimizing the global target class variance, assuming that subclasses may have been formed within as well. In addition, by exploiting the semi-supervised learning smoothness assumption, local neighborhood information between all available (labeled and unlabeled) data is preserved, even in the supervised learning case. We show that the adoption of both terms results in a regularized feature space, where low variance directions have been emphasized, while local geometric data information have been preserved. The proposed method has been evaluated in classification problems related to face recognition, human action recognition and generic One-class classification problems, comparing favorably against related One-class classification methods in both the semi-supervised and the supervised learning cases.

Mapping Regional Urban Extent Using NPP-VIIRS DNB and MODIS NDVI Data

The accurate and timely monitoring of regional urban extent is helpful for analyzing urban sprawl and studying environmental issues related to urbanization. This paper proposes a classification scheme for large-scale urban extent mapping by combining the Day/Night Band of the Visible Infrared Imaging Radiometer Suite on the Suomi National Polar-orbiting Partnership Satellite (NPP-VIIRS DNB) and the Normalized Difference Vegetation Index from the Moderate Resolution Imaging Spectroradiometer products (MODIS NDVI). A Back Propagation (BP) neural network based one-class classification method, the Present-Unlabeled Learning (PUL) algorithm, is employed to classify images into urban and non-urban areas. Experiments are conducted in mainland China (excluding surrounding islands) to detect urban areas in 2012. Results show that the proposed model can successfully map urban area with a kappa of 0.842 on the pixel level. Most of the urban areas are identified with a producer’s accuracy of 79.63%, and only 10.42% the generated urban areas are misclassified with a user’s accuracy of 89.58%. At the city level, among 647 cities, only four county-level cities are omitted. To evaluate the effectiveness of the proposed scheme, three contrastive analyses are conducted: (1) comparing the urban map obtained in this paper with that generated by the Defense Meteorological Satellite Program/Operational Linescan System Nighttime Light Data (DMSP/OLS NLD) and MODIS NDVI and with that extracted from MCD12Q1 in MODIS products; (2) comparing the performance of the integration of NPP-VIIRS DNB and MODIS NDVI with single input data; and (3) comparing the classification method used in this paper (PUL) with a linear method (Large-scale Impervious Surface Index (LISI)). According to our analyses, the proposed classification scheme shows great potential to map regional urban extents in an effective and efficient manner.

Detecting unseen falls from wearable devices using channel-wise ensemble of autoencoders

A fall is an abnormal activity that occurs rarely, so it is hard to collect real data for falls. It is, therefore, difficult to use supervised learning methods to automatically detect falls. Another challenge in using machine learning methods to automatically detect falls is the choice of engineered features. In this paper, we propose to use an ensemble of autoencoders to extract features from different channels of wearable sensor data trained only on normal activities. We show that the traditional approach of choosing a threshold as the maximum of the reconstruction error on the training normal data is not the right way to identify unseen falls. We propose two methods for automatic tightening of reconstruction error from only the normal activities for better identification of unseen falls. We present our results on two activity recognition datasets and show the efficacy of our proposed method against traditional autoencoder models and two standard one-class classification methods.

MST-GEN: An Efficient Parameter Selection Method for One-Class Extreme Learning Machine.

One-class classification (OCC) models a set of target data from one class to detect outliers. OCC approaches like one-class support vector machine (OCSVM) and support vector data description (SVDD) have wide practical applications. Recently, one-class extreme learning machine (OCELM), which inherits the fast learning speed of original ELM and achieves equivalent or higher data description performance than OCSVM and SVDD, is proposed as a promising alternative. However, OCELM faces the same thorny parameter selection problem as OCSVM and SVDD. It significantly affects the performance of OCELM and remains under-explored. This paper proposes minimal spanning tree (MST)-GEN, an automatic way to select proper parameters for OCELM. Specifically, we first build a n -round MST to model the structure and distribution of the given target set. With information from n -round MST, a controllable number of pseudo outliers are generated by edge pattern detection and a novel "repelling" process, which readily overcomes two fundamental problems in previous outlier generation methods: where and how many pseudo outliers should be generated. Unlike previous methods that only generate pseudo outliers, we further exploit n -round MST to generate pseudo target data, so as to avoid the time-consuming cross-validation process and accelerate the parameter selection. Extensive experiments on various datasets suggest that the proposed method can select parameters for OCELM in a highly efficient and accurate manner when compared with existing methods, which enables OCELM to achieve better OCC performance in OCC applications. Furthermore, our experiments show that MST-GEN can also be favorably applied to other prevalent OCC methods like OCSVM and SVDD.

Gompertz binary particle swarm optimization and support vector data description system for fault detection and feature selection applied in automotive pedals components

This work presents an improved fault detection by reference space optimization and simultaneous feature selection applied in a manufacturing complex process of automotive pedals components. Support vector data description (SVDD) one-class classification method uses a hypersphere with the minimum volume to find an enclosed boundary containing almost all target objects. Gompertz binary particle swarm optimization algorithm (GBPSO) is applied to optimize kernel hyperparameters for SVDD and simultaneously solve the feature selection problem. In order to justify and validate the results, also the genetic algorithm (GA) and binary particle swarm optimization algorithm (BPSO) are presented to compare the performances of the three approaches in terms of the misclassification function. The experimental results showed that the proposed approach can correctly select the influencing input variables in order to achieve an efficient fault detection.

An innovative one-class least squares support vector machine model based on continuous cognition

One-class classification is a basic problem in machine learning. Unlike the existing typical one-class classifiers designed from the angle of probability or geometric, this paper attempts to study this problem from the bionics point of view. Using the continuous cognition characteristic as the starting point, we propose a new framework of one-class classifier, named multiple regression model (OC-MR), which can be seen as a natural extension of multiple regression for one-class classification problem. This paper applies least squares support vector machine (LSSVM) as an example to show themodeling process of the proposed method and the corresponding one-class classifier is named one-class least squares support vector machine (OC-LSSVM). Various simulation and real-life datasets are used to test the performance of the proposed OC-LSSVM. The existing popular one-class classification methods including Parzen kernel density estimation, support vector data description and Gaussian mixture model are also applied in order to achieve a comprehensive comparison. The results show that OC-LSSVM has achieved the best performance in most of the simulation and real-life datasets due to its good robustness, which highlights the efficacy of OC-LSSVM.

On the construction of extreme learning machine for online and offline one-class classification—An expanded toolbox

One-class classification (OCC) has been prime concern for researchers and effectively employed in various disciplines. But, traditional methods based one-class classifiers are very time consuming due to its iterative process and various parameters tuning. In this paper, we present six OCC methods and their thirteen variants based on extreme learning machine (ELM) and online sequential ELM (OSELM). Our proposed classifiers mainly lie in two categories: reconstruction based and boundary based, where three proposed classifiers belong to reconstruction based and three belong to boundary based. We are presenting both types of learning viz., online and offline learning for OCC. Out of six methods, four are offline and remaining two are online methods. Out of four offline methods, two methods perform random feature mapping and two methods perform kernel feature mapping. We present a comprehensive discussion on these methods and their comparison to each other. Kernel feature mapping based approaches have been tested with RBF kernel and online version of one-class classifiers is tested with both types of nodes viz., additive and RBF. It is well known fact that threshold decision is a crucial factor in case of OCC, so, three different threshold deciding criteria have been employed so far and analyze the effectiveness of one threshold deciding criteria over another. Further, these methods are tested on two artificial datasets to check their boundary construction capability and on eight benchmark datasets from different discipline to evaluate the performance of the classifiers. Our proposed classifiers exhibit better performance compared to ten traditional one-class classifiers and ELM based two one-class classifiers. Through proposed one-class classifiers, we intend to expand the functionality of the most used toolbox for OCC i.e. DD toolbox. All of our methods are totally compatible with all the present features of the toolbox.

Special Issue on INFORMS 2015 Annual Meeting

Special Issue on INFORMS 2015 Annual Meeting

Risk prediction for cardiovascular disease using ECG data in the China kadoorie biobank.

We set out to use machine learning techniques to analyse ECG data to improve risk evaluation of cardiovascular disease in a very large cohort study of the Chinese population. We performed this investigation by (i) detecting "abnormality" using 3 one-class classification methods, and (ii) predicting probabilities of "normality", arrhythmia, ischemia, and hypertrophy using a multiclass approach. For one-class classification, we considered 5 possible definitions for "normality" and used 10 automatically-extracted ECG features along with 4 blood pressure features. The one-class approach was able to identify abnormality with area-under-curve (AUC) 0.83, and with 75.6% accuracy. For four-class classification, we used 86 features in total, with 72 additional features extracted from the ECG. Accuracy for this four-class classifier reached 75.1%. The methods demonstrated proof-of-principle that cardiac abnormality can be detected using machine learning in a large cohort study.

One-Class Classification Based on Extreme Learning and Geometric Class Information

In this paper, we propose an extreme learning machine (ELM)-based one-class classification method that exploits geometric class information. We formulate the proposed method to exploit data representations in the feature space determined by the network hidden layer outputs, as well as in ELM spaces of arbitrary dimensions. We show that the exploitation of geometric class information enhances performance. We evaluate the proposed approach in publicly available datasets and compare its performance with the recently proposed one-class extreme learning machine algorithm, as well as with standard and recently proposed one-class classifiers. Experimental results show that the proposed method consistently outperforms the remaining approaches.

One-Class Support Vector Machines Identify the Language and Default Mode Regions As Common Patterns of Structural Alterations in Young Children with Autism Spectrum Disorders.

The identification of reliable brain endophenotypes of autism spectrum disorders (ASD) has been hampered to date by the heterogeneity in the neuroanatomical abnormalities detected in this condition. To handle the complexity of neuroimaging data and to convert brain images in informative biomarkers of pathology, multivariate analysis techniques based on Support Vector Machines (SVM) have been widely used in several disease conditions. They are usually trained to distinguish patients from healthy control subjects by making a binary classification. Here, we propose the use of the One-Class Classification (OCC) or Data Description method that, in contrast to two-class classification, is based on a description of one class of objects only. This approach, by defining a multivariate normative rule on one class of subjects, allows recognizing examples from a different category as outliers. We applied the OCC to 314 regional features extracted from brain structural Magnetic Resonance Imaging (MRI) scans of young children with ASD (21 males and 20 females) and control subjects (20 males and 20 females), matched on age [range: 22–72 months of age; mean = 49 months] and non-verbal intelligence quotient (NVIQ) [range: 31–123; mean = 73]. We demonstrated that a common pattern of features characterize the ASD population. The OCC SVM trained on the group of ASD subjects showed the following performances in the ASD vs. controls separation: the area under the receiver operating characteristic curve (AUC) was 0.74 for the male and 0.68 for the female population, respectively. Notably, the ASD vs. controls discrimination results were maximized when evaluated on the subsamples of subjects with NVIQ ≥ 70, leading to AUC = 0.81 for the male and AUC = 0.72 for the female populations, respectively. Language regions and regions from the default mode network—posterior cingulate cortex, pars opercularis and pars triangularis of the inferior frontal gyrus, and transverse temporal gyrus—contributed most to distinguishing individuals with ASD from controls, arguing for the crucial role of these areas in the ASD pathophysiology. The observed brain patterns associate preschoolers with ASD independently of their age, gender and NVIQ and therefore they are expected to constitute part of the ASD brain endophenotype.

One-Class Classification of Remote Sensing Images Using Kernel Sparse Representation

Sparse representations have been widely studied in remote sensing image analysis in recent years. In this paper, we develop a novel method for one-class classification (OCC) using a kernel sparse representation model for remotely sensed imagery. Training samples taken from the target class alone are used to build a learning dictionary for the sparse representation model, which is then optimized to produce a reconstruction residual. In the proposed model, a pixel is classified as the target class if the obtained reconstruction residual for the pixel is smaller than a given threshold; otherwise, the pixel is labeled as the outlier class. To improve the data separability between the target and outliner classes, the training samples taken from the target class are mapped into a high-dimensional feature space using a kernel function to build a learning dictionary for the kernel sparse representation model. OCC is then conducted in the mapped high-dimensional feature space using the reconstruction residual threshold, following the same principle as OCC in the original feature space. The proposed OCC method is evaluated and compared with several existing OCC methods in three different case studies. The experimental results indicate that the proposed method outperforms these existing methods, particularly when using a kernel sparse representation.

Advance and case studies of the DAICOX framework for automated design of multi-sensor intelligent measurement systems

Abstract In this paper, we report on the advance and application of our DAICOX system conceived for the automated and transparent design of integrated intelligent multi-sensor measurement applications. The current implementation allows the front to back design from raw sensory data to decision making. In particular, novelty or One-Class Classification (OCC) methods, improved assessment functionality, and automated design of feature computation by multi-objective optimization techniques to complete the processing chain complement the system. Two representative applications, i. e., basic food analysis and magnetic localization, have been selected as case studies with existing manual solutions to demonstrate and assess the applicability of the conceived architecture and its implementation. DAICOX reduced the design time by 98.62%, while increasing the recognition rate by 5.21% in the first case study and 0.71% in the second case study. The optimized solution can now be deployed to a run time system. In future work, extension of the method portfolio, self-x functionality, and public-domain availability are aspired.

Svclassify: a method to establish benchmark structural variant calls.

BackgroundThe human genome contains variants ranging in size from small single nucleotide polymorphisms (SNPs) to large structural variants (SVs). High-quality benchmark small variant calls for the pilot National Institute of Standards and Technology (NIST) Reference Material (NA12878) have been developed by the Genome in a Bottle Consortium, but no similar high-quality benchmark SV calls exist for this genome. Since SV callers output highly discordant results, we developed methods to combine multiple forms of evidence from multiple sequencing technologies to classify candidate SVs into likely true or false positives. Our method (svclassify) calculates annotations from one or more aligned bam files from many high-throughput sequencing technologies, and then builds a one-class model using these annotations to classify candidate SVs as likely true or false positives.ResultsWe first used pedigree analysis to develop a set of high-confidence breakpoint-resolved large deletions. We then used svclassify to cluster and classify these deletions as well as a set of high-confidence deletions from the 1000 Genomes Project and a set of breakpoint-resolved complex insertions from Spiral Genetics. We find that likely SVs cluster separately from likely non-SVs based on our annotations, and that the SVs cluster into different types of deletions. We then developed a supervised one-class classification method that uses a training set of random non-SV regions to determine whether candidate SVs have abnormal annotations different from most of the genome. To test this classification method, we use our pedigree-based breakpoint-resolved SVs, SVs validated by the 1000 Genomes Project, and assembly-based breakpoint-resolved insertions, along with semi-automated visualization using svviz.ConclusionsWe find that candidate SVs with high scores from multiple technologies have high concordance with PCR validation and an orthogonal consensus method MetaSV (99.7 % concordant), and candidate SVs with low scores are questionable. We distribute a set of 2676 high-confidence deletions and 68 high-confidence insertions with high svclassify scores from these call sets for benchmarking SV callers. We expect these methods to be particularly useful for establishing high-confidence SV calls for benchmark samples that have been characterized by multiple technologies.Electronic supplementary materialThe online version of this article (doi:10.1186/s12864-016-2366-2) contains supplementary material, which is available to authorized users.

System Informatics: From Methodology to Applications

This installment of Trends a Controversies provides an array of perspectives on the latest research in system informatics. Kang Zhao, Yao Xie, and Kwok-Leung Tsui introduce the work in Informatics: From Methodology to Applications. On the methodology side, in Projection-Based Process Monitoring and Empirical Divergence, Wenpo Huang, Wei Jiang, Qingming Wei, and Yanting Li propose a framework of projection-based methods, and in One-Class Classification Methods for Process Monitoring and Diagnosis, Sugon Cho and Seoung Bum Kim discuss how a data analytics algorithm can be used as a control chart. On the application side, IoT-Enabled System Informatics for Service Decision Making, by Kaibo Liu and Jianjun Shi, reviews current trends and future opportunities for IoT, with a special focus on issues related to the big data collected by multiple sensors. Quantifying the Risk Level of Functional Chips in DRAM Wafers, by Young-Seon Jeong, Byunghoon Kimb, Seung Hoon Tong, In-Kap Chang, and Myong K. Jeong, not only identifies research challenges and opportunities for decision making with massive data in the process of semiconductor manufacturing but also quantifies the risk level of functional chips in DRAM wafers. Finally, Flight Operations Monitoring through Cluster Analysis: A Case Study, by Florent Charruaud and Lishuai Li, describes a new method called cluster-based anomaly detection to help airline safety experts monitor daily flights and detect anomalies.

Robust support vector data description for outlier detection with noise or uncertain data

As an example of one-class classification methods, support vector data description (SVDD) offers an opportunity to improve the performance of outlier detection and reduce the loss caused by outlier occurrence in many real-world applications. However, due to limited outliers, the SVDD model is built only by using the normal data. In this situation, SVDD may easily lead to over fitting when the normal data contain noise or uncertainty. This paper presents two types of new SVDD methods, named R-SVDD and εNR-SVDD, which are constructed by introducing cutoff distance-based local density of each data sample and the ε-insensitive loss function with negative samples. We have demonstrated that the proposed methods can improve the robustness of SVDD for data with noise or uncertainty by extensive experiments on ten UCI datasets. The experimental results have shown that the proposed εNR-SVDD is superior to other existing outlier detection methods in terms of the detection rate and the false alarm rate. Meanwhile, the proposed R-SVDD can also achieve a better outlier detection performance with only normal data. Finally, the proposed methods are successfully used to detect the image-based conveyor belt fault.

Mapping US Urban Extents from MODIS Data Using One-Class Classification Method

Urban areas are one of the most important components of human society. Their extents have been continuously growing during the last few decades. Accurate and timely measurements of the extents of urban areas can help in analyzing population densities and urban sprawls and in studying environmental issues related to urbanization. Urban extents detected from remotely sensed data are usually a by-product of land use classification results, and their interpretation requires a full understanding of land cover types. In this study, for the first time, we mapped urban extents in the continental United States using a novel one-class classification method, i.e., positive and unlabeled learning (PUL), with multi-temporal Moderate Resolution Imaging Spectroradiometer (MODIS) data for the year 2010. The Defense Meteorological Satellite Program Operational Linescan System (DMSP-OLS) night stable light data were used to calibrate the urban extents obtained from the one-class classification scheme. Our results demonstrated the effectiveness of the use of the PUL algorithm in mapping large-scale urban areas from coarse remote-sensing images, for the first time. The total accuracy of mapped urban areas was 92.9% and the kappa coefficient was 0.85. The use of DMSP-OLS night stable light data can significantly reduce false detection rates from bare land and cropland far from cities. Compared with traditional supervised classification methods, the one-class classification scheme can greatly reduce the effort involved in collecting training datasets, without losing predictive accuracy.

A fault detection method for railway point systems

Failures of railway point systems (RPSs) often lead to service delays or hazardous situations. A condition monitoring system can be used by railway infrastructure operators to detect the early signs of the deteriorated condition of RPSs and thereby prevent failures. This paper presents a methodology for early detection of the changes in the measurement of the current drawn by the motor of the point operating equipment (POE) of an RPS, which can be used to warn about a possible failure in the system. The proposed methodology uses the one-class support vector machine classification method with the similarity measure of edit distance with real penalties. The technique has been developed taking into account specific features of the data of infield RPSs and therefore is able to detect the changes in the measurements of the current of the POE with greater accuracy compared with the commonly used threshold-based technique. The data from infield RPSs, which relate to incipient failures of RPSs, were used after the deficiencies in the data labelling were removed using expert knowledge. In addition, possible improvements in the proposed methodology were identified in order for it to be used as an automatic online condition monitoring system.

One-Class Classification with Extreme Learning Machine

One-class classification problem has been investigated thoroughly for past decades. Among one of the most effective neural network approaches for one-class classification, autoencoder has been successfully applied for many applications. However, this classifier relies on traditional learning algorithms such as backpropagation to train the network, which is quite time-consuming. To tackle the slow learning speed in autoencoder neural network, we propose a simple and efficient one-class classifier based on extreme learning machine (ELM). The essence of ELM is that the hidden layer need not be tuned and the output weights can be analytically determined, which leads to much faster learning speed. The experimental evaluation conducted on several real-world benchmarks shows that the ELM based one-class classifier can learn hundreds of times faster than autoencoder and it is competitive over a variety of one-class classification methods.