Two-class Problem Research Articles

Class proximity measures – Dissimilarity-based classification and display of high-dimensional data

For two-class problems, we introduce and construct mappings of high-dimensional instances into dissimilarity (distance)-based Class-Proximity Planes. The Class Proximity Projections are extensions of our earlier relative distance plane mapping, and thus provide a more general and unified approach to the simultaneous classification and visualization of many-feature datasets. The mappings display all L-dimensional instances in two-dimensional coordinate systems, whose two axes represent the two distances of the instances to various pre-defined proximity measures of the two classes. The Class Proximity mappings provide a variety of different perspectives of the dataset to be classified and visualized. We report and compare the classification and visualization results obtained with various Class Proximity Projections and their combinations on four datasets from the UCI data base, as well as on a particular high-dimensional biomedical dataset.

Automated Marsh-like classification of celiac disease in children using local texture operators

Automated classification of duodenal texture patches with histological ground truth in case of pediatric celiac disease is proposed. The classical focus of classification in this context is a two-class problem: mucosa affected by celiac disease and unaffected duodenal tissue. We extend this focus and apply classification according to a modified Marsh scheme into four classes. In addition to other techniques used previously for classification of endoscopic imagery, we apply local binary pattern (LBP) operators and propose two new operator types, one of which adapts to the different properties of wavelet transform subbands. The achieved results are promising in that operators based on LBP turn out to achieve better results compared to many other texture classification techniques as used in earlier work. Specifically, the proposed wavelet-based LBP scheme achieved the best overall accuracy of all feature extraction techniques considered in the two-class case and was among the best in the four-class scheme. Results also show that a classification into four classes is feasible in principle however when compared to the two-class case we note that there is still room for improvement due to various reasons discussed.

Efficient prediction algorithms for binary decomposition techniques

Binary decomposition methods transform multiclass learning problems into a series of two-class learning problems that can be solved with simpler learning algorithms. As the number of such binary learning problems often grows super-linearly with the number of classes, we need efficient methods for computing the predictions. In this article, we discuss an efficient algorithm that queries only a dynamically determined subset of the trained classifiers, but still predicts the same classes that would have been predicted if all classifiers had been queried. The algorithm is first derived for the simple case of pairwise classification, and then generalized to arbitrary pairwise decompositions of the learning problem in the form of ternary error-correcting output codes under a variety of different code designs and decoding strategies.

Paper currency number recognition based on binary tree and Adaboost algorithm

A fast weak classifier training algorithm and a fast caching strategy were used to accelerate Adaboost training.Integrated learning algorithm Adaboost can accurately construct two classifiers,so paper currency number recognition was formulated as a series of Adaboost two-class classification problems quickly and flexibly by using binary tree structure.The experimental results demonstrate that the fast Adaboost training algorithm can speed up the training and the paper currency number recognition system based on binary tree and Adaboost algorithm has good recognition rate and processing speed,and it has widely been used in currency counter,cash sorter and ATM.

Partial AUC maximization in a linear combination of dichotomizers

Classifier combination is a useful and common methodology to design an effective classification system. A large number of combination rules has been proposed hitherto, mostly aimed at minimizing the error rate. Recently, some methods have been presented that are devoted to maximize the area under the ROC curve (AUC), a more suitable performance measure when dealing with two-class problems with imprecise environment and/or imbalanced class priors. However, there are several applications that do not operate in the complete range of the ROC curve, but only in particular regions of it. In these cases, it is better to analyze the performance only in a part of the curve and to use the partial AUC (pAUC). This paper presents a new method that aims at maximizing the pAUC by means of linear combination of classifiers. The effectiveness of the proposed method has been proved on two biometric databases.

Rejection measurement based on linear discriminant analysis for document recognition

In document recognition, it is often important to obtain high accuracy or reliability and to reject patterns that cannot be classified with high confidence. This is the case for applications such as the processing of financial documents in which errors can be very costly and therefore far less tolerable than rejections. This paper presents a new approach based on Linear Discriminant Analysis (LDA) to reject less reliable classifier outputs. To implement the rejection, which can be considered a two-class problem of accepting the classification result or otherwise, an LDA-based measurement is used to determine a new rejection threshold. This measurement (LDAM) is designed to take into consideration the confidence values of the classifier outputs and the relations between them, and it represents a more comprehensive measurement than traditional rejection measurements such as First Rank Measurement and First Two Ranks Measurement. Experiments are conducted on the CENPARMI database of numerals, the CENPARMI Arabic Isolated Numerals Database, and the numerals in the NIST Special Database 19. The results show that LDAM is more effective, and it can achieve a higher reliability while maintaining a high recognition rate on these databases of very different origins and sizes.

A sparse nearest mean classifier for high dimensional multi-class problems

The analysis of small datasets in high dimensional spaces is inherently difficult. For two-class classification problems there are a few methods that are able to face the so-called curse of dimensionality. However, for multi-class sparsely sampled datasets there are hardly any specific methods. In this paper, we propose four multi-class classifier alternatives that effectively deal with this type of data. Moreover, these methods implicitly select a feature subset optimized for class separation. Accordingly, they are especially interesting for domains where an explanation of the problem in terms of the original features is desired. In the experiments, we applied the proposed methods to an MDMA powders dataset, where the problem was to recognize the production process. It turns out that the proposed multi-class classifiers perform well, while the few utilized features correspond to known MDMA synthesis ingredients. In addition, to show the general applicability of the methods, we applied them to several other sparse datasets, ranging from bioinformatics to chemometrics datasets having as few as tens of samples in tens to even thousands of dimensions and three to four classes. The proposed methods had the best average performance, while very few dimensions were effectively utilized.

An Iterative Method for Deciding SVM and Single Layer Neural Network Structures

We present two new classifiers for two-class classification problems using a new Beta-SVM kernel transformation and an iterative algorithm to concurrently select the support vectors for a support vector machine (SVM) and the hidden units for a single hidden layer neural network to achieve a better generalization performance. To construct the classifiers, the contributing data points are chosen on the basis of a thresholding scheme of the outputs of a single perceptron trained using all training data samples. The chosen support vectors are used to construct a new SVM classifier that we call Beta-SVN. The number of chosen support vectors is used to determine the structure of the hidden layer in a single hidden layer neural network that we call Beta-NN. The Beta-SVN and Beta-NN structures produced by our method outperformed other commonly used classifiers when tested on a 2-dimensional non-linearly separable data set.

Detection of vocal fold paralysis and oedema using time-domain features and Probabilistic Neural Network

This paper proposes a feature extraction method based on time-domain energy variation for the detection of vocal fold pathology. In this work, two different vocal fold problems (vocal fold paralysis and edema) are taken for analysis and in either case, a two-class pattern recognition problem is investigated. The normal and pathological speech samples are used from Massachusetts Eye and Ear Infirmary database. Probabilistic Neural Network (PNN) is employed for the classification. The experimental results show that the proposed features give very promising classification accuracy of 90% and can be used to detect the vocal fold paralysis and edema clinically.

Learning the areas of expertise of classifiers in an ensemble

Abstract There are various machine learning algorithms for extracting patterns from data; but recently, decision combination has become popular to improve accuracy over single learner systems. The fundamental idea behind combining the decisions of an ensemble of classifiers is that different classifiers most probably misclassify different patterns and by suitably combining the decisions of complementary classifiers, accuracy can be improved. In this paper, we investigate two kinds of classifier systems which are capable of estimating how much to weight each base classifier dynamically; during the calculation of the overall output for a given test data instance: (1) In ‘referee-based system’, a referee is associated with each classifier which learns the area of expertise of its associated classifier and weights it accordingly. (2) However, ‘gating system’ learns to partition the input space among all classifiers. Each referee in referee-based system learns a two-class problem (i.e., whether to use or not to use a classifier) whereas a gating system learns an L-class problem assigning the input to one of L base classifiers. Our analysis on 20 datasets from different domains and a classifier pool including 21 base learning algorithms reveals that the gating system tends to concentrate on a few of the base classifiers whereas a use of referees leads to a more balanced use of the base classifiers. Moreover, in the case of referees, it is better to use a small subset of base classifiers, instead of a single one. The study shows that, by using well-trained selection unit (referee or gating), we can get as high accuracy as using all the base classifiers (to combine their decisions) with drastic decrease in the number of base classifiers used, and also improve accuracy. The improvement is significant especially in cases when none of the base classifiers has high accuracy and it indicates that selecting classifiers appears promising as a means of solving hard learning problems.

Semi-supervised multi-class Adaboost by exploiting unlabeled data

Semi-supervised learning has attracted much attention in pattern recognition and machine learning. Most semi-supervised learning algorithms are proposed for binary classification, and then extended to multi-class cases by using approaches such as one-against-the-rest. In this work, we propose a semi-supervised learning method by using the multi-class boosting, which can directly classify the multi-class data and achieve high classification accuracy by exploiting the unlabeled data. There are two distinct features in our proposed semi-supervised learning approach: (1) handling multi-class cases directly without reducing them to multiple two-class problems, and (2) the classification accuracy of each base classifier requiring only at least 1/K or better than 1/K (K is the number of classes). Experimental results show that the proposed method is effective based on the testing of 21 UCI benchmark data sets.

FL-GrCCA: A granular computing classification algorithm based on fuzzy lattices

Defining a relation between granules and computing ever-changing granules are two important issues in granular computing. In view of this, this work proposes a partial order relation and lattice computing , respectively, for dealing with the aforementioned issues. A fuzzy lattice granular computing classification algorithm, or FL-GrCCA for short, is proposed here in the framework of fuzzy lattices. Algorithm FL-GrCCA computes a fuzzy inclusion relation between granules by using an inclusion measure function based on both a nonlinear positive valuation function, namely arctan , and an isomorphic mapping between lattices. Changeable classification granules are computed with a dilation operator using, conditionally, both the fuzzy inclusion relation between two granules and the size of a dilated granule. We compare the performance of FL-GrCCA with the performance of popular classification algorithms, including support vector machines (SVMs) and the fuzzy lattice reasoning (FLR) classifier, for a number of two-class problems and multi-class problems. Our computational experiments showed that FL-GrCCA can both speed up training and achieve comparable generalization performance.

Infrared Point Target Detection with Fisher Linear Discriminant and Kernel Fisher Linear Discriminant

It is a challenging task to detect point targets from an infrared image. Recently, the pattern recognition theory has been used to detect targets. The principal component analysis (PCA) has gained success in this field. We propose a linear subspace detection method based on Fisher linear discriminant (FLD) in this paper. If we consider images are made up of target class data and background class data, the target detection problem can be translated into a two-class classification problem. The FLD as one of pattern recognition algorithms can be used to find potential targets from image background. After classification by FLD, a map function, Gaussian map function, is developed to generate detection images in which the larger target-to-background contrast is obtained. FLD is a linear detection method without taking the higher-order statistics of image data into account. To improve detection performance, we extend this detection method to its nonlinear version, kernel FLD (KFLD) detection. Because the nonlinear subspace is capable of capturing the part of higher-order statistics, the better detection performance can be achieved. The well-devised experiments verify that KFLD detection outperforms FLD and other common used detection methods.

An End-to-End Error Model for Classification Methods Based on Temporal Change or Polarization Ratio of SAR Intensities

This paper aims at defining the expression of the probability of error of classification methods using a synthetic aperture radar (SAR) intensity ratio as a classification feature. The two SAR intensities involved in this ratio can be measurements from different dates, polarizations, or, also possibly, frequency bands. Previous works provided a baseline expression of the probability of error addressing the two-class problem with equal a priori class probabilities and no calibration error. This study brings up a novel expression of the error, providing the possibility to assess the effect of class probabilities and calibration errors. An extended expression is described for the <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n$</tex> </formula> -class problem. The effect of calibration errors such as channel gain imbalance, radiometric stability, and crosstalk is assessed in the general case. The results indicate that, for the applications under study, channel gain imbalance is usually not a decisive parameter, but radiometric stability is more critical in methods based on the temporal change. Crosstalk has a negligible effect in the case of copolarizations. The impacts of other system parameters, such as ambiguity ratio, time-lapse between repeat-pass orbits, spatial resolution, and number of looks, are illustrated through a set of assumptions on the backscattering values of the considered classes. The model is validated by comparing some of its outputs to experimental results calculated from the application of rice fields mapping methods on real data. This error model constitutes a tool for the design of future SAR missions and for the development of robust classification methods using existing SAR instruments.

An empirical study of binary classifier fusion methods for multiclass classification

One of the most important topics in information fusion is the combination of individual classifiers in multi-classifier systems. We have two different tasks in this area: one is the training and construction of ensembles of classifiers, with each one being able to solve the multiclass problem; the other task is the fusion of binary classifiers, with each one solving a different two-class problem to construct a multiclass classifier. This paper is devoted to the study of several aspects on the fusion process of binary classifiers to obtain a multiclass classifier. In the general case of a classification problem with more than two classes, we are faced with the issue that many algorithms either work better with two-class problems or are specifically designed for two-class problems. In such cases, a binarization method that maps the multiclass problem into several two-class problems must be used. In this task, information fusion plays a central role because of the combination of the prediction of the different binary classifiers into a multiclass classifier. Several issues regarding the way binary learners are trained and combined are raised by this task. Issues such as individual accuracy, diversity, and independence are common to other information fusion tasks such as the construction of ensembles of classifiers. This paper presents a study of the different class binarization methods for the various standard multiclass classification problems that have been proposed while addressing aspects not considered in previous works. We are especially concerned with many of the general assumptions in the field that have not been fully assessed by experimentation. We test the different methods in a large set of real-world problems from the UCI Machine Learning Repository, and we use six different base learners. Our results corroborate some of the previous results present in the literature. Furthermore, we present new results regarding the influence of the base learner on the performance of each method. We also show new results on the behavior of binary testing error and the independence of binary classifiers depending on the coding strategy. Finally, we study the behavior of the methods when the number of classes is high and in the presence of noise.

Visual quality assessment for web videos

The advent of video-sharing sites such as YouTube has led to an unprecedented Internet delivery of community-contributed video content. However, most of these videos are not quality-controlled. This paper reports a first attempt towards assessing web videos in terms of visual quality with significant tests on 30k web videos. We regard the quality assessment as a two-class classification problem: features motivated from domain knowledge are extracted to be the visual representation while the overall quality is the two-class label. Observing that web videos are characterized by a much higher diversity of content, genres, capture devices, and skills than any other traditional video program, we propose to combine two types of domain knowledge to predict the perceived quality score. One of the domain knowledge types is the spatiotemporal factors affecting the overall perceived quality of web videos, including four spatial factors and two temporal factors. We study the effectiveness of various spatiotemporal factors and propose some novel spatial factors pertaining the characteristics of web videos. The other is the video editing style, including shot editing style, frame size, and black side ratio. Comprehensive experiments and evaluations over 30k web videos which add up to 1200 h in total demonstrated the effectiveness of the proposed approach. We show some preliminary results for application to filtering and re-ranking of retrieved web videos.

Building binary-tree-based multiclass classifiers using separability measures

Various popular machine learning techniques, like support vector machines, are originally conceived for the solution of two-class (binary) classification problems. However, a large number of real problems present more than two classes. A common approach to generalize binary learning techniques to solve problems with more than two classes, also known as multiclass classification problems, consists of hierarchically decomposing the multiclass problem into multiple binary sub-problems, whose outputs are combined to define the predicted class. This strategy results in a tree of binary classifiers, where each internal node corresponds to a binary classifier distinguishing two groups of classes and the leaf nodes correspond to the problem classes. This paper investigates how measures of the separability between classes can be employed in the construction of binary-tree-based multiclass classifiers, adapting the decompositions performed to each particular multiclass problem.

Outlier One Class Support Vector Machines

One-Class Support Vector Machines (OCSVMs) distinguish outliers by computing a hyper-plane in feature space. The choice of the origin as separation point is arbitrary, which affects the decision boundary, and the distribution of samples has impact on the performance. Expanding the algorithm into solving two-class classification problems overcomes the drawbacks to a certain degree. However, the class imbalance problem is serious and the labeled outliers are rare or even non-existing. In this paper, a new Outlier OCSVM is proposed and a framework is designed for unsupervised outlier detection. Respectively scored by distance from hyper-plane and probabilistic output value, two definitions of outlier degree are presented. After picking out some suspicious outliers via combining the two criterions of outlier degree, the adjusted Outlier OCSVM starts the training operations, two parts of the dataset are updated interactively through comparison of the outputs. Experiment results on benchmark datasets show that the method can effectively improve the detection rate and reduce false positive rate, easy and reliable.

Automated Screening of Arrhythmia Using Wavelet Based Machine Learning Techniques

Arrhythmia is one of the preventive cardiac problems frequently occurs all over the globe. In order to screen such disease at early stage, this work attempts to develop a system approach based on registration, feature extraction using discrete wavelet transform (DWT), feature validation and classification of electrocardiogram (ECG). This diagnostic issue is set as a two-class pattern classification problem (normal sinus rhythm versus arrhythmia) where MIT-BIH database is considered for training, testing and clinical validation. Here DWT is applied to extract multi-resolution coefficients followed by registration using Pan Tompkins algorithm based R point detection. Moreover, feature space is compressed using sub-band principal component analysis (PCA) and statistically validated using independent sample t-test. Thereafter, the machine learning algorithms viz., Gaussian mixture model (GMM), error back propagation neural network (EBPNN) and support vector machine (SVM) are employed for pattern classification. Results are studied and compared. It is observed that both supervised classifiers EBPNN and SVM lead to higher (93.41% and 95.60% respectively) accuracy in comparison with GMM (87.36%) for arrhythmia screening.

A new method of pseudo absence data generation in landslide susceptibility mapping with a case study of Shenzhen

Landslide susceptibility mapping is a typical two-class classification problem where generating pseudo absence (non-slide) data plays an important role. In this paper, a new method, target space exteriorization sampling method (TSES), is presented to generate pseudo absence data based on presence data directly in feature space. TSES exteriorizes a presence sample to become a pseudo absence one by replacing the value of one of its features with a new one outside the value range of this feature of all presence data. This method is compared with two existing methods, buffer controlled sampling (BCS) and iteratively refined sampling (IRS), in a study area of Shenzhen city. The pseudo absence data generated by each of these three methods are organized into 20 subsets with increasing data sizes to study the effects of the proportion of pseudo absence data to presence data. The landslide susceptibility maps of the study area are calculated with all these datasets by general additive model (GAM). It can be concluded that, through a 10-fold validation, TSES and IRS-based models have similar AUC values that are both greater than that of BCS, but TSES outperforms BCS and IRS in prediction efficiency. TSES results also have more reasonable spatial and histogram distributions than BCS and IRS, which can support categorization of an area into more susceptibility ranks, while IRS shows a tendency to separate the whole study area into two susceptibility extremes. It can be also concluded that when using BCS, the pseudo absence data proportion to the presence data would be about 50% to get a considerable result, while for IRS or TSES the minimum proportion is 40%.