Two-class Problem Research Articles

Classifying dementia using local binary patterns from different regions in magnetic resonance images.

Dementia is an evolving challenge in society, and no disease-modifying treatment exists. Diagnosis can be demanding and MR imaging may aid as a noninvasive method to increase prediction accuracy. We explored the use of 2D local binary pattern (LBP) extracted from FLAIR and T1 MR images of the brain combined with a Random Forest classifier in an attempt to discern patients with Alzheimer's disease (AD), Lewy body dementia (LBD), and normal controls (NC). Analysis was conducted in areas with white matter lesions (WML) and all of white matter (WM). Results from 10-fold nested cross validation are reported as mean accuracy, precision, and recall with standard deviation in brackets. The best result we achieved was in the two-class problem NC versus AD + LBD with total accuracy of 0.98 (0.04). In the three-class problem AD versus LBD versus NC and the two-class problem AD versus LBD, we achieved 0.87 (0.08) and 0.74 (0.16), respectively. The performance using 3DT1 images was notably better than when using FLAIR images. The results from the WM region gave similar results as in the WML region. Our study demonstrates that LBP texture analysis in brain MR images can be successfully used for computer based dementia diagnosis.

Absent data generating classifier for imbalanced class sizes

We propose an algorithm for two-class classification problems when the training data are imbalanced. This means the number of training instances in one of the classes is so low that the conventional classification algorithms become ineffective in detecting the minority class. We present a modification of the kernel Fisher discriminant analysis such that the imbalanced nature of the problem is explicitly addressed in the new algorithm formulation. The new algorithm exploits the properties of the existing minority points to learn the effects of other minority data points, had they actually existed. The algorithm proceeds iteratively by employing the learned properties and conditional sampling in such a way that it generates sufficient artificial data points for the minority set, thus enhancing the detection probability of the minority class. Implementing the proposed method on a number of simulated and real data sets, we show that our proposed method performs competitively compared to a set of alternative state-of-the-art imbalanced classification algorithms.

Quadratic programming for class ordering in rule induction

Separate-and-conquer type rule induction algorithms such as Ripper, solve a K > 2 class problem by converting it into a sequence of K − 1 two-class problems. As a usual heuristic, the classes are fed into the algorithm in the order of increasing prior probabilities. Although the heuristic works well in practice, there is much room for improvement. In this paper, we propose a novel approach to improve this heuristic. The approach transforms the ordering search problem into a quadratic optimization problem and uses the solution of the optimization problem to extract the optimal ordering. We compared new Ripper (guided by the ordering found with our approach) with original Ripper (guided by the heuristic ordering) on 27 datasets. Simulation results show that our approach produces rulesets that are significantly better than those produced by the original Ripper.

Research Of Two Class Confidence Classification Based On One Class Classifier

Abstract To have simple and efficient confidence machine learning is an important focus in confidence machine researches. Using one class classifier as a tool, the paper applies it twice for two-class classification problems. Setting reject options and a multi-layer ensemble learning method are used in this study. In this method there is no necessity to set up a specific threshold and the confidence computation is omitted. Realizing five experiments, the study proves it as efficient.

Application of ECOC SVMS in Remote Sensing Image Classification

Abstract. Image processing has been one of the efficient technologies for GIS data requisition. Support Vector Machines (SVMs) have peculiar advantages in handling problems with small sample sizes, nonlinearity, and high dimensionality. However, SVMs can only solve two-class problems while multi-class decision is impossible. Error correcting output coding (ECOC) SVMs enhance the ability of fault tolerance when solving multi-class classification problems, which makes ECOC SVMs suitable for remote sensing image classification. In this paper, the generalization ability of ECOC SVMs is discussed. ECOC SVMs with optimum coding matrices are selected by experiment, and applied to remote sensing image classification. Experimental results show that, compared with Conventional multi-class classification methods, less SVM sub-classifiers are needed for ECOC SVMs in remote sensing image classification, and the classification accuracy is also improved.

Eye detection using discriminatory Haar features and a new efficient SVM

This paper presents an accurate and efficient eye detection method using the discriminatory Haar features (DHFs) and a new efficient support vector machine (eSVM). The DHFs are extracted by applying a discriminating feature extraction (DFE) method to the 2D Haar wavelet transform. The DFE method is capable of extracting multiple discriminatory features for two-class problems based on two novel measure vectors and a new criterion in the whitened principal component analysis (PCA) space. The eSVM significantly improves the computational efficiency upon the conventional SVM for eye detection without sacrificing the generalization performance. Experiments on the Face Recognition Grand Challenge (FRGC) database and the BioID face database show that (i) the DHFs exhibit promising classification capability for eye detection problem; (ii) the eSVM runs much faster than the conventional SVM; and (iii) the proposed eye detection method achieves near real-time eye detection speed and better eye detection performance than some state-of-the-art eye detection methods.

Liquid State Machine With Dendritically Enhanced Readout for Low-Power, Neuromorphic VLSI Implementations

In this paper, we describe a new neuro-inspired, hardware-friendly readout stage for the liquid state machine (LSM), a popular model for reservoir computing. Compared to the parallel perceptron architecture trained by the p-delta algorithm, which is the state of the art in terms of performance of readout stages, our readout architecture and learning algorithm can attain better performance with significantly less synaptic resources making it attractive for VLSI implementation. Inspired by the nonlinear properties of dendrites in biological neurons, our readout stage incorporates neurons having multiple dendrites with a lumped nonlinearity (two compartment model). The number of synaptic connections on each branch is significantly lower than the total number of connections from the liquid neurons and the learning algorithm tries to find the best 'combination' of input connections on each branch to reduce the error. Hence, the learning involves network rewiring (NRW) of the readout network similar to structural plasticity observed in its biological counterparts. We show that compared to a single perceptron using analog weights, this architecture for the readout can attain, even by using the same number of binary valued synapses, up to 3.3 times less error for a two-class spike train classification problem and 2.4 times less error for an input rate approximation task. Even with 60 times larger synapses, a group of 60 parallel perceptrons cannot attain the performance of the proposed dendritically enhanced readout. An additional advantage of this method for hardware implementations is that the 'choice' of connectivity can be easily implemented exploiting address event representation (AER) protocols commonly used in current neuromorphic systems where the connection matrix is stored in memory. Also, due to the use of binary synapses, our proposed method is more robust against statistical variations.

Privacy-Preserving Multi-Class Support Vector Machine for Outsourcing the Data Classification in Cloud

Emerging cloud computing infrastructure replaces traditional outsourcing techniques and provides flexible services to clients at different locations via Internet. This leads to the requirement for data classification to be performed by potentially untrusted servers in the cloud. Within this context, classifier built by the server can be utilized by clients in order to classify their own data samples over the cloud. In this paper, we study a privacy-preserving (PP) data classification technique where the server is unable to learn any knowledge about clients’ input data samples while the server side classifier is also kept secret from the clients during the classification process. More specifically, to the best of our knowledge, we propose the first known client-server data classification protocol using support vector machine. The proposed protocol performs PP classification for both two-class and multi-class problems. The protocol exploits properties of Pailler homomorphic encryption and secure two-party computation. At the core of our protocol lies an efficient, novel protocol for securely obtaining the sign of Pailler encrypted numbers.

Least Squares Twin Support Tensor Machine for Classification

Least Squares Support Vector Machine (LS-SVM) and Twin Support Vector Machine (TSVM) are efiective learning methods for classiflcation based on Support Vector Machine which has been widely used in many aspects and received extensive attention by academic community. At present, data representation is one of the core problems in machine learning. In practice, many objects are naturally represented by tensors. In this paper, we propose Least Squares Twin Support Tensor Machine (LSTSTM) which based on tensor data. We use two non-parallel hyperplanes and least squares idea for classiflcation, which is difierent form Support Tensor Machine (STM). LS-TSTM combines the characteristics of many learning machines. It makes full use of the structural information of the data, and has the features of less computation cost and higher precision. The numerical experiments of the two-class classiflcation problem for tensor data show that LS-TSTM has its advantages compared with other learning machines.

Using Empirical Mode Decomposition with Spatio-Temporal dynamics to classify single-trial Motor Imagery in BCI.

This paper introduces a new signal processing method called Spatio-Temporal Multivariate Empirical Mode Decomposition (ST-MEMD). It is a new variation of Empirical Mode Decomposition (EMD) that takes spatial and temporal information into account simultaneously rather than processing each signal source in isolation. The original and new methods were tested on single-trial electroencephalogram data with a two-class problem, in this case data using the Motor Imagery paradigm in brain-computer interfacing. However, whilst ST-MEMD retained the increase in sensitivity and specificity from adding spatial data, the new temporal data made no meaningful difference in terms of performance.

Combining local and global learners in the pairwise multiclass classification

Pairwise classification is a well-known class binarization technique that converts a multiclass problem into a number of two-class problems, one problem for each pair of classes. However, in the pairwise technique, nuisance votes of many irrelevant classifiers may result in a wrong class prediction. To overcome this problem, a simple, but efficient method is proposed and evaluated in this paper. The proposed method is based on excluding some classes and focusing on the most probable classes in the neighborhood space, named Local Crossing Off (LCO). This procedure is performed by employing a modified version of standard K-nearest neighbor and large margin nearest neighbor algorithms. The LCO method takes advantage of nearest neighbor classification algorithm because of its local learning behavior as well as the global behavior of powerful binary classifiers to discriminate between two classes. Combining these two properties in the proposed LCO technique will avoid the weaknesses of each method and will increase the efficiency of the whole classification system. On several benchmark datasets of varying size and difficulty, we found that the LCO approach leads to significant improvements using different base learners. The experimental results show that the proposed technique not only achieves better classification accuracy in comparison to other standard approaches, but also is computationally more efficient for tackling classification problems which have a relatively large number of target classes.

A radial basis function network classifier to maximise leave-one-out mutual information

We develop an orthogonal forward selection (OFS) approach to construct radial basis function (RBF) network classifiers for two-class problems. Our approach integrates several concepts in probabilistic modelling, including cross validation, mutual information and Bayesian hyperparameter fitting. At each stage of the OFS procedure, one model term is selected by maximising the leave-one-out mutual information (LOOMI) between the classifier's predicted class labels and the true class labels. We derive the formula of LOOMI within the OFS framework so that the LOOMI can be evaluated efficiently for model term selection. Furthermore, a Bayesian procedure of hyperparameter fitting is also integrated into the each stage of the OFS to infer the l2-norm based local regularisation parameter from the data. Since each forward stage is effectively fitting of a one-variable model, this task is very fast. The classifier construction procedure is automatically terminated without the need of using additional stopping criterion to yield very sparse RBF classifiers with excellent classification generalisation performance, which is particular useful for the noisy data sets with highly overlapping class distribution. A number of benchmark examples are employed to demonstrate the effectiveness of our proposed approach.

Recognition of candidate aptamer sequences for human hepatocellular carcinoma in SELEX screening using structure–activity relationships

Selecting and synthesizing aptamers for human hepatic carcinoma (HCC) cells with high affinity and specificity would be of critical importance for diagnosing liver cancer early. This paper is the first report on pattern recognition used for SELEX-based aptamer screening by applying a support vector classification (SVC) technique for a two-class problem. The candidate aptamer sequences that show different degrees of affinity and specificity for SMMC-7721 liver carcinoma cells were selected through whole cell-SELEX. After calculating 1670 molecular descriptors, 13 descriptors were selected, which were compressed to 6 latent variables used as the inputs for classification models. The predicted fractions of winner aptamers from the SELEX selection of the 3rd, 5th, 7th, 9th, 11th, and 13th rounds are 0.033, 0.427, 0.678, 0.828, 0.912 and 0.983, respectively, which conform to the aptamer evolutionary principles of SELEX based screening. By the pattern recognition analysis based on a structure–activity relationship (SAR) model, 6 DNA candidate aptamer sequences belonging to the class of sequences with high affinity and specificity have experimental dissociation constants Kd in the nanomolar range. The feasibility of applying pattern recognition for the design and selection of aptamers has been demonstrated.

CFS-SMO based classification of breast density using multiple texture models.

It is highly acknowledged in the medical profession that density of breast tissue is a major cause for the growth of breast cancer. Increased breast density was found to be linked with an increased risk of breast cancer growth, as high density makes it difficult for radiologists to see an abnormality which leads to false negative results. Therefore, there is need for the development of highly efficient techniques for breast tissue classification based on density. This paper presents a hybrid scheme for classification of fatty and dense mammograms using correlation-based feature selection (CFS) and sequential minimal optimization (SMO). In this work, texture analysis is done on a region of interest selected from the mammogram. Various texture models have been used to quantify the texture of parenchymal patterns of breast. To reduce the dimensionality and to identify the features which differentiate between breast tissue densities, CFS is used. Finally, classification is performed using SMO. The performance is evaluated using 322 images of mini-MIAS database. Highest accuracy of 96.46% is obtained for two-class problem (fatty and dense) using proposed approach. Performance of selected features by CFS is also evaluated by Naïve Bayes, Multilayer Perceptron, RBF Network, J48 and kNN classifier. The proposed CFS-SMO method outperforms all other classifiers giving a sensitivity of 100%. This makes it suitable to be taken as a second opinion in classifying breast tissue density.

Automated Focal Liver Lesion Staging Classification based on Haralick Texture Features and Multi-SVM

paper proposes automated identification and classification of various stages of focal liver lesions based on the Multi-Support Vector Machine (Multi-SVM). The proposed system can be used to discriminate focal liver diseases such as Cyst, Hemangioma, and Hepatocellular carcinoma along with normal liver. The multi-class scenario is a composition of a series of two-class problems, using one- against-all which is the earliest and one of the most widely used implementations. We formulate the discrimination between cysts, cavernous hemangioma, hepatocellular carcinoma, and normal tissue as a supervised learning problem, and apply Multi-SVM to classify the diseases using Haralick local texture descriptors and histogram based features calculated from Regions Of Interest (ROIs), as input. Selection of ROI significantly impact the classification performances, thus we proposes an automatic ROI selection using Fuzzy c-means initialized by level set technique. For multi-class classification, we adopt the One-Against-All (OAA) method. The proposed Multi-SVM based CAD system using 10-fold cross validation yielded classification accuracy of 96.11% with the individual class accuracy of 97.78%, 95.56%, 93.33% and 97.78% for NOR, Cyst, HEM and HCC cases respectively. The proposed Multi-SVM based system is compared with the K-Nearest Neighbor (KNN) based approaches. Experimental results have demonstrated that the Multi-SVM based system greatly outperforms KNN-based approaches and other methods in the literature. The good performance of the proposed method shows a reliable indicator that can improve the information in the staging of focal liver lesion diseases.

Growing construction of conlitron and multiconlitron

Based on the concepts of conlitron and multiconlitron, we propose a growing construction technique for improving the performance of piecewise linear classifiers on two-class problems. This growing technique consists of two basic operations: SQUEEZE and INFLATE, which can produce relatively reliable linear boundaries. In the convexly separable case, the growing process, forming a growing support conlitron algorithm (GSCA), starts with an initial conlitron and uses SQUEEZE to train a new conlitron, moving its classification boundary closer to the interior convex region and fitting the data distribution better statistically. In the commonly separable case, the growing process, forming a growing support multiconlitron algorithm (GSMA), starts with an initial multiconlitron and uses INFLATE and SQUEEZE to train a new multiconlitron, making its classification boundary adjusted to improve the generalization ability. Experimental evaluation shows that the growing technique can simplify the structure of a conlitron/multiconlitron effectively by reducing the number of linear functions, largely keeping and even greatly improving the level of classification performances. Therefore, it would come to play an important role in the subsequent development of piecewise linear learning, with the main goal to improve piecewise linear classifiers in a general framework.

Optimal cut-off for rare events and unbalanced misclassification costs

This paper develops a method for handling two-class classification problems with highly unbalanced class sizes and misclassification costs. When the class sizes are highly unbalanced and the minority class represents a rare event, conventional classification methods tend to strongly favour the majority class, resulting in very low detection of the minority class. A method is proposed to determine the optimal cut-off for asymmetric misclassification costs and for unbalanced class sizes. Monte Carlo simulations show that this proposal performs better than the method based on the notion of classification accuracy. Finally, the proposed method is applied to empirical data on Italian small and medium enterprises to classify them into default and non-default groups.

PDFOS: PDF estimation based over-sampling for imbalanced two-class problems

This contribution proposes a novel probability density function (PDF) estimation based over-sampling (PDFOS) approach for two-class imbalanced classification problems. The classical Parzen-window kernel function is adopted to estimate the PDF of the positive class. Then according to the estimated PDF, synthetic instances are generated as the additional training data. The essential concept is to re-balance the class distribution of the original imbalanced data set under the principle that synthetic data sample follows the same statistical properties. Based on the over-sampled training data, the radial basis function (RBF) classifier is constructed by applying the orthogonal forward selection procedure, in which the classifier׳s structure and the parameters of RBF kernels are determined using a particle swarm optimisation algorithm based on the criterion of minimising the leave-one-out misclassification rate. The effectiveness of the proposed PDFOS approach is demonstrated by the empirical study on several imbalanced data sets.

A bias correction function for classification performance assessment in two-class imbalanced problems

This paper introduces a framework that allows to mitigate the impact of class imbalance on most scalar performance measures when used to evaluate the behavior of classifiers. Formally, a correction function is defined with the aim of highlighting those classification results that present moderately higher prediction rates on the minority class. Besides, this function punishes those scenarios that are biased towards the majority class, but also those that are strongly biased to favor the minority class. This strategy assumes a typical imbalance task, in which the minority class contains the most relevant samples to the research purposes. A novel experimental framework is designed to show the advantages of our approach when compared to the standard use of well-established measures, demonstrating its consistency and validity.

Import Vector Machines for Quantitative Analysis of Hyperspectral Data

In this letter we explore probabilities derived from an import vector machines (IVM) classifier as quantitative measures of class proportion. We have developed a parameter selection strategy that improves the description of class proportions. This strategy incorporates the use of spectral mixtures, which represent gradual class transitions, into the parameter selection process. In addition, we evaluated the sensitivity of our approach in regard to increasing training uncertainty and signal-to-noise ratio. The approach was tested for binary, two-class problems on hyperspectral in situ measurements. The IVM models generated with our parameter selection strategy achieved similar or even improved classification accuracies compared to parameter selection with the standard IVM classification approach. Furthermore, the respective class probabilities correlated highly with reference class proportions. This new strategy is less affected by the inclusion of random noise and relatively stable against increased training errors.