Multi-class Support Vector Classification Research Articles

MK-MSMCR: An efficient multiple kernel approach to multi-class classification

This paper introduces a novel multi-class support vector classification and regression (MSVCR) algorithm with multiple kernel learning (MK-MSVCR). We present a new MK-MSVCR algorithm based on two-stage learning (MK-MSVCRTSL). The two-stage learning aims to make classification algorithms better when dealing with complex data by using the first stage of learning to generate ?representative? or ?important? samples. We first establish the fast learning rate of MKMSVCR algorithm for multi-class classification with independent and identically distributed (i.i.d.) samples amd uniformly ergodic Markov chain (u.e.M.c.) smaples, and prove that MK-MSVCR algorithm is consistent. We show the numerical investigation on the learning performance of MK-MSVCR-TSL algorithm. The experimental studies indicate that the proposed MK-MSVCR-TSL algorithm has better learning performance in terms of prediction accuracy, sampling and training total time than other multi-class classification algorithms.

Generalization and learning rate of multi-class support vector classification and regression

The generalization is one of the main concerns of machine learning method. k-SVCR is an important multi-class classification algorithm, but there is no theoretical analysis on the generalization of k-SVCR algorithm up to now. Therefore, in this paper, we consider multi-class support vector classification and regression (MSVCR) algorithm. We first establish the generalization bounds of MSVCR based on uniformly ergodic Markovian chain (u.e.M.c.) samples, and obtain its fast learning rate of MSVCR. As applications, we estimate the generalization of MSVCR for independent and identically distributed (i.i.d.) observations and strongly mixing observations, respectively. We also propose a new MSVCR algorithm based on q-times Markovian resampling (MSVCR-Mar-q). The experimental studies indicate that compared to the classical k-SVCR and other multi-class SVM algorithms, the proposed algorithm not only has smaller misclassification rate, but also has less sampling and training total time.

Ramp loss for twin multi-class support vector classification

ABSTRACTTwin K-class support vector classification (TKSVC) adopts ‘One-vs.-One-vs.-Rest’ structure to utilise all the samples to increase the prediction accuracy. However, TKSVC is sensitive to noises or outliers due to the use of the Hinge loss function. To reduce the negative influence of outliers, in this paper, we propose a more robust algorithm termed as Ramp loss for twin K-class support vector classification (Ramp-TKSVC) where we use the Ramp loss function to substitute the Hinge loss function in TKSVC. Because the Ramp-TKSVC is a non-differentiable non-convex optimisation problem, we adopt Concave–Convex Procedure (CCCP) to solve it. To overcome the drawbacks of conventional multi-classification methodologies, the TKSVC is utilised as a core of our Ramp-TKSVC. In the Ramp-TKSVC, the outliers are prevented from becoming support vectors, thus they are not involved in the construction of hyperplanes, making the Ramp-TKSVC more robust. Besides, the Ramp-TKSVC is sparser than the TKSVC. To verify the validity of our Ramp-TKSVC, we conduct experiments on 12 benchmark datasets in both linear and nonlinear cases. The experimental results indicate that our algorithm outperforms the other five compared algorithms.

Real-time mode recognition based assistive torque control of bionic knee exoskeleton for sit-to-stand and stand-to-sit transitions

Assistive torque control is important for people who cannot fully accomplish sit-to-stand and stand-to-sit (STS) transitions. In this paper, we proposed a three-level control strategy for a bionic knee exoskeleton based on real-time STS transition recognition. Motion features were obtained from one potentiometer and two inertial measurement units integrated in the exoskeleton. A multi-class support vector classifier was utilized to infer the subject’s real-time motion intent. Twelve able-bodied subjects were recruited in experiments. Mean accuracy across subjects was 97.63%±0.01%. Once STS transition was detected, the proposed control system could add assistive torque in time to assist the subject to accomplish the transition.

Multi-category ternion support vector machine

This paper proposes a three-class classifier termed as ‘Ternion support vector machine’ (TerSVM) and its tree based multi-category classification approach termed as ‘Multi-category ternion support vector machine’ (M-TerSVM). The proposed classifier, TerSVM, is motivated by twin multi-class support vector classification (Twin-KSVC) and evaluates the data patterns for three outputs (+1,−1,0). Twin-KSVC has very high computational complexity, which makes it infeasible for real-world problems. Our proposed classifier, TerSVM, overcomes this limitation, as it formulates three unconstrained minimization problems (UMPs), instead of quadratic programming problems as solved by Twin-KSVC. The UMPs of TerSVM are solved as systems of linear equations which determine three proximal nonparallel hyperplanes. TerSVM can also be used as a binary classifier. This work also proposes a multi-category classification algorithm, M-TerSVM, that extents our three-class classifier (TerSVM) into multi-category framework. For a K-class problem, M-TerSVM constructs a classifier model in the form of a ternion tree of height ⌊K∕2⌋, where the data is partitioned into three groups at each level. Our algorithm uses a novel procedure to identify a reduced training set which improves its learning time. Numerical experiments performed on synthetic and benchmark datasets indicate that M-TerSVM outperforms other classical multi-category approaches like one-against-all and Twin-KSVC, in terms of generalization ability and learning time. This paper proposes the application of M-TerSVM for handwritten digit recognition and color image classification.

Evaluation of One-Class Support Vector Classification for Mapping the Paddy Rice Planting Area in Jiangsu Province of China from Landsat 8 OLI Imagery

Identification of paddy fields is essential for monitoring the rice cultivated area and predicting rice productivity. Timely and accurate extraction of rice distribution can bring vital information for national food security, agricultural policy formulation, and regional environmental sustainability. Conventional classification methods usually suffered from low accuracy, multi-class training samples, or demanding imagery requirements. This paper proposes to use one-class support vector classification (OCSVC) to extract rice cultivated area with Landsat Optical Land Imager (OLI) imagery. Instead of sampling and training all land cover types as performed by multi-class classification methods, OCSVC only used the training samples of target class (rice) for rice mapping. The performance of OCSVC was evaluated in terms of the classification accuracy of rice mapping and rice acreage estimation based on high-resolution imagery, field survey data and rice acreage data from government reports for Jiangsu Province, China. At the county-level, OCSVC was also compared with the commonly used multi-class support vector classification (MCSVC), decision tree classification (DTC), and vegetation index-based thresholding (VIT). Our results demonstrated that OCSVC produced a comparable overall accuracy to DTC and outperformed MCSVC and VIT. The computational efficiency of OCSVC increased approximately ten times as compared to MCSVC. The OCSVC produced the best correlation between its classified area and reported area among the four classification methods evaluated. When applied to the provincial level, the classification overall accuracy for OCSVC was 88.54%. The detected rice planting area for Jiangsu Province was 22,602 km2, which was consistent with the statistics from the National Bureau of Statistics (22,948 km2). This OCSVC-based mapping strategy provides a practical and efficient way to detect the rice planting extent with Landsat imagery at a large scale.

A unified view on multi-class support vector classification

A unified view on multi-class support vector machines (SVMs) is presented, covering most prominent variants including the one-vs-all approach and the algorithms proposed by Weston & Watkins, Cramme...

Process knowledge based multi-class support vector classification (PK-MSVM) approach for surface defects in hot rolling

Random surface defects occur during the hot bar rolling of steels and are identified either by manual or by automated inspection techniques. Manual inspection techniques are purely based on the process knowledge of the inspector such as the location, type and kind of defects, and the primary sources of these defects. The automated techniques, to identify and classify the defects, rely on machine vision technologies and image processing algorithms based on support vector machines, wavelets, image processing and statistical inference. Both these approaches have their own advantages and limitations. To improve the accuracy of classification of these defects a process knowledge based support vector classification scheme is proposed (called PK-MSVM) which combines feature extraction task of automated inspection with the process knowledge. The defect observation data from the imaging sensor is transformed to include this process knowledge. Three attributes of the defects – length to width ratio, longitudinal location and transverse location- are used for this transformation are they are closely related to the thermo-mechanics of the rolling process. Different formulations of the multi-class support vector machines (MSVMs) are compared for this classification with or without process knowledge based transformation: one-against-one, one-against-all and Hastie’s algorithm of multi class SVM. It is found that the new approach (PK-MSVM) performs better than traditional MSVM for all the three formulations. For the best case, the performance sees a jump of more than 100%. Thus incorporating process knowledge in identification and classification does increase the reliability of inspection considerably.

Multiclass support vector classification via coding and regression

The multiclass classification problem is considered and resolved through coding and regression. There are various coding schemes for transforming class labels into response scores. An equivalence notion of coding schemes is developed, and the regression approach is adopted for extracting a low-dimensional discriminant feature subspace. This feature subspace can be a linear subspace of the column span of original input data or kernel-mapped feature data. The classification training and prediction are carried out in this feature subspace using a linear classifier, which lead to a simple and computationally light but yet powerful toolkit for classification. Experimental results, including prediction ability and CPU time comparison with LIBSVM, show that the regression-based approach is a competent alternative for the multiclass problem.

A multi-class large margin classifier

Currently there are two approaches for a multi-class support vector classifier (SVC). One is to construct and combine several binary classifiers while the other is to directly consider all classes of data in one optimization formulation. For a K-class problem (K>2), the first approach has to construct at least K classifiers, and the second approach has to solve a much larger optimization problem proportional to K by the algorithms developed so far. In this paper, following the second approach, we present a novel multi-class large margin classifier (MLMC). This new machine can solve K-class problems in one optimization formulation without increasing the size of the quadratic programming (QP) problem proportional to K. This property allows us to construct just one classifier with as few variables in the QP problem as possible to classify multi-class data, and we can gain the advantage of speed from it especially when K is large. Our experiments indicate that MLMC almost works as well as (sometimes better than) many other multi-class SVCs for some benchmark data classification problems, and obtains a reasonable performance in face recognition application on the AR face database.