Twin-hypersphere Support Vector Machine Research Articles

A novel multi-task twin-hypersphere support vector machine for classification

Both regularized multi-task learning (RMTL) and direct multi-task twin support vector machine (DMTSVM) have shown good performances in dealing with multi-task problems. They all use hyperplanes to realize classification. However, the hyperplane cannot reflect the distribution of data well. Therefore, in this paper, we propose a novel multi-task twin hypersphere support vector machine (MTTHSVM) to solve multi-task classification problems. It will generate two hyperspheres rather than hyperplanes for each task. So, the proposed method could better describe the distribution information of all training samples compared with the existing RMTL and DMTSVM. Based on Hierarchical Bayes theory, MTTHSVM divides the center of each hypersphere into task-specific and task-common parts to better measure the commonality and individuality of tasks. Then the shared information contained in multiple related tasks could be adaptively mined well. Therefore, the prediction accuracy will be improved to some extent. Besides, MTTHSVM is superior to RMTL and DMTSVM in terms of computational efficiency. This is because our MTTHSVM just solves two smaller-sized quadratic programming problems without any matrix inverse operations. Experimental results on one artificial data set, thirty-five benchmark data sets and a real image data set Cifar100 have verified the effectiveness of our method.

Fast multi-view twin hypersphere support vector machine with consensus and complementary principles

Fast multi-view twin hypersphere support vector machine with consensus and complementary principles

An Effective Multiclass Twin Hypersphere Support Vector Machine and Its Practical Engineering Applications

Twin-KSVC (Twin Support Vector Classification for K class) is a novel and efficient multiclass twin support vector machine. However, Twin-KSVC has the following disadvantages. (1) Each pair of binary sub-classifiers has to calculate inverse matrices. (2) For nonlinear problems, a pair of additional primal problems needs to be constructed in each pair of binary sub-classifiers. For these disadvantages, a new multi-class twin hypersphere support vector machine, named Twin Hypersphere-KSVC, is proposed in this paper. Twin Hypersphere-KSVC also evaluates each sample into 1-vs-1-vs-rest structure, as in Twin-KSVC. However, our Twin Hypersphere-KSVC does not seek two nonparallel hyperplanes in each pair of binary sub-classifiers as in Twin-KSVC, but a pair of hyperspheres. Compared with Twin-KSVC, Twin Hypersphere-KSVC avoids computing inverse matrices, and for nonlinear problems, can apply the kernel trick to linear case directly. A large number of comparisons of Twin Hypersphere-KSVC with Twin-KSVC on a set of benchmark datasets from the UCI repository and several real engineering applications, show that the proposed algorithm has higher training speed and better generalization performance.

Concave-Convex Programming for Ramp Loss-Based Maximum Margin and Minimum Volume Twin Spheres Machine

Twin hyper-sphere support vector machine (THSVM) classifies two classes of samples via two hyper-spheres instead of a pair of nonparallel hyper-planes in the conventional twin support vector machine. It avoids the matrix inverse operation. However, the THSVM uses hinge loss which easily leads to sensitivity of the noises. In this paper, we propose a ramp loss-based maximum margin and minimum volume twin spheres machine ( $$\hbox {RM}^{3}\hbox {TSM}$$ ) to enhance the ability of noise resistance. $$\hbox {RM}^{3}\hbox {TSM}$$ is robust because of introducing the ramp loss function, but it is non-convex. $$\hbox {RM}^{3}\hbox {TSM}$$ has several admirable advantages. For example, it can explicitly incorporate noises in the training process. In addition, the ramp loss function can be decomposed into the difference of a convex hinge loss and another convex loss. Then the concave-convex programming procedure is employed to efficiently solve the non-convex problems of $$\hbox {RM}^{3}\hbox {TSM}$$ by solving a sequence of convex programs iteratively. Experimental results on one artificial data set, 14 imbalanced binary datasets and 3 multiclass datasets indicate that our proposed $$\hbox {RM}^{3}\hbox {TSM}$$ yields a good generalization performance.

Improvements on twin-hypersphere support vector machine using local density information

In this paper, we propose a novel binary classifier called twin-hypersphere support vector machine with local density information (LDTHSVM). Firstly we extract local density for each training sample and treat it as the weight of that sample, next prune training dataset according to these local density degrees, finally introduce these local density degrees into twin-hypersphere support vector machine (THSVM) and reconstruct classification model on the pruned training dataset. LDTHSVM not only inherits good properties from THSVM, but also gives more robust description for dataset. The experimental results on synthetic and publicly available benchmark datasets show the excellent performance of the LDTHSVM classifier in terms of classification accuracy and learning time.

KNN-based maximum margin and minimum volume hyper-sphere machine for imbalanced data classification

Imbalanced data classification is often met in our real life. In this paper, a novel k-nearest neighbor (KNN)-based maximum margin and minimum volume hyper-sphere machine (KNN-M3VHM) is presented for the imbalanced data classification. The basic idea is to construct two hyper-spheres with different centres and radiuses. The first one contains majority examples and the second one covers minority examples. When constructing the first hyper-sphere, we remove some redundant majority samples using k-nearest neighbor (KNN)-based strategy to balance two classes of samples. Meanwhile, we maximize the margin between two hyper-spheres and minimize their volumes, which can result in two tight boundaries around each class. Similar to the twin hyper-sphere support vector machine (THSVM), KNN-M3VHM solves two related SVM-type problems and avoids the matrix inverse operation when solving the convex optimization problems. KNN-M3VHM considers not only the within-class information but also the between-class margin, then it achieves better performance in comparison with other state-of-the-art algorithms. Experimental results on twenty-five datasets validate the significant advantages of our proposed algorithm.

A maximum margin and minimum volume hyper-spheres machine with pinball loss for imbalanced data classification

The twin hyper-sphere support vector machine (THSVM) classifies two classes of samples via two hyper-spheres instead of a pair of nonparallel hyper-planes as in the conversional twin support vector machine (TSVM). Moreover THSVM avoids the matrix inverse operation when solving two dual quadratic programming problems (QPPs). However it cannot yield a desirable result when dealing with the imbalanced data classification. To improve the generalization performance, we propose a maximum margin and minimum volume hyper-spheres machine with pinball loss (Pin-M3HM) for the imbalanced data classification in this paper. The basic idea is to construct two hyper-spheres with different centers and radiuses in a sequential order. The first one contains as many examples in majority class as possible, and the second one covers minority class of examples as possible. Moreover the margin between two hyper-spheres is as large as possible. Besides, the pinball loss function is introduced into it to avoid the noise disturbance. Experimental results on 24 imbalanced datasets from the repositories of UCI and KEEL, and a real spectral dataset of Chinese grape wines indicate that our proposed Pin-M3HM yields a good generalization performance for the imbalanced data classification.

A structural information-based twin-hypersphere support vector machine classifier

Twin-hypersphere support vector machine (THSVM) for binary pattern recognition aims at generating two hyperspheres in the feature space such that each hypersphere contains as many as possible samples in one class and is as far as possible from the other one. THSVM has a fast learning speed since it solves two small sized support vector machine (SVM)-type quadratic programming problems (QPPs). However, it only simply considers the prior class-based structural information in the optimization problems. In this paper, a structural information-based THSVM (STHSVM) classifier for binary classification is presented. This proposed STHSVM focuses on the cluster-based structural information of the corresponding class in each optimization problem, which is vital for designing a good classifier in different real-world problems. In addition, it also leads to a fast learning speed since this STHSVM solves a series of smaller-sized QPPs compared with THSVM. Experimental results demonstrate that STHSVM is superior in generalization performance to other classifiers.

A twin hyper-sphere multi-class classification support vector machine

For the binary classification problem, twin hyper-sphere support vector machine (THSVM) is an improvement on the twin support vector machine (TSVM), it generates two hyper-spheres instead of a pair...

A twin hyper-sphere multi-class classification support vector machine

For the binary classification problem, twin hyper-sphere support vector machine (THSVM) is an improvement on the twin support vector machine (TSVM), it generates two hyper-spheres instead of a pair of hyper-planes. However, multi-class classification problems are always met in our real life. In this paper, we propose a new algorithm for the multi-classification problem, which is called twin hyper-sphere multi-classification support vector machine (THKSVM). Different from the “1-versus-rest” TSVM, THKSVM employs the “rest-versus-1” structure. Only one class of samples are restricted in the constraint, then it improves the computational speed of the classifier, especially for the case of the number of classes being large. Moreover, THKSVM avoids the inverse operation of a matrix when solving its dual quadratic programming problems (QPPs) compared with Twin-KSVC, so it is available for the large-scale data. Experimental results on six benchmark datasets indicate that THKSVM yields the comparable prediction accuracy with other algorithms, but it costs the shortest computational time among four algorithms.

A twin-hypersphere support vector machine classifier and the fast learning algorithm

This paper formulates a twin-hypersphere support vector machine (THSVM) classifier for binary recognition. Similar to the twin support vector machine (TWSVM) classifier, this THSVM determines two hyperspheres by solving two related support vector machine (SVM)-type problems, each one is smaller than the classical SVM, which makes the THSVM be more efficient than the classical SVM. In addition, the THSVM avoids the matrix inversions in its two dual quadratic programming problems (QPPs) compared with the TWSVM. By considering the characteristics of the dual QPPs of THSVM, an efficient Gilbert’s algorithm for the THSVM based on the reduced convex hull (RCH) instead of directly optimizing its pair of QPPs is further presented. Computational results on several synthetic as well as benchmark datasets indicate the significant advantages of the THSVM classifier in the computational time and test accuracy.