Laplacian Twin Support Vector Machine Research Articles

Laplacian Twin Support Vector Machine With Pinball Loss for Semi-Supervised Classification

Semi-supervised learning utilizes labeled data and the geometric information in the unlabeled data to construct a model whereas supervised learning makes use of the only label data. So, semi-supervised learning establishes a more reasonable classifier. In recent years, the Laplacian support vector machine (Lap-SVM) has received a lot of interest in the framework of semi-supervised classification. To develop the performance of Lap-SVM, Laplacian twin support vector machine (Lap-TSVM) has shown exceptional performance as an addition to improve the computational complexity. However, dealing with noise sensitivity and instability for resampling due to its hinge loss function is still a challenge. In this paper, we provide a novel Laplacian twin support vector machine by combining pinball loss function, termed as Lap-PTSVM, to effectively handle the aforementioned problems. As a result, it improves a better generalization ability of the classifier. Several experiments have been performed on artificial and UCI datasets. The results show that the proposed model has noise insensitivity comparable to the Lap-TSVM and has a great generalization performance. Furthermore, non-parametric statistical test are also conducted to justify the competitive performance of Lap-PTSVM.

Publisher Correction to: Intuitionistic Fuzzy Laplacian Twin Support Vector Machine for Semi-supervised Classification

Publisher Correction to: Intuitionistic Fuzzy Laplacian Twin Support Vector Machine for Semi-supervised Classification

Intuitionistic Fuzzy Laplacian Twin Support Vector Machine for Semi-supervised Classification

In general, data contain noises which come from faulty instruments, flawed measurements or faulty communication. Learning with data in the context of classification or regression is inevitably affected by noises in the data. In order to remove or greatly reduce the impact of noises, we introduce the ideas of fuzzy membership functions and the Laplacian twin support vector machine (Lap-TSVM). A formulation of the linear intuitionistic fuzzy Laplacian twin support vector machine (IFLap-TSVM) is presented. Moreover, we extend the linear IFLap-TSVM to the nonlinear case by kernel function. The proposed IFLap-TSVM resolves the negative impact of noises and outliers by using fuzzy membership functions and is a more accurate reasonable classifier by using the geometric distribution information of labeled data and unlabeled data based on manifold regularization. Experiments with constructed artificial datasets, several UCI benchmark datasets and MNIST dataset show that the IFLap-TSVM has better classification accuracy than other state-of-the-art twin support vector machine (TSVM), intuitionistic fuzzy twin support vector machine (IFTSVM) and Lap-TSVM.

Semi-supervised Weighted Ternary Decision Structure for Multi-category Classification

Semi-supervised learning has attracted researchers due to its advantages over supervised learning. In this paper, an extremely fast multi-category classification algorithm, termed as weighted ternary decision structure (WTDS) is proposed. WTDS is a generic algorithm that can extend any binary classifier into multi-category framework. This work also proposes a novel semi-supervised binary classifier termed as Weighted Laplacian least-squares twin support vector machine which is further extended using WTDS. The novel semi-supervised classifier obtains the solution by formulating a pair of Unconstrained Minimization Problems which are solved as systems of linear equation. WTDS takes advantage of the strengths of the classifier and efficiently constructs the multi-category classifier model in the form of a decision structure. WTDS outperforms other state-of-the-art multi-category approaches in terms of classification accuracy and time complexity. To confirm the feasibility and efficacy of proposed algorithm, experiments are conducted on benchmark UCI datasets.

A novel kernel-free nonlinear SVM for semi-supervised classification in disease diagnosis

Semi-supervised classification methods are widely-used and attractive for dealing with both labeled and unlabeled data in real-world problems. In this paper, a novel kernel-free Laplacian twin support vector machine method is proposed for semi-supervised classification. Its main idea is to classify data points into two classes by constructing two nonparallel quadratic surfaces so that each surface is close to one class of points and far away from the other class of points. The proposed method not only saves much computational time by avoiding choosing a kernel function and its related parameters in the classical support vector machine, but also addresses the issue of computational complexity by adopting manifold regularization technique. Moreover, two small-sized convex quadratic programming problems need to be solved to implement the proposed method, which is much easier than solving the non-convex problem of mixed integer programming to implement the well-known semi-supervised support vector machine. Finally, the numerical results on some artificial and benchmark data sets validate the competitive performance of proposed method in terms of efficiency, classification accuracy and generalization ability, by comparing to well-known semi-supervised methods. In particular, the proposed method handles five benchmarking disease diagnosis problems well and efficiently, which indicates the potential of proposed method in diagnosing and forecasting the diseases.

P.1.11 Inflammatory markers in anorexia nervosa: An exploratory study

In this paper, we propose a semi-supervised classifier termed as Fast Laplacian Twin Support Vector Machine (FLap−TWSVM) with an objective to reduce the requirement of labeled data and simultaneously lessen the training time complexity of a traditional Laplacian Twin Support Vector Machine semi-supervised classifier. FLap−TWSVM is faster than existing Laplacian twin support vector machine as it solves a smaller size Quadratic Programming Problem (QPP) along with an Unconstrained Minimization Problem (UMP) to obtain decision hyperplanes which can also handle heteroscedastic noise present in the training data. Traditional semi-supervised classifiers generally have no explicit control over the choice of labeled data available for training, hence to overcome this limitation, we propose a pool-based active learning framework which identifies most informative examples to train the learning model. Moreover, the aforementioned framework has been extended to deal with multi-category classification scenarios. Several experiments have been performed on machine learning benchmark datasets which proves the utility of the proposed classifier over traditional Laplacian Twin Support Vector Machine (Lap−TWSVM) and active learning based Support Vector Machine (SVMAL). The efficacy of the proposed framework has also been tested on human activity recognition problem and content based image retrieval system.

Fast Laplacian twin support vector machine with active learning for pattern classification

In this paper, we propose a semi-supervised classifier termed as Fast Laplacian Twin Support Vector Machine (FLap−TWSVM) with an objective to reduce the requirement of labeled data and simultaneously lessen the training time complexity of a traditional Laplacian Twin Support Vector Machine semi-supervised classifier. FLap−TWSVM is faster than existing Laplacian twin support vector machine as it solves a smaller size Quadratic Programming Problem (QPP) along with an Unconstrained Minimization Problem (UMP) to obtain decision hyperplanes which can also handle heteroscedastic noise present in the training data. Traditional semi-supervised classifiers generally have no explicit control over the choice of labeled data available for training, hence to overcome this limitation, we propose a pool-based active learning framework which identifies most informative examples to train the learning model. Moreover, the aforementioned framework has been extended to deal with multi-category classification scenarios. Several experiments have been performed on machine learning benchmark datasets which proves the utility of the proposed classifier over traditional Laplacian Twin Support Vector Machine (Lap−TWSVM) and active learning based Support Vector Machine (SVMAL). The efficacy of the proposed framework has also been tested on human activity recognition problem and content based image retrieval system.

Multi-category laplacian least squares twin support vector machine

In this paper, we have formulated a Laplacian Least Squares Twin Support Vector Machine called Lap-LST-KSVC for semi-supervised multi-category k-class classification problem. Similar to Least Squares Twin Support Vector Machine for multi-classification(LST-KSVC), Lap-LST-KSVC, evaluates all the training samples into “1-versus-1-versus-rest” classification paradigm, so as to generate ternary output {−1, 0, +1}. Experimental results prove the efficacy of the proposed method over other inline Laplacian Twin Support Vector Machine(Lap-TWSVM) in terms of classification accuracy and computational time.

Tree based multi-category Laplacian TWSVM for content based image retrieval

This paper attempts to combine Laplacian Twin Support Vector Machine (Lap-TWSVM) and Decision Tree Twin Support Vector Machine classifier to obtain an effective tree based classifier for semi-supervised multi-category classification. This classifier is termed as Tree Based Multi-Category Laplacian Twin Support Vector Machine (TB-Lap-TWSVM) classifier. TB-Lap-TWSVM is an improved One Against All (OAA) partition based tree classifier which takes advantage of both Lap-TWSVM and Decision Tree methodology. This paper concentrates on the application of TB-Lap-TWSVM to Content Based Image Retrieval problem. Further, extensive experiments on color images are carried out on different databases to establish the efficacy of the proposed model vis a vis TB-Lap-SVM and OAA-Lap-TWSVM.

Laplacian twin parametric-margin support vector machine for semi-supervised classification

As an extension of twin support vector machine (TSVM), twin parametric-margin support vector machine (TPMSVM) makes the learning speed faster than that of the parametric-margin ν-support vector machine (par-ν-SVM), and it is suitable for many cases, especially when the data has heteroscedastic error structure. This algorithm needs all the labels of training samples, which belongs to a supervised method. However, it is sometimes difficult to achieve each label of the data. To effectively handle this case, we propose a Laplacian twin parametric-margin support vector machine (LTPMSVM) for the semi-supervised classification, which exploits the geometric information of the marginal distribution embedded in unlabeled data to construct a more reasonable classifier. Additionally, the LTPMSVM has helpful properties to shed light on theoretical interpretation of parameters which control the bounds on proportions of support vectors and boundary errors. Experimental results on artificial datasets testify the properties of its parameters. Furthermore, results on the ABCDETC and twelve benchmark datasets indicate that our proposed LTPMSVM yields a good generalization performance with the comparable computing time to Laplacian twin support vector machine (LTSVM).

Laplacian least squares twin support vector machine for semi-supervised classification

The recently proposed Laplacian twin support vector machine (Lap-TSVM) is an excellent nonparallel-based kernel tool for semi-supervised classification problems, where its optimal decision hyperplane is determined by solving two quadratic programming problems (QPPs) with matrix inversion operations. In order to reduce its computation cost, in this paper, we formulate a least squares version of Lap-TSVM, termed as Lap-LSTSVM, leading to an extremely fast approach for generating semi-supervised classifiers. Besides, a meaningful regularization parameter is introduced for each problem in Lap-LSTSVM to balance the regularization terms between the reproducing kernel Hilbert spaces (RHKS) term and the manifold regularization (MR) term, instead of two parameters used in Lap-TSVM. In addition, an efficient conjugate gradient (CG) algorithm is further developed for solving the systems of linear equations (LEs) appeared to speed up the training procedure. Experimental results on both several synthetic and real-world datasets confirm the feasibility and the effectiveness of the proposed method.

NONPARALLEL HYPERPLANES PROXIMAL CLASSIFIERS BASED ON MANIFOLD REGULARIZATION FOR LABELED AND UNLABELED EXAMPLES

In this paper, we propose two Laplacian nonparallel hyperplane proximal classifiers (LapNPPCs) for semi-supervised and full-supervised classification problem respectively by adding manifold regularization terms. Due to the manifold regularization terms, our LapNPPCs are able to exploit the intrinsic structure of the patterns of the training set. Furthermore, our classifiers only need to solve two systems of linear equations rather than two quadratic programming (QP) problems as needed in Laplacian twin support vector machine (LapTSVM) (Z. Qi, Y. Tian and Y. Shi, Neural Netw.35 (2012) 46–53). Numerical experiments on toy and UCI benchmark datasets show that the accuracy of our LapNPPCs is comparable with other classifiers, such as the standard SVM, TWSVM and LapTSVM, etc. It is also the case that based on our LapNPPCs, some other TWSVM type classifiers with manifold regularization can be constructed by choosing different norms and loss functions to deal with semi-supervised binary and multi-class classification problems.

Laplacian smooth twin support vector machine for semi-supervised classification

Laplacian twin support vector machine (Lap-TSVM) is a state-of-the-art nonparallel-planes semi-supervised classifier. It tries to exploit the geometrical information embedded in unlabeled data to boost its generalization ability. However, Lap-TSVM may endure heavy burden in training procedure since it needs to solve two quadratic programming problems (QPPs) with the matrix “inversion” operation. In order to enhance the performance of Lap-TSVM, this paper presents a new formulation of Lap-TSVM, termed as Lap-STSVM. Rather than solving two QPPs in dual space, firstly, we convert the primal constrained QPPs of Lap-TSVM into unconstrained minimization problems (UMPs). Afterwards, a smooth technique is introduced to make these UMPs twice differentiable. At last, a fast Newton–Armijo algorithm is designed to solve the UMPs in Lap-STSVM. Experimental evaluation on both artificial and real-world datasets demonstrate the benefits of the proposed approach.

Improving Lap-TSVM with Successive Overrelaxation and Differential Evolution

Recent proposed laplacian twin support vector machine (Lap-TSVM) gains good generation by solving a pair of quadratic programming problems (QPPs). However, the training procedure of Lap-TSVM is time-consuming. More- over, compared with SVM and TWSVM, Lap-TSVM has more parameters need to regulate, which affects its practical applications. In this paper, we improve the Lap-TSVM from the following two aspects: (1) By introducing the suc- cessive overrelaxation (SOR) technique, the QPPs of Lap-TSVM are solved with fast training speed without loss of generalization. (2) A differential evolution (DE)-based model for Lap-TSVM's parameters selection is further sug- gested. Our DE-based model uses the real-value encoding instead of binary numbers, which enhances the efficiency of parameters selection greatly. Computational results on several benchmark datasets confirm the merits of the greatly improvements on the training procedure of Lap-TSVM.

Laplacian twin support vector machine for semi-supervised classification

Semi-supervised learning has attracted a great deal of attention in machine learning and data mining. In this paper, we have proposed a novel Laplacian Twin Support Vector Machine (called Lap-TSVM)...

Laplacian twin support vector machine for semi-supervised classification

Semi-supervised learning has attracted a great deal of attention in machine learning and data mining. In this paper, we have proposed a novel Laplacian Twin Support Vector Machine (called Lap-TSVM) for the semi-supervised classification problem, which can exploit the geometry information of the marginal distribution embedded in unlabeled data to construct a more reasonable classifier and be a useful extension of TSVM. Furthermore, by choosing appropriate parameters, Lap-TSVM degenerates to either TSVM or TBSVM. All experiments on synthetic and real data sets show that the Lap-TSVM's classifier combined by two nonparallel hyperplanes is superior to Lap-SVM and TSVM in both classification accuracy and computation time.