Lagrangian Support Vector Machines Research Articles

Elastic Net Nonparallel Hyperplane Support Vector Machine and Its Geometrical Rationality.

Twin support vector machine (TWSVM), which constructs two nonparallel classifying hyperplanes, is widely applied to various fields. However, TWSVM solves two quadratic programming problems (QPPs) separately such that the final classifiers lack consistency and enough prediction accuracy. Moreover, by reason of only considering the 1-norm penalty for slack variables, TWSVM is not well defined in the geometrical view. In this article, we propose a novel elastic net nonparallel hyperplane support vector machine (ENNHSVM), which adopts elastic net penalty for slack variables and constructs two nonparallel separating hyperplanes simultaneously. We further discuss the properties of ENNHSVM theoretically and derive the violation tolerance upper bound to better demonstrate the relative violations of training samples in the same class. In particular, we design a safe screening rule for ENNHSVM to speed up the calculations. We finally compare the performance of ENNHSVM on both synthetic datasets and benchmark datasets with the Lagrangian SVM, the twin parametric-margin SVM, the elastic net SVM, the TWSVM, and the nonparallel hyperplane SVM.

Generalization performance of Lagrangian support vector machine based on Markov sampling

In this paper, we first establish the generalization bounds of Lagrangian Support Vector Machines (LSVM) based on uniformly ergodic Markov chain (u.e.M.c.) samples. As an application, we also obtain the generalization bounds of LSVM based on strongly mixing sequence, independent and identically distributed (i.i.d.) samples, respectively. The fast learning rates of LSVM for u.e.M.c., strongly mixing sequence and i.i.d. samples are established. We also propose a new LSVM algorithm based on Markov sampling (LSVM̲MS) and show the learning performance of LSVM̲MS for UCI datasets. The experimental results show that the LSVM̲MS can improve obviously the learning performance of the classical LSVM algorithm. If the sampling and training total time is a main concern, the LSVM̲MS algorithm is the preferred method compared the known SVM algorithm based on Markov sampling.

Fuzzy classifier based on clustering with pairs of ε-hyperballs and its application to support fetal state assessment

Objective: In this study we propose a fuzzy classifier whose rule antecedents are determined based on the new approach to Clustering with Pairs of Prototypes (CPP). After demonstrating the high generalization ability of the classifier for six various benchmark datasets, a particular emphasis was placed on the application to support fetal state assessment based on the classification of cardiotocographic (CTG) signals.Methods: The CPP is a solution aimed at increasing the performance of fuzzy classifiers by introducing additional prototypes to those obtained using a given basal clustering method. The CPP improvement was achieved by applying the Fuzzy Clustering with ε-Hyperballs (FCεH) as basal clustering, as well as a new ant algorithm-based method of searching for pairs of prototypes.Results: The results were compared with three reference methods: the Lagrangian SVM with the Gaussian kernel function, and the same fuzzy classifier, but using the antecedents determined with the fuzzy c-means and the fuzzy (c+p)-means clustering. In case of five out of six benchmark datasets as well as for the CTG signals classification problem we achieved the highest generalization ability measured with the classification accuracy (benchmark data) and the classification quality index defined as geometric mean of sensitivity and specificity (CTG signals).Conclusions: The results of the numerical experiments showed high accuracy of the CPP-based fuzzy classifier when assessing various types of data. Moreover, the two-step classification of the CTG signals based on the proposed method allows for the efficient signal evaluation aiming to support the automated fetal state assessment.Significance and main impact: The most significant feature of the proposed method is the high generalization ability being the result of the ε-insensitive learning (FCεH clustering), while maintaining the possibility of interpreting the learning outcomes thanks to the linguistic representation of the knowledge in the form of fuzzy conditional (if-then) rules. Therefore, we believe that this solution will have a positive impact on other studies on intelligent systems.

An Anomaly Detection Algorithm based on Online Learning Lagrangian SVM for Cloud Computing Environment

An Anomaly Detection Algorithm based on Online Learning Lagrangian SVM for Cloud Computing Environment

Clustering with Pairs of Prototypes to Support Automated Assessment of the Fetal State

ABSTRACTCardiotocographic (CTG) monitoring, consisting in analysis of the fetal heart rate, uterine contractions, and fetal movements is the primary noninvasive method for the fetal state assessment. The visual interpretation of the CTG signals is characterized by the large inter- and intraobserver disagreement. Hence, the automated methods supporting the diagnosis process are the topic of researches. In the presented study, the evaluation of the CTG signals, based on fuzzy clustering with pairs of prototypes, is described. The efficiency of the proposed method is verified using two benchmark datasets of the CTG signals (CTU-UHB and SisPorto), and the problems of the two- and three-class classification are considered. The obtained results show the improved quality of the automated fetal state assessment in accordance with the applied reference procedures: the fuzzy (c+p)-means clustering and the Lagrangian Support Vector Machines.

Performance analysis of support vector machines classifiers in breast cancer mammography recognition

Support vector machine (SVM) is a supervised machine learning approach that was recognized as a statistical learning apotheosis for the small-sample database. SVM has shown its excellent learning and generalization ability and has been extensively employed in many areas. This paper presents a performance analysis of six types of SVMs for the diagnosis of the classical Wisconsin breast cancer problem from a statistical point of view. The classification performance of standard SVM (St-SVM) is analyzed and compared with those of the other modified classifiers such as proximal support vector machine (PSVM) classifiers, Lagrangian support vector machines (LSVM), finite Newton method for Lagrangian support vector machine (NSVM), Linear programming support vector machines (LPSVM), and smooth support vector machine (SSVM). The experimental results reveal that these SVM classifiers achieve very fast, simple, and efficient breast cancer diagnosis. The training results indicated that LSVM has the lowest accuracy of 95.6107 %, while St-SVM performed better than other methods for all performance indices (accuracy = 97.71 %) and is closely followed by LPSVM (accuracy = 97.3282). However, in the validation phase, the overall accuracies of LPSVM achieved 97.1429 %, which was superior to LSVM (95.4286 %), SSVM (96.5714 %), PSVM (96 %), NSVM (96.5714 %), and St-SVM (94.86 %). Value of ROC and MCC for LPSVM achieved 0.9938 and 0.9369, respectively, which outperformed other classifiers. The results strongly suggest that LPSVM can aid in the diagnosis of breast cancer.

Computerized analysis of fetal heart rate signals as the predictor of neonatal acidemia

Cardiotocography is the primary method for biophysical assessment of fetal state, which is mainly based on the recording and analysis of fetal heart rate (FHR) signal. Computerized systems for fetal monitoring provide a quantitative analysis of FHR signals, however the effective methods of qualitative assessment that could support the process of medical diagnosis are still needed. The measurements of hydronium ions concentration (pH) in neonatal cord blood are an objective indicator of the fetal outcome. Improper pH level is a symptom of acidemia being the result of fetal hypoxia. The paper proposes a two-step analysis of fetal heart rate recordings that allows for effective prediction of the acidemia risk. The first step consists in fuzzy classification of FHR signals. Fuzzy inference corresponds to the clinical interpretation of signals based on the FIGO guidelines. The goal of inference is to eliminate recordings indicating the fetal wellbeing from the further classification process. In the second step, the remained recordings are nonlinearly classified using multilayer perceptron and Lagrangian Support Vector Machines (LSVM). The proposed procedures are evaluated using data collected with computerized fetal surveillance system. The assessment performance is evaluated with the number of correct classifications (CC) and quality index (QI) defined as the geometric mean of sensitivity and specificity. The highest CC=92.0% and QI=88.2% were achieved for the Weighted Fuzzy Scoring System combined with the LSVM algorithm. The obtained results confirm the efficacy of the proposed methods of computerized analysis of FHR signals in the evaluation of the risk of neonatal acidemia.

Aplikasi Metode Cross Entropy untuk Support Vector Machines

Support vector machines (SVM) is a robust method for classification problem. In the original formulation, the dual form of SVM must be solved by a quadratic programming in order to get the optimal solution. The shortcoming of the standard version is as the classification problem is getting larger, the high computing time is needed. Cross entropy (CE) is a newly discovered optimization method with two main procedures: generating data samples by a chosen distribution, and updating the distribution parameters due to elite samples to generate a better sample in the next iteration. The CE method has been applied in many optimization problems with satisfying result. In this research, CE is applied to solve the optimization problem of Lagrangian SVM for faster computational time. This method is tested in some real world datasets for classification problem. The results show that the application of CE in SVM is comparable to standard SVM in classifying two class data in terms of accuracy. In addition, this method can solve large datasets classification problem faster than standard SVM.

Active learning based on coupled KNN pseudo pruning

It is very expensive and time-consuming to annotate huge amounts of data. Active learning would be a suitable approach to minimize the effort of annotation. A novel active learning approach, coupled K nearest neighbor pseudo pruning (CKNNPP), is proposed in the paper, which is based on querying examples by KNNPP method. The KNNPP method applies k nearest neighbor technique to search for k neighbor samples from labeled samples of unlabeled samples. When k labeled samples are not belong to the same class, the corresponded unlabeled sample is queried and given its right label by supervisor, and then it is added to labeled training set. In contrast with the previous depiction, the unlabeled sample is not selected and pruned, that is the pseudo pruning. This definition is enlightened from the K nearest neighbor pruning preprocessing. These samples selected by KNNPP are considered to be near or on the optimal classification hyperplane that is crucial for active learning. Especially, in order to avoid the excursion of the optimal classification hyperplane after adding a queried sample, CKNNPP method is proposed finally that two samples with different class label (like a couple, annotated by supervisor) are queried by KNNPP and added in the training set simultaneously for updating training set in each iteration. The CKNNPP can provide a good performance, and especially it is simple, effective, and robust, and can solve the classification problem with unbalanced dataset compared with the existing methods. Then, the computational complexity of CKNNPP is analyzed. Additionally, a new stopping criterion is applied in the proposed method, and the classifier is implemented by Lagrangian Support Vector Machines in iterations of active learning. Finally, twelve UCI datasets, image datasets of aircrafts, and the dataset of radar high-resolution range profile are used to validate the feasibility and effectiveness of the proposed method. The results illuminate that CKNNPP gains superior performance compared with the other seven state-of-the-art active learning approaches.

A new weighted approach to imbalanced data classification problem via support vector machine with quadratic cost function

In this paper, a new weighted approach on Lagrangian support vector machine for imbalanced data classification problem is proposed. The weight parameters are embedded in the Lagrangian SVM formulation. The training method for weighted Lagrangian SVM is presented and its convergence is proven. The weighted Lagrangian SVM classifier is tested and compared with some other SVMs using synthetic and real data to show its effectiveness and feasibility.

An incremental learning algorithm for Lagrangian support vector machines

Incremental learning has attracted more and more attention recently, both in theory and application. In this paper, the incremental learning algorithms for Lagrangian support vector machine (LSVM) are proposed. LSVM is an improvement to the standard linear SVM for classifications, which leads to the minimization of an unconstrained differentiable convex programming. The solution to this programming is obtained by an iteration scheme with a simple linear convergence. The inversion of the matrix in the solving algorithm is converted to the order of the original input space’s dimensionality plus one at the beginning of the algorithm. The algorithm uses the Sherman–Morrison–Woodbury identity to reduce the computation time. The incremental learning algorithms for LSVM presented in this paper include two cases that are namely online and batch incremental learning. Because the inversion of the matrix after increment is solved based on the previous computed information, it is unnecessary to repeat the computing process. Experimental results show that the algorithms are superior to others.

Semismooth Newton support vector machine

Support vector machines can be posed as quadratic programming problems in a variety of ways. This paper investigates the 2-norm soft margin SVM with an additional quadratic penalty for the bias term that leads to a positive definite quadratic program in feature space only with the nonnegative constraint. An unconstrained programming problem is proposed as the Lagrangian dual of the quadratic programming for the linear classification problem. The resulted problem minimizes a differentiable convex piecewise quadratic function with lower dimensions in input space, and a Semismooth Newton algorithm is introduced to solve it quickly, then a Semismooth Newton Support Vector Machine (SNSVM) is presented. After the kernel matrix is factorized by the Cholesky factorization or the incomplete Cholesky factorization, the nonlinear kernel classification problem can also be solved by SNSVM, and the complexity of the algorithms has no apparent increase. Many numerical experiments demonstrate that our algorithm is comparable with the similar algorithms such as Lagrangian Support Vector Machines (LSVM) and Semismooth Support Vector Machines (SSVM).

Projection Support Vector Machine Generators

Large-scale Support Vector Machine (SVM) classification is a very active research line in data mining. In recent years, several efficient SVM generation algorithms based on quadratic problems have been proposed, including: Successive OverRelaxation (SOR), Active Support Vector Machines (ASVM) and Lagrangian Support Vector Machines (LSVM). These algorithms have been used to solve classification problems with millions of points. ASVM is perhaps the fastest among them. This paper compares a new projection-based SVM algorithm with ASVM on a selection of real and synthetic data sets. The new algorithm seems competitive in terms of speed and testing accuracy.