Proximal Support Vector Machine Research Articles

Twin Support Vector Machines Based on Quantum Particle Swarm Optimization

Twin Support Vector Machines (TWSVM) are developed on the basis of Proximal Support Vector Machines (PSVM) and Proximal Support Vector Machine based on the generalized eigenvalues(GEPSVM). The solving of binary classification problem is converted to the solving of two smaller quadratic programming problems by TWSVM. And then it gets two non-parallel hyperplanes. Its efficiency of dealing with the problems and performance are better than the traditional support vector machines. However, it also has some problems. Its own parameters are difficult to be appointed. In order to solve this problem, on the basis of in-depth study of TWSVM, this paper proposes an algorithm that is the Twin Support Vector Machines based on Quantum Particle Swarm Optimization (QPSO-TWSVM) .By the use of the global searching ability of the Quantum Particle Swarm Optimization (QPSO), QPSO-TWSVM can search the optimal parameters in the global scope and avoid itself falling into the local optimum prematurely to find the values of the parameters which are the closest to the optimal parameters. QPSO-TWSVM avoids using the empirical values to appoint the parameters successfully. Compared with the traditional TWSVM, QPSO-TWSVM can appoint the parameters more accurately and avoid selecting the parameters blindly. Because of the better parameter selections, QPSO-TWSVM improves the classification accuracy of TWSVM.

Local and Global Regularized Twin SVM

The generalized eigenvalue proximal support vector machine (GEPSVM) and twin support vector machine (TWSVM) was proposed by Mangasarian and Jayadeva respectively, which aroused the interest of academia for its less computation cost and better generalization ability. They use the nonparallel hyperplane classifiers to solve the classification problem. Different from traditionally local or global TWSVM methods, a new Twin SVM algorithm called Local and Global Regularized Twin SVM (TWSVMLG) is proposed in this paper. A global regularizer was imposed across local models to smooth the data labels predicted by those local classifiers and avoid overfitting risk for the local classifiers. The classifier could get stronger discriminating ability when exploring local and global information than traditional algorithms. Finally some experimental results are presented to show the effectiveness of our algorithm.

Hybrid Designing of a Neural System by Combining Fuzzy Logical Framework and PSVM for Visual Haze-Free Task

Brain-like computer research and development have been growing rapidly in recent years. It is necessary to design large scale dynamical neural networks (more than 106 neurons) to simulate complex process of our brain. But such kind of task is not easy to achieve only based on the analysis of partial differential equations, especially for those complex neural models, e.g. Rose-Hindmarsh (RH) model. So in this paper, we develop a novel approach by combining fuzzy logical designing with Proximal Support Vector Machine Classifiers (PSVM) learning in the designing of large scale neural networks. Particularly, our approach can effectively simplify the designing process, which is crucial for both cognition science and neural science. At last, we conduct our approach on an artificial neural system with more than 108 neurons for haze-free task, and the experimental results show that texture features extracted by fuzzy logic can effectively increase the texture information entropy and improve the effect of haze-removing in some degree.

Ensemble learning for generalised eigenvalues proximal support vector machines

In this paper, to improve the generalisation ability of generalised eigenvalues proximal support vector machines GEPSVM, we propose an ensemble GEPSVM, called EnGEP for short. Note that GEPSVM is not sensitive to different weights of the points, to increase the potential diversity of GEPSVM, firstly, we introduce an extra parameter in GEPSVM, which gives different penalties for two non-hyperplanes determines by GEPSVM. Then, we use a novel bagging strategy to ensemble GEPSVM with additional parameters. Experimental results both on artificial and benchmark datasets show that our EnGEP improves the generalisation performance of GEPSVM greatly, and it also reveals the effectiveness of our EnGEP.

Improvement of Proximal Support Vector Machine and its Application to Enhance Antifreeze Heat Transfer Capability in Ground Source Heat Pump System

To solve the problem of enhancing the heat transfer capability of antifreeze mixture in a ground source heat pump system, the existing proximal support vector machines [1] was updated into a weighted Proximal Support Vector Machine (PSVM) model. Also, a new classification method of mixed antifreeze heat transfer capability was given in the paper by analyzing antifreeze [2] heat transfer capability of the ground source heat pump system and applying the weighted PSVM mode.

A new semi-supervised PSVM classifier

Based on the ideas of LIAM and the U-support vector machine, this paper proposes a new semi-supervised proximal support vector machine, which only requires solving the inverse of an n+1-by-n+1 matrix to obtain the final classification hyperplane just as the PLIAM and is faster and more efficient than other mathematical programming-based methods. The most essential is that this method overcomes the two fundamental drawbacks of the general LIAM semi-supervised support vector classifiers: (1) they included the whole information provided by both the positively and negatively labeled instances through the unlabeled instances that are in its neighborhood in the linear constraints, which greatly added the number of constraints and made the optimization solver more complex; (2) they can only utilize the unlabeled points that are in the neighborhood of a labeled point, which may influence the accuracy of classification. The experiments on public benchmarks indicate that our semi-supervised PSVM classifier is more accurate than the original PLIAM semi-supervised classification method.

Weighted Twin Support Vector Machines with Local Information and its application

A Twin Support Vector Machine (TWSVM), as a variant of a Multisurface Proximal Support Vector Machine via Generalized Eigenvalues (GEPSVM), attempts to improve the generalization of GEPSVM, whose s...

A total least squares proximal support vector classifier for credit risk evaluation

In this paper, a total least squares (TLS) version of proximal support vector machines (PSVM) is proposed for credit risk evaluation. The formulation of this new model is different from the original PSVM model, so a novel iterative algorithm is proposed to solve this model. A simulation test is first implemented on a classic two-spiral dataset, and then an empirical experiment is conducted on two publicly available credit datasets. The experimental results show that the proposed total least squares PSVM (TLS-PSVM) is at least comparable with PSVM and better than other models including standard SVM model.

Weighted Twin Support Vector Machines with Local Information and its application

A Twin Support Vector Machine (TWSVM), as a variant of a Multisurface Proximal Support Vector Machine via Generalized Eigenvalues (GEPSVM), attempts to improve the generalization of GEPSVM, whose solution follows from solving two quadratic programming problems (QPPs), each of which is smaller than in a standard SVM. Unfortunately, the two QPPs still lead to rather high computational costs. Moreover, although TWSVM has better classification performance than GEPSVM, a major disadvantage is it fails to exploit the underlying correlation or similarity information between any pair of data points with the same labels that may be important for classification performance as much as possible. To mitigate the above deficiencies, in this paper, we propose a novel nonparallel plane classifier, called Weighted Twin Support Vector Machines with Local Information (WLTSVM). WLTSVM mines as much underlying similarity information within samples as possible. This method not only retains the superior characteristics of TWSVM, but also has its additional advantages: (1) comparable or better classification accuracy compared to SVM, GEPSVM and TWSVM; (2) taking motivation from standard SVM, the concept of support vectors is retained; (3) more efficient than TWSVM in terms of computational costs; and (4) only one penalty parameter is considered as opposed to two in TWSVM. Finally, experiments on both simulated and real problems confirm the effectiveness of our method.

A feature selection method for nonparallel plane support vector machine classification

Over the past decades, 1-norm techniques based on algorithms are widely used to suppress input features. Quite different from traditional 1-norm support vector machine (SVM), direct 1-norm optimization based on the primal problem of nonparallel plane classifiers like generalized proximal support vector machine, twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) are not capable of generating very sparse solutions that are vital for classification and can make them easier to store and faster to compute. To address the issue, in this paper, we develop a feature selection method for LSTSVM, called a feature selection method for nonparallel plane support vector machine classification (FLSTSVM), which is specially designed for strong feature suppression. We incorporate a Tikhonov regularization term to the objective of LSTSVM, and then minimize its 1-norm measure. Solution of FLSTSVM can follow directly from solving two smaller quadratic programming problems (QPPs) arising from two primal QPPs as opposed to two dual ones in TWSVM. FLSTSVM is capable of generating very sparse solutions. This means that FLSTSVM can reduce input features, for the linear case. When a nonlinear classifier is used, few kernel functions determine the classifier. In addition to having strong feature suppression, the edge of our method still lies in its faster computing time compared to that of TWSVM, Newton Method for Linear Programming SVM (NLPSVM) and LPNewton. Lastly, this algorithm is compared on public data sets, as well as an Exclusive Or (XOR) example.

An overview on twin support vector machines

Twin support vector machines (TWSVM) is based on the idea of proximal SVM based on generalized eigenvalues (GEPSVM), which determines two nonparallel planes by solving two related SVM-type problems, so that its computing cost in the training phase is 1/4 of standard SVM. In addition to keeping the superior characteristics of GEPSVM, the classification performance of TWSVM significantly outperforms that of GEPSVM. However, the stand-alone method requires the solution of two smaller quadratic programming problems. This paper mainly reviews the research progress of TWSVM. Firstly, it analyzes the basic theory and the algorithm thought of TWSVM, then tracking describes the research progress of TWSVM including the learning model and specific applications in recent years, finally points out the research and development prospects.

Inverse matrix-free incremental proximal support vector machine

Traditional Support Vector Machines (SVMs) based learners are commonly regarded as strong classifiers for many learning tasks. Their efficiency for large-scale high dimensional data, however, has shown to be unsatisfactory. Consequently, many alternative SVM solutions exist for large-scale and/or high dimensional data. Among them, proximal support vector machine (PSVM) is a simple but effective SVM classifier. Its incremental version (ISVM) is also available for large-scale data. Nevertheless, the computational efficiency of the ISVM for high dimensional data still needs to be improved, mainly because it requires explicit matrix inversion for updating the decision model. To solve this problem, we propose, in this paper, an inverse matrix-free incremental PSVM (IMISVM) with the following two characteristics. Firstly, IMISVM avoids explicit matrix inversion and hence derives simple formulas for updating model parameters. Secondly, IMISVM achieves faster convergence speed than ISVM. Experimental results on synthetic and real-world data sets confirm that the proposed incremental classifier outperforms ISVM.

Least squares twin support vector machine classification via maximum one-class within class variance

A twin support vector machine (TWSVM), as an effective classification tool, tries to find two non-parallel planes that can be produced by solving two quadratic programming problems (QPPs). The QPPs lead to higher computational costs. The least squares twin SVM (LSTSVM), as a variant of TWSVM, attempts to avoid the above deficiency and obtain two non-parallel planes directly by solving two sets of linear equations. Both TWSVM and LSTSVM operate directly on patterns using two optimizations with constraints and, respectively, use such constraints to estimate the distance between each plane for its own class and patterns of other classes. However, such approaches weaken the geometric interpretation of the generalized proximal SVM (GEPSVM) so that in many Exclusive Or examples for different distributions, they may obtain the worse classification performance. Moreover, in addition to failing to discover the local geometry inside the samples, they are sensitive to outliers. In this paper, inspired by several geometrically motivated learning algorithms and the advantages of LSTSVM, we first propose a new classifier, called LSTSVM classification via maximum one-class within-class variance (MWSVM), which is specially designed for avoiding the aforementioned deficiencies and keeping the advantages of LSTSVM. The new method directly incorporates the one-class within-class variance to the classifier, such that it is expected that the genuine geometric interpretation of GEPSVM can be kept in LSTSVM. Undoubtedly, like LSTSVM, MWSVM may lead to a worse classification performance in many cases, especially when the outliers are present. Therefore, a localized version (LMWSVM) of MWSVM is further proposed to remove the outliers effectively. Another advantage of LMWSVM is that it takes the nearby points which are closest to each other as a training set, such that the MWSVM classifier is determined by smaller size of training samples than that of LSTSVM. Naturally, it can reduce the storage space of LSTSVM, especially when extended to nonlinear cases. Experiments carried out on both toy and real-world problems disclose the effectiveness of both MWSVM and LMWSVM.

Supervised classification of distributed data streams for smart grids

The electricity system inherited from the 19th and 20th centuries has been a reliable but centralized system. With the spreading of local, distributed and intermittent renewable energy resources, top-down central control of the grid no longer meets modern requirements. For these reasons, the power grid has been equipped with smart meters integrating bi-directional communications, advanced power measurement and management capabilities. Smart meters make it possible to remotely turn power on or off to a customer, read usage information, detect a service outage and the unauthorized use of electricity. To fully exploit their capabilities, we foresee the usage of distributed supervised classification algorithms. By gathering data available from meters and other sensors, such algorithms can create local classification models for attack detection, online monitoring, privacy preservation, workload balancing, prediction of energy demand and incoming faults. In this paper we present a decentralized distributed classification algorithm based on proximal support vector machines. The method uses partial knowledge, in form of data streams, to build its local model on each meter. We demonstrate the performance of the proposed scheme on synthetic datasets.

Soft computing approach to fault diagnosis of centrifugal pump

Fault detection and isolation in rotating machinery is very important from an industrial viewpoint as it can help in maintenance activities and significantly reduce the down-time of the machine, resulting in major cost savings. Traditional methods have been found to be not very accurate. Soft computing based methods are now being increasingly employed for the purpose. The proposed method is based on a genetic programming technique which is known as gene expression programming (GEP). GEP is somewhat a new member of the genetic programming family. The main objective of this paper is to compare the classification accuracy of the proposed evolutionary computing based method with other pattern classification approaches such as support vector machine (SVM), Wavelet-GEP, and proximal support vector machine (PSVM). For this purpose, six states viz., normal, bearing fault, impeller fault, seal fault, impeller and bearing fault together, cavitation are simulated on centrifugal pump. Decision tree algorithm is used to select the features. The results obtained using GEP is compared with the performance of Wavelet-GEP, support vector machine (SVM) and proximal support vector machine (PSVM) based classifiers. It is observed that both GEP and SVM equally outperform the other two classifiers (PSVM and Wavelet-GEP) considered in the present study.

Performance Evaluation of PSVM Using Various Combination of Kernel Function for Intrusion Detection System

The internet and computer networks are exposed to an increasing number of security threats. With new types of attacks appearing continuously, developing flexible and adaptive security oriented approaches is a severe challenge. In this context, intrusion detection technique is a valuable technology to protect target systems and networks against malicious activities. But this system doesn't provide the required accuracy. Thus to meet this requirement, this paper proposes an intrusion detection system as a model based on Proximal Support Vector Machines (PSVMs) implemented with various combination of basic kernel functions. PSVM is a light and simple modification of support vector machine. We have implemented PSVM for binary classification of intrusion detection data. For experimental training and testing NSL- KDD dataset is preprocessed using Principle Component Analysis technique. Using proposed classification model, we have achieved up to 79% classification accuracy.

On proximal bilateral-weighted fuzzy support vector machine classifiers

A new approach for classification problems, called proximal bilateral-weighted fuzzy support vector machine, is proposed wherein each input example is treated as belonging to both positive and negative classes with different fuzzy memberships. The assumption of treating every input example belonging to both the classes is very well justified in real world applications. For example, for the study of credit risk assessment a customer can not always be assumed to be absolutely good or bad as he may default or pay his debit at times and therefore he may be treated as belonging to both the classes. Our formulation leads to solving a system of linear equations of size equals to the number of input examples. Computational results of the proposed method on publicly available datasets including two credit risk analysis datasets to that of the standard, proximal and bilateral-weighted fuzzy support vector machine methods clearly demonstrates its efficiency and usefulness.

Extreme Learning Machine for Regression and Multiclass Classification

Due to the simplicity of their implementations, least square support vector machine (LS-SVM) and proximal support vector machine (PSVM) have been widely used in binary classification applications. The conventional LS-SVM and PSVM cannot be used in regression and multiclass classification applications directly, although variants of LS-SVM and PSVM have been proposed to handle such cases. This paper shows that both LS-SVM and PSVM can be simplified further and a unified learning framework of LS-SVM, PSVM, and other regularization algorithms referred to extreme learning machine (ELM) can be built. ELM works for the "generalized" single-hidden-layer feedforward networks (SLFNs), but the hidden layer (or called feature mapping) in ELM need not be tuned. Such SLFNs include but are not limited to SVM, polynomial network, and the conventional feedforward neural networks. This paper shows the following: 1) ELM provides a unified learning platform with a widespread type of feature mappings and can be applied in regression and multiclass classification applications directly; 2) from the optimization method point of view, ELM has milder optimization constraints compared to LS-SVM and PSVM; 3) in theory, compared to ELM, LS-SVM and PSVM achieve suboptimal solutions and require higher computational complexity; and 4) in theory, ELM can approximate any target continuous function and classify any disjoint regions. As verified by the simulation results, ELM tends to have better scalability and achieve similar (for regression and binary class cases) or much better (for multiclass cases) generalization performance at much faster learning speed (up to thousands times) than traditional SVM and LS-SVM.

1-Norm least squares twin support vector machines

During the last few years, nonparallel plane classifiers, such as Multisurface Proximal Support Vector Machine via Generalized Eigenvalues (GEPSVM), and Least Squares TWSVM (LSTSVM), have attracted much attention. However, there are not any modifications of them that have been presented to automatically select the input features. This motivates the rush towards new classifiers. In this paper, we develop a new nonparallel plane classifier, which is designed for automatically selecting the relevant features. We first introduce a Tikhonov regularization (TR) term that is usually used for regularizing least squares into the LSTSVM learning framework, and then convert this formulation to a linear programming (LP) problem. By minimizing an exterior penalty (EP) problem of the dual of the LP formulation and using a fast generalized Newton algorithm, our method yields very sparse solutions, such that it generates a classifier that depends on only a smaller number of input features. In other words, this approach is capable of suppressing input features. This makes the classifier easier to store and faster to compute in the classification phase. Lastly, experiments on both toy and real problems disclose the effectiveness of our method.

Network Intrusion Detection Based on Improved Proximal SVM

Intrusion detection is one of the most essential factors for security infrastructures in network environments, and it is widely used in detecting, identifying and tracking the intruders. To solve the drawback of the SVM algorithm to meet the requirements of the network intrusion detection, we propose network intrusion detection based on improved proximal SVM. Experiment results illustrate the formulation of PSVM greatly simplifies the problem with considerably faster computational time than SVM for network intrusion. This method also can shorten the training time and improve detection performance by improved kernel function.