Generalized Eigenvalue Proximal Support Vector Machine Research Articles

Intuitionistic fuzzy generalized eigenvalue proximal support vector machine

Generalized eigenvalue proximal support vector machine (GEPSVM) has attracted widespread attention due to its simple architecture, rapid execution, and commendable performance. GEPSVM gives equal significance to all samples, thereby diminishing its robustness and efficacy when confronted with real-world datasets containing noise and outliers. In order to reduce the impact of noises and outliers, we propose a novel intuitionistic fuzzy generalized eigenvalue proximal support vector machine (IF-GEPSVM). The proposed IF-GEPSVM assigns the intuitionistic fuzzy score to each training sample based on its location and surroundings in the high-dimensional feature space by using a kernel function. The solution of the IF-GEPSVM optimization problem is obtained by solving a generalized eigenvalue problem. Further, we propose an intuitionistic fuzzy improved generalized eigenvalue proximal support vector machine (IF-IGEPSVM) by solving standard eigenvalue decomposition resulting in simpler optimization problems with less computation cost which leads to an efficient intuitionistic fuzzy-based model. We conduct a comprehensive evaluation of the proposed IF-GEPSVM and IF-IGEPSVM models on UCI and KEEL benchmark datasets. Moreover, to evaluate the robustness of the proposed IF-GEPSVM and IF-IGEPSVM models, label noise is introduced into some UCI and KEEL datasets. The experimental findings showcase the superior generalization performance of the proposed IF-GEPSVM and IF-IGEPSVM models when compared to the existing baseline models, both with and without label noise. Our experimental results, supported by rigorous statistical analyses, confirm the superior generalization abilities of the proposed IF-GEPSVM and IF-IGEPSVM models over the baseline models. Furthermore, we implement the proposed IF-GEPSVM and IF-IGEPSVM models on the USPS recognition dataset, yielding promising results that underscore the models’ effectiveness in practical and real-world applications. The source code of the proposed IF-GEPSVM and IF-IGEPSVM models are available at https://github.com/mtanveer1/IF-GEPSVM.

Multiplane Convex Proximal Support Vector Machine.

As an effective method for XOR problems, generalized eigenvalue proximal support vector machine (GEPSVM) recently has gained widespread attention accompanied with many variants proposed. Although these variants strengthen the classification performance to different extents, the number of fitting hyperplanes, similar to GEPSVM, for each class is still limited to just one. Intuitively, using single hyperplane seems not enough, especially for the datasets with complex feature structures. Therefore, this article mainly focuses on extending the fitting hyperplanes for each class from single one to multiple ones. However, such an extension from the original GEPSVM is not trivial even though, if possible, the elegant solution via generalized eigenvalues will also not be guaranteed. To address this issue, we first make a simple yet crucial transformation for the optimization problem of GEPSVM and then propose a novel multiplane convex proximal support vector machine (MCPSVM), where a set of hyperplanes determined by the features of the data are learned for each class. We adopt a strictly (geodesically) convex objective to characterize this optimization problem; thus, a more elegant closed-form solution is obtained, which only needs a few lines of MATLAB codes. Besides, MCPSVM is more flexible in form and can be naturally and seamlessly extended to the feature weighting learning, whereas GEPSVM and its variants can hardly straightforwardly work like this. Extensive experiments on benchmark and large-scale image datasets indicate the advantages of our MCPSVM.

Multiview Learning With Robust Double-Sided Twin SVM.

Multiview learning (MVL), which enhances the learners' performance by coordinating complementarity and consistency among different views, has attracted much attention. The multiview generalized eigenvalue proximal support vector machine (MvGSVM) is a recently proposed effective binary classification method, which introduces the concept of MVL into the classical generalized eigenvalue proximal support vector machine (GEPSVM). However, this approach cannot guarantee good classification performance and robustness yet. In this article, we develop multiview robust double-sided twin SVM (MvRDTSVM) with SVM-type problems, which introduces a set of double-sided constraints into the proposed model to promote classification performance. To improve the robustness of MvRDTSVM against outliers, we take L1-norm as the distance metric. Also, a fast version of MvRDTSVM (called MvFRDTSVM) is further presented. The reformulated problems are complex, and solving them are very challenging. As one of the main contributions of this article, we design two effective iterative algorithms to optimize the proposed nonconvex problems and then conduct theoretical analysis on the algorithms. The experimental results verify the effectiveness of our proposed methods.

Generalized elastic net Lp-norm nonparallel support vector machine

Generalized eigenvalue proximal support vector machine (GEPSVM) is the first nonparallel support vector machine. Compared to standard support vector machine (SVM), GEPSVM coped with the “Xor” problem well. In this paper, by defining a generalized elastic net regularization which is the combination of the L2-norm and Lq-norm, we propose a generalized elastic net Lp-norm nonparallel proximal support vector machine (GLpNPSVM), where p,q>0. GLpNPSVM measures the distance of a sample to each hyperplane by the Lp-norm, and hence can achieve desired performance by choosing appropriate p. In addition, the generalized elastic net regularization makes GLpNPSVM own good generalization ability. GLpNPSVM is a generalized formulation, and GEPSVM and some of its improvements are special cases of GLpNPSVM. A simple but effective iterative technique is introduced to solve GLpNPSVM, and we prove its convergence for certain p,q>0. Experimental results on different types of contaminated data sets show the effectiveness of GLpNPSVM.

A Trace Lasso Regularized Robust Nonparallel Proximal Support Vector Machine for Noisy Classification

Generalized eigenvalue proximal support vector machine (GEPSVM) and its improvement IGEPSVM are excellent nonparallel classification methods due to their excellent generalization. However, all of them adopt the square L 2 -norm metric to implement their empirical risk or penalty, which is sensitive to noise and outliers. Moreover, in many real-world learning tasks, it is a significant challenge for GEPSVMs when the data appears highly correlated. To alleviate the above issues, in this paper, we propose a novel trace lasso regularized robust nonparallel proximal support vector machine (RNPSVM) for noisy classification. Compared with GEPSVMs, our RNPSVM enjoys the following advantages. First, the empirical risk of RNPSVM is implemented by the robust L 1 -norm metric with a maximum margin criterion. Namely, it aims to maximize the L 1 -norm inter-class distance dispersion while minimizing the L 1 -norm intra-class distance dispersion simultaneously. Second, to capture the sparsity and the underlying correlation of data, a trace lasso (adaptive norm-based training data) is further introduced to regularize RNPSVM. Third, an iterative algorithm is designed to solve the maximization optimization problem of RNPSVM, whose convergence is guaranteed theoretically. The extensive experimental results on both synthetic and real-world noisy datasets demonstrate the effectiveness of RNPSVM.

Robust Nonparallel Proximal Support Vector Machine With Lp-Norm Regularization

As a useful classification method, generalized eigenvalue proximal support vector machine (GEPSVM) is recently studied extensively. However, it may suffer from the sensitivity to outliers, since the L2-norm is used as a measure distance. In this paper, based on the robustness of the L1-norm, we propose an improved robust L1-norm nonparallel proximal SVM with an arbitrary Lp-norm regularization (LpNPSVM), where $p>0$ . Compared with GEPSVM, the LpNPSVM is more robust to outliers and noise. A simple but effective iterative technique is introduced to solve the LpNPSVM, and its convergence guarantee is also given when $0 . Experimental results on different types of contaminated data sets show the effectiveness of LpNPSVM. At last, we investigate our LpNPSVM on a real spare parts inspection problem. Computational results demonstrate the effectiveness of the proposed method over the GEPSVM on all the noise data.

Sensorineural hearing loss detection via discrete wavelet transform and principal component analysis combined with generalized eigenvalue proximal support vector machine and Tikhonov regularization

In the past, scholars used various computer vision and artificial intelligence methods to detect brain diseases via magnetic resonance imaging (MRI). In this paper, we proposed a novel system to detect sensorineural hearing loss (SNHL). First, we used three-level bior4.4 wavelet to decompose original brain image. Second, principal component analysis (PCA) was utilized for dimensionality reduction. Third, the generalized eigenvalue proximal support vector machine (GEPSVM) with Tikhonov regularization was employed as the classifier. The 10 repetitions of five-fold cross validation showed our method achieved an overall accuracy of 95.71 %. Our sensitivities over healthy control, left-sided SNHL, and right-sided SNHL are 96.00 %, 95.33 %, and 95.71 %, respectively. The proposed system is promising and effective in SNHL detection. It gives better performance than four state-of-the-art methods.

Dual-Tree Complex Wavelet Transform and Twin Support Vector Machine for Pathological Brain Detection

(Aim) Classification of brain images as pathological or healthy case is a key pre-clinical step for potential patients. Manual classification is irreproducible and unreliable. In this study, we aim to develop an automatic classification system of brain images in magnetic resonance imaging (MRI). (Method) Three datasets were downloaded from the Internet. Those images are of T2-weighted along axial plane with size of 256 × 256. We utilized an s-level decomposition on the basis of dual-tree complex wavelet transform (DTCWT), in order to obtain 12s “variance and entropy (VE)” features from each subband. Afterwards, we used support vector machine (SVM) and its two variants: the generalized eigenvalue proximal SVM (GEPSVM) and the twin SVM (TSVM), as the classifiers. In all, we proposed three novel approaches: DTCWT + VE + SVM, DTCWT + VE + GEPSVM, and DTCWT + VE + TSVM. (Results) The results showed that our “DTCWT + VE + TSVM” obtained an average accuracy of 99.57%, which was not only better than the two other proposed methods, but also superior to 12 state-of-the-art approaches. In addition, parameter estimation showed the classification accuracy achieved the largest when the decomposition level s was assigned with a value of 1. Further, we used 100 slices from real subjects, and we found our proposed method was superior to human reports from neuroradiologists. (Conclusions) This proposed system is effective and feasible.

Detection of Alzheimer's disease by displacement field and machine learning.

Aim. Alzheimer’s disease (AD) is a chronic neurodegenerative disease. Recently, computer scientists have developed various methods for early detection based on computer vision and machine learning techniques.Method. In this study, we proposed a novel AD detection method by displacement field (DF) estimation between a normal brain and an AD brain. The DF was treated as the AD-related features, reduced by principal component analysis (PCA), and finally fed into three classifiers: support vector machine (SVM), generalized eigenvalue proximal SVM (GEPSVM), and twin SVM (TSVM). The 10-fold cross validation repeated 50 times.Results. The results showed the “DF + PCA + TSVM” achieved the accuracy of 92.75 ± 1.77, sensitivity of 90.56 ± 1.15, specificity of 93.37 ± 2.05, and precision of 79.61 ± 2.21. This result is better than or comparable with not only the other proposed two methods, but also ten state-of-the-art methods. Besides, our method discovers the AD is related to following brain regions disclosed in recent publications: Angular Gyrus, Anterior Cingulate, Cingulate Gyrus, Culmen, Cuneus, Fusiform Gyrus, Inferior Frontal Gyrus, Inferior Occipital Gyrus, Inferior Parietal Lobule, Inferior Semi-Lunar Lobule, Inferior Temporal Gyrus, Insula, Lateral Ventricle, Lingual Gyrus, Medial Frontal Gyrus, Middle Frontal Gyrus, Middle Occipital Gyrus, Middle Temporal Gyrus, Paracentral Lobule, Parahippocampal Gyrus, Postcentral Gyrus, Posterior Cingulate, Precentral Gyrus, Precuneus, Sub-Gyral, Superior Parietal Lobule, Superior Temporal Gyrus, Supramarginal Gyrus, and Uncus.Conclusion. The displacement filed is effective in detection of AD and related brain-regions.

Pathological brain detection based on wavelet entropy and Hu moment invariants

With the aim of developing an accurate pathological brain detection system, we proposed a novel automatic computer-aided diagnosis (CAD) to detect pathological brains from normal brains obtained by magnetic resonance imaging (MRI) scanning. The problem still remained a challenge for technicians and clinicians, since MR imaging generated an exceptionally large information dataset. A new two-step approach was proposed in this study. We used wavelet entropy (WE) and Hu moment invariants (HMI) for feature extraction, and the generalized eigenvalue proximal support vector machine (GEPSVM) for classification. To further enhance classification accuracy, the popular radial basis function (RBF) kernel was employed. The 10 runs of k-fold stratified cross validation result showed that the proposed "WE + HMI + GEPSVM + RBF" method was superior to existing methods w.r.t. classification accuracy. It obtained the average classification accuracies of 100%, 100%, and 99.45% over Dataset-66, Dataset-160, and Dataset-255, respectively. The proposed method is effective and can be applied to realistic use.

Sparse Proximal Support Vector Machine with a Specialized Interior-Point Method

Support vector machine (SVM) is a widely used method for classification. Proximal support vector machine (PSVM) is an extension of SVM and a promising method to lead to a fast and simple algorithm for generating a classifier. Motivated by the fast computational efforts of PSVM and the properties of sparse solution yielded by $$\ell _{1}$$ -norm, in this paper, we first propose a PSVM with a cardinality constraint which is eventually relaxed by $$\ell _{1}$$ -norm and leads to a trade-off $$\ell _{1}-\ell _{2}$$ regularized sparse PSVM. Next we convert this $$\ell _{1}-\ell _{2}$$ regularized sparse PSVM into an equivalent form of $$\ell _{1}$$ regularized least squares (LS) and solve it by a specialized interior-point method proposed by Kim et al. (J Sel Top Signal Process 12:1932–4553, 2007). Finally, $$\ell _{1}-\ell _{2}$$ regularized sparse PSVM is illustrated by means of a real-world dataset taken from the University of California, Irvine Machine Learning Repository (UCI Repository). Moreover, we compare the numerical results with the existing models such as generalized eigenvalue proximal SVM (GEPSVM), PSVM, and SVM-Light. The numerical results show that the $$\ell _{1}-\ell _{2}$$ regularized sparse PSVM achieves not only better accuracy rate of classification than those of GEPSVM, PSVM, and SVM-Light, but also a sparser classifier compared with the $$\ell _{1}$$ -PSVM.

Differential Search Algorithm-based Parametric Optimization of Fuzzy Generalized Eigenvalue Proximal Support Vector Machine

Support Vector Machine (SVM) is an effective model for many classification problems. However, SVM needs the solution of a quadratic program which require specialized code. In addition, SVM has many parameters, which affects the performance of SVM classifier. Recently, the Generalized Eigenvalue Proximal SVM (GEPSVM) has been presented to solve the SVM complexity. In real world applications data may affected by error or noise, working with this data is a challenging problem. In this paper, an approach has been proposed to overcome this problem. This method is called DSA-GEPSVM. The main improvements are carried out based on the following: 1) a novel fuzzy values in the linear case. 2) A new Kernel function in the nonlinear case. 3) Differential Search Algorithm (DSA) is reformulated to find near optimal values of the GEPSVM parameters and its kernel parameters. The experimental results show that the proposed approach is able to find the suitable parameter values, and has higher classification accuracy compared with some other algorithms.

A Novel Twin Support Vector Machines with Manifold Regularization

Twin support vector machine (TWSVM), as a variant of the generalized eigenvalue proximal support vector machine (GEPSVM), attempts to improve the generalization of GEPSVM, whose solution follows from solving two quadratic programming problems (QPPs), each of which is smaller than in a standard SVM. Unfortunately, TWSVM fails to fully consider the local geometry structure and the local underlying descriminant information inside the samples that may be important for classification performance and only preserves the global data structure. In this paper, a novel TWSVM with manifold regularization is proposed by introducing the basic idea of the locality preserving within-class scatter matrix (LPWSM) into TWSVM. We termed this method manifold TWSVM (MTWSVM). MTWSVM not only retains the superior characteristics of TWSVM, but also preserves the local geometry structure between samples and shows the local underlying discriminant information. Experimental results confirm the effectiveness of our method.

An overview on nonparallel hyperplane support vector machine algorithms

Support vector machine (SVM) has attracted substantial interest in the community of machine learning. As the extension of SVM, nonparallel hyperplane SVM (NHSVM) classification algorithms have become current researching hot spots in machine learning during the last few years. For binary classification tasks, the idea of NHSVM algorithms is to find a hyperplane for each class, such that each hyperplane is proximal to the data points of one class and far from the data points of the other class. Compared with the classical SVM, NHSVM algorithms have lower computational complexity, work better on XOR problems and can get better generalization performance. This paper reviews three representative NHSVM algorithms, including generalized eigenvalue proximal SVM (GEPSVM), twin SVM (TWSVM) and projection twin SVM (PTSVM), and gives the research progress of them. The aim of this overview is to provide an insightful organization of current developments of NHSVM algorithms, identify their limitations and give suggestions for further research.

Nonparallel Support Vector Machines for Pattern Classification

We propose a novel nonparallel classifier, called nonparallel support vector machine (NPSVM), for binary classification. Our NPSVM that is fully different from the existing nonparallel classifiers, such as the generalized eigenvalue proximal support vector machine (GEPSVM) and the twin support vector machine (TWSVM), has several incomparable advantages: 1) two primal problems are constructed implementing the structural risk minimization principle; 2) the dual problems of these two primal problems have the same advantages as that of the standard SVMs, so that the kernel trick can be applied directly, while existing TWSVMs have to construct another two primal problems for nonlinear cases based on the approximate kernel-generated surfaces, furthermore, their nonlinear problems cannot degenerate to the linear case even the linear kernel is used; 3) the dual problems have the same elegant formulation with that of standard SVMs and can certainly be solved efficiently by sequential minimization optimization algorithm, while existing GEPSVM or TWSVMs are not suitable for large scale problems; 4) it has the inherent sparseness as standard SVMs; 5) existing TWSVMs are only the special cases of the NPSVM when the parameters of which are appropriately chosen. Experimental results on lots of datasets show the effectiveness of our method in both sparseness and classification accuracy, and therefore, confirm the above conclusion further. In some sense, our NPSVM is a new starting point of nonparallel classifiers.

Improved Generalized Eigenvalue Proximal Support Vector Machine

In this letter, we propose an improved version of generalized eigenvalue proximal support vector machine (GEPSVM), called IGEPSVM for short. The main improvements are 1) the generalized eigenvalue decomposition is replaced by the standard eigenvalue decomposition, resulting in simpler optimization problems without the possible singularity. 2) An extra meaningful parameter is introduced, resulting in the stronger classification generalization ability. Experimental results on both the artificial datasets and several benchmark datasets show that our IGEPSVM is superior to GEPSVM in both computation time and classification accuracy.

Local and Global Regularized Twin SVM

The generalized eigenvalue proximal support vector machine (GEPSVM) and twin support vector machine (TWSVM) was proposed by Mangasarian and Jayadeva respectively, which aroused the interest of academia for its less computation cost and better generalization ability. They use the nonparallel hyperplane classifiers to solve the classification problem. Different from traditionally local or global TWSVM methods, a new Twin SVM algorithm called Local and Global Regularized Twin SVM (TWSVMLG) is proposed in this paper. A global regularizer was imposed across local models to smooth the data labels predicted by those local classifiers and avoid overfitting risk for the local classifiers. The classifier could get stronger discriminating ability when exploring local and global information than traditional algorithms. Finally some experimental results are presented to show the effectiveness of our algorithm.

Improvements on Twin Support Vector Machines

For classification problems, the generalized eigenvalue proximal support vector machine (GEPSVM) and twin support vector machine (TWSVM) are regarded as milestones in the development of the powerful SVMs, as they use the nonparallel hyperplane classifiers. In this brief, we propose an improved version, named twin bounded support vector machines (TBSVM), based on TWSVM. The significant advantage of our TBSVM over TWSVM is that the structural risk minimization principle is implemented by introducing the regularization term. This embodies the marrow of statistical learning theory, so this modification can improve the performance of classification. In addition, the successive overrelaxation technique is used to solve the optimization problems to speed up the training procedure. Experimental results show the effectiveness of our method in both computation time and classification accuracy, and therefore confirm the above conclusion further.