Generalized Eigenvalue Proximal Support Vector Research Articles

A novel robust generalized eigenvalue proximal support vector machine for pattern classification

A novel robust generalized eigenvalue proximal support vector machine for pattern classification

Intuitionistic fuzzy generalized eigenvalue proximal support vector machine

Generalized eigenvalue proximal support vector machine (GEPSVM) has attracted widespread attention due to its simple architecture, rapid execution, and commendable performance. GEPSVM gives equal significance to all samples, thereby diminishing its robustness and efficacy when confronted with real-world datasets containing noise and outliers. In order to reduce the impact of noises and outliers, we propose a novel intuitionistic fuzzy generalized eigenvalue proximal support vector machine (IF-GEPSVM). The proposed IF-GEPSVM assigns the intuitionistic fuzzy score to each training sample based on its location and surroundings in the high-dimensional feature space by using a kernel function. The solution of the IF-GEPSVM optimization problem is obtained by solving a generalized eigenvalue problem. Further, we propose an intuitionistic fuzzy improved generalized eigenvalue proximal support vector machine (IF-IGEPSVM) by solving standard eigenvalue decomposition resulting in simpler optimization problems with less computation cost which leads to an efficient intuitionistic fuzzy-based model. We conduct a comprehensive evaluation of the proposed IF-GEPSVM and IF-IGEPSVM models on UCI and KEEL benchmark datasets. Moreover, to evaluate the robustness of the proposed IF-GEPSVM and IF-IGEPSVM models, label noise is introduced into some UCI and KEEL datasets. The experimental findings showcase the superior generalization performance of the proposed IF-GEPSVM and IF-IGEPSVM models when compared to the existing baseline models, both with and without label noise. Our experimental results, supported by rigorous statistical analyses, confirm the superior generalization abilities of the proposed IF-GEPSVM and IF-IGEPSVM models over the baseline models. Furthermore, we implement the proposed IF-GEPSVM and IF-IGEPSVM models on the USPS recognition dataset, yielding promising results that underscore the models’ effectiveness in practical and real-world applications. The source code of the proposed IF-GEPSVM and IF-IGEPSVM models are available at https://github.com/mtanveer1/IF-GEPSVM.

Multiplane Convex Proximal Support Vector Machine.

As an effective method for XOR problems, generalized eigenvalue proximal support vector machine (GEPSVM) recently has gained widespread attention accompanied with many variants proposed. Although these variants strengthen the classification performance to different extents, the number of fitting hyperplanes, similar to GEPSVM, for each class is still limited to just one. Intuitively, using single hyperplane seems not enough, especially for the datasets with complex feature structures. Therefore, this article mainly focuses on extending the fitting hyperplanes for each class from single one to multiple ones. However, such an extension from the original GEPSVM is not trivial even though, if possible, the elegant solution via generalized eigenvalues will also not be guaranteed. To address this issue, we first make a simple yet crucial transformation for the optimization problem of GEPSVM and then propose a novel multiplane convex proximal support vector machine (MCPSVM), where a set of hyperplanes determined by the features of the data are learned for each class. We adopt a strictly (geodesically) convex objective to characterize this optimization problem; thus, a more elegant closed-form solution is obtained, which only needs a few lines of MATLAB codes. Besides, MCPSVM is more flexible in form and can be naturally and seamlessly extended to the feature weighting learning, whereas GEPSVM and its variants can hardly straightforwardly work like this. Extensive experiments on benchmark and large-scale image datasets indicate the advantages of our MCPSVM.

Multiview Learning With Robust Double-Sided Twin SVM.

Multiview learning (MVL), which enhances the learners' performance by coordinating complementarity and consistency among different views, has attracted much attention. The multiview generalized eigenvalue proximal support vector machine (MvGSVM) is a recently proposed effective binary classification method, which introduces the concept of MVL into the classical generalized eigenvalue proximal support vector machine (GEPSVM). However, this approach cannot guarantee good classification performance and robustness yet. In this article, we develop multiview robust double-sided twin SVM (MvRDTSVM) with SVM-type problems, which introduces a set of double-sided constraints into the proposed model to promote classification performance. To improve the robustness of MvRDTSVM against outliers, we take L1-norm as the distance metric. Also, a fast version of MvRDTSVM (called MvFRDTSVM) is further presented. The reformulated problems are complex, and solving them are very challenging. As one of the main contributions of this article, we design two effective iterative algorithms to optimize the proposed nonconvex problems and then conduct theoretical analysis on the algorithms. The experimental results verify the effectiveness of our proposed methods.

Capped L 1 -Norm Proximal Support Vector Machine

Compared to the standard support vector machine, the generalized eigenvalue proximal support vector machine coped well with the “Xor” problem. However, it was based on the squared Frobenius norm and hence was sensitive to outliers and noise. To improve the robustness, this paper introduces capped L 1 -norm into the generalized eigenvalue proximal support vector machine, which employs nonsquared L 1 -norm and “capped” operation, and further proposes a novel capped L 1 -norm proximal support vector machine, called CPSVM. Due to the use of capped L 1 -norm, CPSVM can effectively remove extreme outliers and suppress the effect of noise data. CPSVM can also be viewed as a weighted generalized eigenvalue proximal support vector machine and is solved through a series of generalized eigenvalue problems. The experimental results on an artificial dataset, some UCI datasets, and an image dataset demonstrate the effectiveness of CPSVM.

Generalized multi-view learning based on generalized eigenvalues proximal support vector machines

Multi-view learning based on generalized eigenvalue proximal support vector machines has brought enormous success by mining the consistency information of two views. Nevertheless, it only aims to handle two-view cases and cannot handle generalized multi-view learning cases (above two views). It also omits the complementarity information among views. In this paper, two generalized multi-view extensions of generalized eigenvalue proximal support vector machines are presented which take advantage of the multi-view co-regularization term to mine the consistency information and the weighted value to mine complementarity information. Experimental results performed on synthetic and real world datasets demonstrate that they can provide higher performance than the relevant two-view classification algorithms.

Generalized Eigenvalue Proximal Support Vector Machine for Functional Data Classification

Functional data analysis has become a research hotspot in the field of data mining. Traditional data mining methods regard functional data as a discrete and limited observation sequence, ignoring the continuity. In this paper, the functional data classification is addressed, proposing a functional generalized eigenvalue proximal support vector machine (FGEPSVM). Specifically, we find two nonparallel hyperplanes in function space, a positive functional hyperplane, and a functional negative hyperplane. The former is closest to the positive functional data and furthest from the negative functional data, while the latter has the opposite properties. By introducing the orthonormal basis, the problem in function space is transformed into the ones in vector space. It should be pointed out that the higher-order derivative information is applied from two aspects. We apply the derivatives alone or the weighted linear combination of the original function and the derivatives. It can be expected that to improve the classification accuracy by using more data information. Experiments on artificial datasets and benchmark datasets show the effectiveness of our FGEPSVM for functional data classification.

Multi-view generalized support vector machine via mining the inherent relationship between views with applications to face and fire smoke recognition

Multiview Generalized Eigenvalue Proximal Support Vector Machines (MvGSVMs) is an effective multi-view classification algorithm, which effectively combines multi-view learning and classification. Then it was found that in the classification learning task, the classifier combined with multi-view learning has a better classification effect than considering only a single view. In order to utilize the multi-view learning framework more fully and accurately, we further research this. We explore the internal relationship between different views between samples to replace the method of connecting different views through distance combinations. We propose a new method named Multi-view Generalized Support Vector Machine via Mining the Inherent Relationship between Views (MRMvGSVM). At the same time, we use the L2,1-norm constraint relationship matrix as a multi-view regularization term to select the most relevant sample data from different views. It not only helps to improve the accuracy of classification but also reduces the influence of extraneous factors to a certain extent and improves the robustness of the algorithm. The effectiveness of the algorithm is proved by theory and experiments on UCI, and Face and Fire Smoke image datasets.

Improved multi-view GEPSVM via Inter-View Difference Maximization and Intra-view Agreement Minimization

Multiview Generalized Eigenvalue Proximal Support Vector Machine (MvGEPSVM) is an effective method for multiview data classification proposed recently. However, it ignores discriminations between different views and the agreement of the same view. Moreover, there is no robustness guarantee. In this paper, we propose an improved multiview GEPSVM (IMvGEPSVM) method, which adds a multi-view regularization that can connect different views of the same class and simultaneously considers the maximization of the samples from different classes in heterogeneous views for promoting discriminations. This makes the classification more effective. In addition, L1-norm rather than squared L2-norm is employed to calculate the distances from each of the sample points to the hyperplane so as to reduce the effect of outliers in the proposed model. To solve the resulting objective, an efficient iterative algorithm is presented. Theoretically, we conduct the proof of the algorithm’s convergence. Experimental results show the effectiveness of the proposed method.

Generalized elastic net Lp-norm nonparallel support vector machine

Generalized eigenvalue proximal support vector machine (GEPSVM) is the first nonparallel support vector machine. Compared to standard support vector machine (SVM), GEPSVM coped with the “Xor” problem well. In this paper, by defining a generalized elastic net regularization which is the combination of the L2-norm and Lq-norm, we propose a generalized elastic net Lp-norm nonparallel proximal support vector machine (GLpNPSVM), where p,q>0. GLpNPSVM measures the distance of a sample to each hyperplane by the Lp-norm, and hence can achieve desired performance by choosing appropriate p. In addition, the generalized elastic net regularization makes GLpNPSVM own good generalization ability. GLpNPSVM is a generalized formulation, and GEPSVM and some of its improvements are special cases of GLpNPSVM. A simple but effective iterative technique is introduced to solve GLpNPSVM, and we prove its convergence for certain p,q>0. Experimental results on different types of contaminated data sets show the effectiveness of GLpNPSVM.

A Trace Lasso Regularized Robust Nonparallel Proximal Support Vector Machine for Noisy Classification

Generalized eigenvalue proximal support vector machine (GEPSVM) and its improvement IGEPSVM are excellent nonparallel classification methods due to their excellent generalization. However, all of them adopt the square L 2 -norm metric to implement their empirical risk or penalty, which is sensitive to noise and outliers. Moreover, in many real-world learning tasks, it is a significant challenge for GEPSVMs when the data appears highly correlated. To alleviate the above issues, in this paper, we propose a novel trace lasso regularized robust nonparallel proximal support vector machine (RNPSVM) for noisy classification. Compared with GEPSVMs, our RNPSVM enjoys the following advantages. First, the empirical risk of RNPSVM is implemented by the robust L 1 -norm metric with a maximum margin criterion. Namely, it aims to maximize the L 1 -norm inter-class distance dispersion while minimizing the L 1 -norm intra-class distance dispersion simultaneously. Second, to capture the sparsity and the underlying correlation of data, a trace lasso (adaptive norm-based training data) is further introduced to regularize RNPSVM. Third, an iterative algorithm is designed to solve the maximization optimization problem of RNPSVM, whose convergence is guaranteed theoretically. The extensive experimental results on both synthetic and real-world noisy datasets demonstrate the effectiveness of RNPSVM.

Multi-View Learning With Robust Generalized Eigenvalue Proximal SVM

Multi-view learning mechanism, which enhances learning performance by training multi-model data sets, is a popular filed in recent years. Multi-view generalized eigenvalue proximal support vector machine (MvGSVM), as a most recently proposed classifier, has been shown to be successful in multi-model classification, which incorporates multi-view learning into classical GEPSVM. However, this method is still based on squared L2-norm distance measure, thus its robustness is not guaranteed in the presence of outliers. To address this problem, we propose a robust multi-view GEPSVM based on Lp-norm minimization and Ls-norm maximization. But, the introduction of the Lp-norm and Ls-norm makes the problem different from the generalized eigenvalue problem. So an efficient iterative algorithm is designed to solve this problem, and we also give the proof of convergence of the algorithm. The performances in extensive experiments demonstrate the effectiveness and robustness of the algorithm.

Multiview Learning With Generalized Eigenvalue Proximal Support Vector Machines.

Generalized eigenvalue proximal support vector machines (GEPSVMs) are a simple and effective binary classification method in which each hyperplane is closest to one of the two classes and as far as possible from the other class. They solve a pair of generalized eigenvalue problems to obtain two nonparallel hyperplanes. Multiview learning considers learning with multiple feature sets to improve the learning performance. In this paper, we propose multiview GEPSVMs (MvGSVMs) which effectively combine two views by introducing a multiview co-regularization term to maximize the consensus on distinct views, and skillfully transform a complicated optimization problem to a simple generalized eigenvalue problem. We also propose multiview improved GEPSVMs (MvIGSVMs), which use the minus instead of ratio in MvGSVMs to measure the differences of the distances between the two classes and the hyperplane and lead to a simpler eigenvalue problem. Linear MvGSVMs and MvIGSVMs are generalized to the nonlinear case by the kernel trick. Experimental results on multiple data sets show the effectiveness of our proposed approaches.

Robust Nonparallel Proximal Support Vector Machine With Lp-Norm Regularization

As a useful classification method, generalized eigenvalue proximal support vector machine (GEPSVM) is recently studied extensively. However, it may suffer from the sensitivity to outliers, since the L2-norm is used as a measure distance. In this paper, based on the robustness of the L1-norm, we propose an improved robust L1-norm nonparallel proximal SVM with an arbitrary Lp-norm regularization (LpNPSVM), where $p>0$ . Compared with GEPSVM, the LpNPSVM is more robust to outliers and noise. A simple but effective iterative technique is introduced to solve the LpNPSVM, and its convergence guarantee is also given when $0 . Experimental results on different types of contaminated data sets show the effectiveness of LpNPSVM. At last, we investigate our LpNPSVM on a real spare parts inspection problem. Computational results demonstrate the effectiveness of the proposed method over the GEPSVM on all the noise data.

Generalized eigenvalue proximal support vector regressor for the simultaneous learning of a function and its derivatives

Generalized eigenvalue proximal support vector regressor (GEPSVR) determines a pair of \(\epsilon\)-insensitive bounding regressors by solving a pair of generalized eigenvalue problem. On the lines of GEPSVR, in this paper we propose a novel regressor for the simultaneous learning of a function and its derivatives, termed as GEPSVR of a Function and its Derivatives. The proposed method is fast as it requires the solution of a pair of generalized eigenvalue problems as compared to the solution of a large Quadratic Programming Problem required in other existing approaches. The experiment results on several benchmark functions of more than one variable proves the efficacy of our proposed method.

Sensorineural hearing loss detection via discrete wavelet transform and principal component analysis combined with generalized eigenvalue proximal support vector machine and Tikhonov regularization

In the past, scholars used various computer vision and artificial intelligence methods to detect brain diseases via magnetic resonance imaging (MRI). In this paper, we proposed a novel system to detect sensorineural hearing loss (SNHL). First, we used three-level bior4.4 wavelet to decompose original brain image. Second, principal component analysis (PCA) was utilized for dimensionality reduction. Third, the generalized eigenvalue proximal support vector machine (GEPSVM) with Tikhonov regularization was employed as the classifier. The 10 repetitions of five-fold cross validation showed our method achieved an overall accuracy of 95.71 %. Our sensitivities over healthy control, left-sided SNHL, and right-sided SNHL are 96.00 %, 95.33 %, and 95.71 %, respectively. The proposed system is promising and effective in SNHL detection. It gives better performance than four state-of-the-art methods.

Weighted multicategory nonparallel planes SVM classifiers

We formulate K-category non-parallel classifiers by using “One-Versus-All” (OVA) approach: Multicategory Generalized Eigenvalue Proximal Support Vector Machine (MGEPSVM) and Multicategory Improved Generalized Eigenvalue Proximal Support Vector Machine (MIGEPSVM). These classifiers generate K nonparallel decision surfaces for K classes where each surface is closest to its corresponding class and the farthest from the rest of the classes. However, in some cases, this approach leads to poor performance due to the resultant of two unbalanced classes. To minimize the effect of unbalanced classes, we proposed Weighted MGEPSVM (WMGEPSVM) and Weighted MIGEPSVM (WMIGEPSVM) where the weight factor is determined by using proposed modified balancing technique. In this paper, we also propose K-category non-parallel classifiers by using “One-Versus-One” (OVO) approach: Multicategory Generalized Eigenvalue Proximal Support Vector Machine (OVO-MGEPSVM) and Multicategory Improved Generalized Eigenvalue Proximal Support Vector Machine (OVO-MIGEPSVM). To check the robustness of the model numerical experiments have been carried out on twelve different benchmark datasets. Experimental results indicate that WMGEPSVM and WMIGEPSVM improve the testing accuracy of MGEPSVM and MIGEPSVM, while maintaining the computation time. WMGEPSVM and MGEPSVM are superior to multi-class SVM, MLSTSVM and comparable with multicategory Proximal Support Vector Machine (PSVM).

Twin support vector machine: theory, algorithm and applications

Twin support vector machine (TWSVM) has gained increasing interest from various research fields recently. In this paper, we aim to report the current state of the theoretical research and practical advances on TWSVM. We first give the basic thought and theory of TWSVM, including the theory of proximal support vector machine, generalized eigenvalue proximal support vector machine, and TWSVM. Then, we focus on the various improvements made to TWSVM, mainly including least squares twin support vector machine, smooth twin support vector machine, regularized twin support vector machine, projection twin support vector machine, and modified TWSVM on the model selection problem. These newly emerging algorithms greatly expand the applications of TWSVM. In recent years, there is a lot of research on application of TWSVM. Next, we list some research on application of TWSVM in detail. Finally, we try to provide a comprehensive view of these advances in TWSVM and discuss the direction of future research.

Detection of Alzheimer's Disease by Three-Dimensional Displacement Field Estimation in Structural Magnetic Resonance Imaging.

Within the past decade, computer scientists have developed many methods using computer vision and machine learning techniques to detect Alzheimer's disease (AD) in its early stages. However, some of these methods are unable to achieve excellent detection accuracy, and several other methods are unable to locate AD-related regions. Hence, our goal was to develop a novel AD brain detection method. In this study, our method was based on the three-dimensional (3D) displacement-field (DF) estimation between subjects in the healthy elder control group and AD group. The 3D-DF was treated with AD-related features. The three feature selection measures were used in the Bhattacharyya distance, Student's t-test, and Welch's t-test (WTT). Two non-parallel support vector machines, i.e., generalized eigenvalue proximal support vector machine and twin support vector machine (TSVM), were then used for classification. A 50 × 10-fold cross validation was implemented for statistical analysis. The results showed that "3D-DF+WTT+TSVM" achieved the best performance, with an accuracy of 93.05 ± 2.18, a sensitivity of 92.57 ± 3.80, a specificity of 93.18 ± 3.35, and a precision of 79.51 ± 2.86. This method also exceled in 13 state-of-the-art approaches. Additionally, we were able to detect 17 regions related to AD by using the pure computer-vision technique. These regions include sub-gyral, inferior parietal lobule, precuneus, angular gyrus, lingual gyrus, supramarginal gyrus, postcentral gyrus, third ventricle, superior parietal lobule, thalamus, middle temporal gyrus, precentral gyrus, superior temporal gyrus, superior occipital gyrus, cingulate gyrus, culmen, and insula. These regions were reported in recent publications. The 3D-DF is effective in AD subject and related region detection.

Pathological brain detection based on wavelet entropy and Hu moment invariants

With the aim of developing an accurate pathological brain detection system, we proposed a novel automatic computer-aided diagnosis (CAD) to detect pathological brains from normal brains obtained by magnetic resonance imaging (MRI) scanning. The problem still remained a challenge for technicians and clinicians, since MR imaging generated an exceptionally large information dataset. A new two-step approach was proposed in this study. We used wavelet entropy (WE) and Hu moment invariants (HMI) for feature extraction, and the generalized eigenvalue proximal support vector machine (GEPSVM) for classification. To further enhance classification accuracy, the popular radial basis function (RBF) kernel was employed. The 10 runs of k-fold stratified cross validation result showed that the proposed "WE + HMI + GEPSVM + RBF" method was superior to existing methods w.r.t. classification accuracy. It obtained the average classification accuracies of 100%, 100%, and 99.45% over Dataset-66, Dataset-160, and Dataset-255, respectively. The proposed method is effective and can be applied to realistic use.