Twin Bounded Support Vector Machine Research Articles

A novel method for solving universum twin bounded support vector machine in the primal space

A novel method for solving universum twin bounded support vector machine in the primal space

Functional iterative approach for Universum-based primal twin bounded support vector machine to EEG classification (FUPTBSVM)

Functional iterative approach for Universum-based primal twin bounded support vector machine to EEG classification (FUPTBSVM)

SUPPORT VECTOR MACHINE MODELS FOR DIABETIC RETINOPATHY DIAGNOSIS AND GRADING

This work applies five variants of the support vector machine to the classification of the various stages of nonproliferative diabetic retinopathy. Four hundred eye fundus images from the Messidor repository are preprocessed and thirteen features extracted. The features best suited as inputs for support vector machine classification are identified. Initially, binary classification of severe nonproliferative diabetic retinopathy alone versus a normal eye is performed, achieving an accuracy of 97.44% using the standard support vector machine with Gaussian kernel and when optimized for accuracy. When optimized for sensitivity, the twin bounded support vector machine achieves the highest sensitivity of 99.06%. Then multiclass grading into all four stages of nonproliferative diabetic retinopathy is performed. Best performance with regards to four performance metrics, namely accuracy, sensitivity, specificity, and precision is achieved with the twin bounded support vector machine variant, when one-versus-one decision configuration is used in combination with a novel decision strategy that includes accumulated distances from the decision hyperplanes in the decision algorithm. The results compare favorably with data published in the literature.

A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification

In this work, we address the problem of improving the classification performance of machine learning models, especially in the presence of noisy and outlier data. To this end, we first innovatively design a generalized adaptive robust loss function called Vθ(x). Intuitively, Vθ(x) can improve the robustness of the model by selecting different robust loss functions for different learning tasks during the learning process via the adaptive parameter θ. Compared with other robust loss functions, Vθ(x) has some desirable salient properties, such as symmetry, boundedness, robustness, nonconvexity, and adaptivity, making it suitable for a wide range of machine learning applications. Secondly, a new robust semi-supervised learning framework for pattern classification is proposed. In this learning framework, the proposed robust loss function Vθ(x) and capped L2,p-norm robust distance metric are introduced to improve the robustness and generalization performance of the model, especially when the outliers are far from the normal data distributions. Simultaneously, based on this learning framework, the Welsch manifold robust twin bounded support vector machine (WMRTBSVM) and its least-squares version are developed. Finally, two effective iterative optimization algorithms are designed, their convergence is proved, and their complexity is calculated. Experimental results on several datasets with different noise settings and different evaluation criteria show that our methods have better classification performance and robustness. With the Cancer dataset, when there is no noise, the classification accuracy of our proposed methods is 94.17% and 95.62%, respectively. When the Gaussian noise is 50%, the classification accuracy of our proposed methods is 91.76% and 90.59%, respectively, demonstrating that our method has satisfactory classification performance and robustness.

A Novel Method for Solving Universum Twin Bounded Support Vector Machine in the Primal Space

A Novel Method for Solving Universum Twin Bounded Support Vector Machine in the Primal Space

Capped L2,p-Norm Metric Based on Robust Twin Support Vector Machine with Welsch Loss

A twin bounded support vector machine (TBSVM) is a phenomenon of symmetry that improves the performance of the traditional support vector machine classification algorithm. In this paper, we propose an improved model based on a TBSVM, called a Welsch loss with capped L2,p-norm distance metric robust twin bounded support vector machine (WCTBSVM). On the one hand, by introducing the capped L2,p-norm metric in the TBSVM, the problem of the non-sparse output of the regularization term is solved; thus, the generalization and robustness of the TBSVM is improved and the principle of minimizing the structural risk is realized. On the other hand, a bounded, smooth, and non-convex Welsch loss function is introduced to reduce the influence of noise, which further improves the classification performance of the TBSVM. We use a half-quadratic programming algorithm to solve the model non-convexity problem caused by Welsch loss. Therefore, the WCTBSVM is more robust and effective in dealing with noise compared to the TBSVM. In addition, to reduce the time complexity and speed up the convergence of the algorithm, we constructed a least squares version of the WCTBSVM, named the fast WCTBSVM (FWCTBSVM). Experimental results on both UCI and artificial datasets show that our model can show better classification performance on classification problems.

Multi-task twin bounded support vector machine and its safe screening rule

Multi-task twin bounded support vector machine and its safe screening rule

Gas Pipeline Leakage Detection Method Based on IUPLCD and GS-TBSVM

To improve the identification accuracy of gas pipeline leakage and reduce the false alarm rate, a pipeline leakage detection method based on improved uniform-phase local characteristic-scale decomposition (IUPLCD) and grid search algorithm-optimized twin-bounded support vector machine (GS-TBSVM) was proposed. First, the signal was decomposed into several intrinsic scale components (ISC) by the UPLCD algorithm. Then, the signal reconstruction process of UPLCD was optimized and improved according to the energy and standard deviation of the amplitude of each ISC, the ISC components dominated by the signal were selected for signal reconstruction, and the denoised signal was obtained. Finally, the TBSVM was optimized using a grid search algorithm, and a GS-TBSVM model for pipeline leakage identification was constructed. The input of the GS-TBSVM model was the data processed by the IUPLCD algorithm, and the output was the real-time working conditions of the gas pipeline. The experimental results show that IUPLCD can effectively filter the noise in the signal and GS-TBSVM can accurately judge the working conditions of the gas pipeline, with a maximum identification accuracy of 98.4%.

Multiclass Weighted Least Squares Twin Bounded Support Vector Machine for Intelligent Water Leakage Diagnosis

Monitoring the operating status of the pipeline and determining the location of the leak in time are very important to ensure the safe operation of the pipeline. Least squares twin support vector machine (LSTSVM) is a classic fast classification method, which has been used to identify different pipeline conditions. However, LSTSVM assumes that all samples share the same weight when generating the hyperplane, including data points that may be polluted in the sample (i.e., outliers), and outlier samples with equal weights will mislead the generation of the hyperplane. Inspired by the above research, this paper proposes a weighted least squares twin bounded support vector machine based on Gaussian mixture models (GMM), referred to as G-WLSTSVM. The proposed G-WLSTSVM introduces a weight matrix for the objective function through GMM, and assigns a larger weight to the normal samples and a smaller weight to the outliers, which reduces the impact effect of the outliers on the generation of the classification hyperplane. Furthermore, since LSTSVM only considers the empirical risk minimization principle, it may lead to overfitting. The proposed G-WLSTSVM introduces an extra regularization term based on the margin maximization idea to realize the principle of structural risk minimization, which improves the generalization performance of the model. Finally, since the practical problems are mostly multi-classification problems, the G-WLSTSVM for binary classification cannot be satisfied. Therefore, the proposed G-WLSTSVM combined with a "One-versus-One" strategy is extended to handle multi-classification problems, namely the multi-class G-WLSTSVM. We evaluate the effectiveness of the multi-class G-WLSTSVM in identifying different pipeline conditions and localize the identified leakage. Numerical experimental results on several UCI datasets further demonstrate that compared with other related methods, the proposed G-WLSTSVM not only retains the advantages of simplicity and speed of the LSTSVM, but also improves the classification accuracy and generalization ability. The code for this article is available at https://github.com/cmq-456/glstsvm.

Least squares structural twin bounded support vector machine on class scatter

Least squares structural twin bounded support vector machine on class scatter

Bipolar fuzzy based least squares twin bounded support vector machine

Data classification is a key domain of research in real-world applications. One of the big challenges of real-world data classification is to tackle the presence of noise and outliers. In this paper, we handle the computational cost of Bipolar evaluation pairs score value based support vector machine model by formulating two efficient variants, referred to as bipolar fuzzy least squares support vector machines and bipolar fuzzy least squares twin bounded support vector machines, in which the score value is obtained as a bipolar evaluation pair with membership and non-membership functions. The solution of primal problems is attained by solving a system of linear equations that leads to less training time complexity unlike other fuzzy based twin models obtain the solution by handling two quadratic programming problems. Furthermore, our proposed models minimize the impact of noise in the data and facilitate the separation of support vectors from noise. The proposed works have been computationally analyzed on many publicly available real-world benchmark datasets as well as simulated artificial datasets in the non-linear case for different significant levels of noise, namely 0% (noise-free) and 5% (noise-corrupted). In comparison to other similar models, our suggested model is showing phenomenal generalization performance by controlling the negative impact of noise and needs substantially less training time. The results of the proposed models are also validated using quality metrics including AUC, F1-score, G-mean, and Precision Predictive Value.

A lagrangian-based approach for universum twin bounded support vector machine with its applications

A lagrangian-based approach for universum twin bounded support vector machine with its applications

Smooth augmented Lagrangian method for twin bounded support vector machine

<p style='text-indent:20px;'>In this paper, we propose a method for solving the twin bounded support vector machine (TBSVM) for the binary classification. To do so, we use the augmented Lagrangian (AL) optimization method and smoothing technique, to obtain new unconstrained smooth minimization problems for TBSVM classifiers. At first, the augmented Lagrangian method is recruited to convert TBSVM into unconstrained minimization programming problems called as AL-TBSVM. We attempt to solve the primal programming problems of AL-TBSVM by converting them into smooth unconstrained minimization problems. Then, the smooth reformulations of AL-TBSVM, which we called AL-STBSVM, are solved by the well-known Newton's algorithm. Finally, experimental results on artificial and several University of California Irvine (UCI) benchmark data sets are provided along with the statistical analysis to show the superior performance of our method in terms of classification accuracy and learning speed.</p>

Kernel-Target Alignment Based Fuzzy Lagrangian Twin Bounded Support Vector Machine

To improve the generalization performance, we develop a new technique for handling the impacts of outliers using Lagrangian twin bounded SVM (TBSVM) with kernel fuzzy membership values, which is termed kernel-target alignment-based fuzzy Lagrangian twin bounded support vector machine (KTA-FLTBSVM). Here, the objective functions are having L2-norm vectors of the slack variable that leads to the optimization problem more convex and yields a unique global solution. Also, the fuzzy membership values are employing the importance of data samples assigned to each sample to minimize the impacts of outlier and noise. Further, we have suggested a linearly convergent iterative approach to obtain the solution of the problem unlike in place to solve the quadratic programming problem in Twin SVM (TSVM) and TBSVM. To investigate the effectiveness of the proposed KTA-FLTBSVM, the comprehensive experiments demonstrate with other reported models on artificial datasets along with benchmark real-life publicly available datasets. Our KTA-FLTBSVM outperforms to other models in terms of better classification accuracy.

Universum based Lagrangian twin bounded support vector machine to classify EEG signals

Background and objectiveThe detection of brain-related problems and neurological disorders like epilepsy, sleep disorder, and so on is done by using electroencephalogram (EEG) signals which contain noisy signals and outliers. Universum data contains a set of a sample that does not belong to any of the concerned classes and serves as the advanced knowledge about the data distribution. Earlier information has been utilized viably in improving classification performance. Recently a novel universum support vector machine (USVM) was proposed for EEG signal classification and further, a universum twin support vector machine (UTWSVM) was proposed based on USVM to improve the performance. Inspired by USVM and UTWSVM, this paper suggests a novel method called universum based Lagrangian twin bounded support vector machine (ULTBSVM), where universum data is utilized to incorporate the prior information about the data distribution to classify healthy and seizure EEG signals. MethodsIn the proposed ULTBSVM the square of the 2-norm of the slack variables is used to formulate the objective function strongly convex; hence it always gives unique solutions. Unlike twin support vector machine (TWSVM) and universum twin support vector machine (UTWSVM), the proposed ULTBSVM is having regularization terms that follow the structural risk minimization (SRM) principle and enhance the stability in the dual formulations, make the model well-posed and prevents the overfitting problem. Here, interracial EEG data have been considered as universum data to classify healthy and seizure signals. Several feature extraction techniques have been implemented to get important noiseless features. ResultsSeveral EEG datasets, as well as publicly available UCI datasets, are utilized to assess the performance of the proposed method. An analytical comparison has been performed of the proposed method with USVM and UTWSVM to detect seizure and healthy signals and for real-world data, the ULTBSVM is compared with the universum based models as well as TWSVM and the proposed method gives better results in most of the cases as compared to the other methods. ConclusionThe results clearly show that ULTBSVM is a potential method for the classification of EEG signals as well as real-world datasets having interracial data as universum data. Here we have used universum points for the binary class classification problem, but one can extend and use it for multi-class classification problems as well.

On Lagrangian L2-norm pinball twin bounded support vector machine via unconstrained convex minimization

With the introduction of the regularization term in the formulation of the well-known twin support vector machine (TWSVM) for classification , twin bounded support vector machine (TBSVM) method was proposed recently as an improved version by implementing the structural risk minimization principle. However, TBSVM employs hinge loss function and it is sensitive to noise and unstable to re-sampling. Since the pinball loss function related to quantile distance enjoys noise insensitivity property , a novel TBSVM method with squared pinball loss function for classification is proposed. The noise insensitivity and scatter minimization properties are discussed. Our formulation is further simplified as a pair of unconstrained strongly convex minimization problems in the dual space free of matrix inversion terms and having only m variables where m is the number of training examples. As opposed to TWSVM and TBSVM wherein approximate kernel generated surfaces are constructed, kernel trick is applied directly in our formulation and thereby elegant formulation as in the classical support vector machine (SVM) is achieved. Numerical experiments performed on a synthetic and thirteen benchmark datasets with noise where better or comparable generalization performance with faster learning speed by the proposed method confirms its suitability and applicability to problems of interest.

Ensemble of classification models with weighted functional link network

Ensemble classifiers with random vector functional link network have shown improved performance in classification problems. In this paper, we propose two approaches to solve the classification problems. In the first approach, the original input space’s data points are mapped explicitly into a randomized feature space via neural network wherein the weights of the hidden layer are generated randomly. After feature projection, classification models twin bounded support vector machines (SVM), least squares twin SVM, twin k-class SVM, least squares twin k-class SVM and robust energy based least squares twin SVM are trained on the extended features (original features and randomized features). In the second approach, twin bounded support vector machines (SVM), least squares twin SVM, twin k-class SVM, least squares twin k-class SVM and robust energy based least squares twin SVM models are used to generate the weights of the hidden layer architecture and the weights of output layer are optimized via closed form solution. The performance of both the proposed architectures is evaluated on 33 datasets — including datasets from the UCI repository and fisheries data (not in UCI). Both the experimental results and statistical tests conducted demonstrate that the proposed approaches perform significantly better than the other baseline models. We also analyze the effect of the number of enhanced features on the performance of the given models.

Smooth twin bounded support vector machine with pinball loss

The twin support vector machine improves the classification performance of the support vector machine by solving two small quadratic programming problems. However, this method has the following defects: (1) For the twin support vector machine and some of its variants, the constructed models use a hinge loss function, which is sensitive to noise and unstable in resampling. (2) The models need to be converted from the original space to the dual space, and their time complexity is high. To further enhance the performance of the twin support vector machine, the pinball loss function is introduced into the twin bounded support vector machine, and the problem of the pinball loss function not being differentiable at zero is solved by constructing a smooth approximation function. Based on this, a smooth twin bounded support vector machine model with pinball loss is obtained. The model is solved iteratively in the original space using the Newton-Armijo method. A smooth twin bounded support vector machine algorithm with pinball loss is proposed, and theoretically the convergence of the iterative algorithm is proven. In the experiments, the proposed algorithm is validated on the UCI datasets and the artificial datasets. Furthermore, the performance of the presented algorithm is compared with those of other representative algorithms, thereby demonstrating the effectiveness of the proposed algorithm.

Robust twin bounded support vector machines for outliers and imbalanced data

Truncated loss functions are robust to class noise and outliers. A robust twin bounded support vector machine is proposed in this paper that truncates the growth of its loss functions at a pre-specified point, thus, flattens the function that pre-specified score afterwards. Moreover, to make the proposed method capable of handling datasets with different imbalance ratio, cost-sensitive learning is implemented by scaling the total error of the classes based on the number of samples of each class. However, the adopted loss functions take a non-convex structure which does not always assure global optimum. To handle this issue, we suggest concave-convex procedure (CCCP) to ensure global convergence by decomposing the cost functions into additions of one convex and one concave part. The behaviour of the proposed method for varied imbalance ratio is analysed experimentally and depicted graphically. Classification performance of the proposed method in terms of AUC, F-Measure and G-Mean is compared with other related methods, viz. Hinge loss support vector machine (SVM), Ramp loss SVM (RSVM), twin SVM (TWSVM), twin bounded SVM (TBSVM), Pinball loss SVM (pin-SVM), entropy-based fuzzy SVM (EFSVM), non-parallel hyperplane Universum SVM (U-NHSVM), stochastic gradient twin support vector machine (SGTSVM), k-nearest neighbor (KNN)-based maximum margin and minimum volume hyper-sphere machine (KNN-M3VHM) and affinity and class probability-based fuzzy SVM (ACFSVM) on several real-world datasets with imbalance ratio raging from low to high. Further, to establish the significance of the proposed method in pattern classification, pair-wise statistical comparison of the methods is performed based on their average ranks on AUC. Experimental results are convincing.

Decoding of voluntary and involuntary upper-limb motor imagery based on graph fourier transform and cross-frequency coupling coefficients

Objective.Brain-computer interface (BCI) technology based on motor imagery (MI) control has become a research hotspot but continues to encounter numerous challenges. BCI can assist in the recovery of stroke patients and serve as a key technology in robot control. Current research on MI almost exclusively focuses on the hands, feet, and tongue. Therefore, the purpose of this paper is to establish a four-class MI BCI system, in which the four types are the four articulations within the right upper limbs, involving the shoulder, elbow, wrist, and hand. Approach.Ten subjects were chosen to perform nine upper-limb analytic movements, after which the differences were compared in P300, movement-related potentials(MRPS), and event-related desynchronization/event-related synchronization under voluntary MI (V-MI) and involuntary MI (INV-MI). Next, the cross-frequency coupling (CFC) coefficient based on mutual information was extracted from the electrodes and frequency bands with interest. Combined with the image Fourier transform and twin bounded support vector machine classifier, four kinds of electroencephalography data were classified, and the classifier’s parameters were optimized using a genetic algorithm. Main results.The results were shown to be encouraging, with an average accuracy of 93.2% and 92.2% for V-MI and INV-MI, respectively, and over 95% for any three classes and any two classes. In most cases, the accuracy of feature extraction using the proximal articulations as the basis was found to be relatively high and had better performance. Significance.This paper discussed four types of MI according to three aspects under two modes and classed them by combining graph Fourier transform and CFC. Accordingly, the theoretical discussion and classification methods may provide a fundamental theoretical basis for BCI interface applications.