Twin Support Vector Machine Research Articles

Indefinite twin support vector machine with DC functions programming

Twin support vector machine (TWSVM) is an efficient algorithm for binary classification. However, the lack of the structural risk minimization principle restrains the generalization of TWSVM and the guarantee of convex optimization constraints TWSVM to only use positive semi-definite kernels (PSD). In this paper, we propose a novel TWSVM for indefinite kernel called indefinite twin support vector machine with difference of convex functions programming (ITWSVM-DC). The indefinite TWSVM (ITWSVM) leverages a maximum margin regularization term to improve the generalization of TWSVM and a smooth quadratic hinge loss function to make the model continuously differentiable. The representer theorem is applied to the ITWSVM and the convexity of the ITWSVM is analyzed. In order to address the non-convex optimization problem when the kernel is indefinite, a difference of convex functions (DC) is used to decompose the non-convex objective function into the subtraction of two convex functions and a line search method is applied in the DC algorithm to accelerate the convergence rate. A theoretical analysis illustrates that ITWSVM-DC can converge to a local optimum and extensive experiments on indefinite and positive semi-definite kernels show the superiority of ITWSVM-DC.

Robust truncated L$$_2$$-norm twin support vector machine

Robust truncated L$$_2$$-norm twin support vector machine

Hierarchical Multi Twin Support Vector Machine

In binary classification problems, two classes normally have different tendencies. More complex, the clusters in each class also tend to be different. Traditional algorithms as Support Vector Machine (SVM) or Twin Support Vector Machine (TWSVM) don't sufficiently exploit structural information with cluster granularity of the data, cause of restricts the capability of simulation of data trends. Structural twin support vector machine (S-TWSVM) sufficiently exploits structural information with cluster granularity of one class for learning a represented hyperplane of that class. This makes S-TWSVM's data simulation capabilities better than TWSVM. However, for the data type that each class consists of clusters of different trends, the capability of simulation of S-TWSVM is restricted. In this paper, we propose a new Hierarchical Multi Twin Support Vector Machine (called HM-TWSVM) for classification problem with each cluster-vs-class strategy. HM-TWSVM overcomes the limitations of S-TWSVM. Experiment results show that HM-TWSVM could describe the tendency of each cluster.

Class imbalance learning using fuzzy ART and intuitionistic fuzzy twin support vector machines

The classification in imbalanced datasets is one of the main problems for machine learning techniques. Support vector machine (SVM) is biased to the majority class samples, and the minority class samples may incorrectly be considered as noise. Therefore, SVM has poor predictive accuracy for imbalanced datasets and generates inaccurate classification models. Existing class imbalance learning (CIL) techniques can make SVM less sensitive to class imbalance, but these methods suffer from issues related to noise and outliers. Moreover, despite the solid theoretical basis and good classification performance, SVM is not appropriate for the classification of large-scale datasets because the training complexity of SVM is closely related to the dataset size. Class imbalance learning (CIL) using Fuzzy adaptive resonance theory (ART) and intuitionistic fuzzy twin SVM (CIL-FART-IFTSVM), which can be applied to address the class imbalance issue in the presence of noise and outliers and large scale datasets, is proposed to overcome these substantial difficulties. In this method, we modify the distribution of the datasets using fuzzy adaptive resonance theory (Fuzzy ART) as a clustering method to overcome the imbalance problem. Then, after data reduction, IFTSVM is utilized to find excellent non-parallel hyperplanes in the generated data points. Finally, a coordinate descent system with shrinking by an active set is applied to reduce the computational complexity. Forty-five imbalanced datasets are considered to validate the performance of the proposed CIL-FART-IFTSVM method. The Friedman test and the bootstrap technique with 95% confidence intervals are applied to quantify the results statistically. The experimental results indicate that the method proposed in this paper has a better performance compared with other methods, and the training time is significantly better than that of other classifiers for large-scale datasets.

Extraction of the Major Features of Brain Signals using Intelligent Networks

The brain-computer interface is considered one of the main tools for implementing and designing smart medical software. The analysis of brain signal data, called EEG, is one of the main tasks of smart medical diagnostic systems. While EEG signals have many components, one of the most important brain activities pursued is the P300 component. Detection of this component can help detect abnormalities and visualize the movement of organs of the body. In this research, a new method for processing EEG signals is proposed with the aim of detecting the P300 component. Major features were extracted from the BCI Competition IV EEG data set in a number of steps, i.e. normalization with the purpose of noise reduction using a median filter, feature extraction using a recurrent neural network, and classification using Twin Support Vector Machine. Then, a series of evaluation criteria were used to validate the proposed approach and compare it with similar methods. The results showed that the proposed approach has high accuracy.

Capped [formula omitted]-norm metric based robust least squares twin support vector machine for pattern classification

Least squares twin support vector machine (LSTSVM) is an effective and efficient learning algorithm for pattern classification. However, the distance in LSTSVM is measured by squared L2-norm metric that may magnify the influence of outliers. In this paper, a novel robust least squares twin support vector machine framework is proposed for binary classification, termed as CL2,p-LSTSVM, which utilizes capped L2,p-norm distance metric to reduce the influence of noise and outliers. The goal of CL2,p-LSTSVM is to minimize the capped L2,p-norm intra-class distance dispersion, and eliminate the influence of outliers during training process, where the value of the metric is controlled by the capped parameter, which can ensure better robustness. The proposed metric includes and extends the traditional metrics by setting appropriate values of p and capped parameter. This strategy not only retains the advantages of LSTSVM, but also improves the robustness in solving a binary classification problem with outliers. However, the nonconvexity of metric makes it difficult to optimize. We design an effective iterative algorithm to solve the CL2,p-LSTSVM. In each iteration, two systems of linear equations are solved. Simultaneously, we present some insightful analyses on the computational complexity and convergence of algorithm. Moreover, we extend the CL2,p-LSTSVM to nonlinear classifier and semi-supervised classification. Experiments are conducted on artificial datasets, UCI benchmark datasets, and image datasets to evaluate our method. Under different noise settings and different evaluation criteria, the experiment results show that the CL2,p-LSTSVM has better robustness than state-of-the-art approaches in most cases, which demonstrates the feasibility and effectiveness of the proposed method.

Artificial Intelligence Analysis of EEG Amplitude in Intensive Heart Care.

This article first studied the morphological characteristics of the EEG for intensive cardiac care; that is, based on the analysis of the mechanism of disease diagnosis and treatment, a signal processing and machine learning model was constructed. Then, the methods of signal preprocessing, signal feature extraction, new neural network model structure, training mechanism, optimization algorithm, and efficiency are studied, and experimental verification is carried out for public data sets and clinical big data. Then, the principle of intensive cardiac monitoring, the mechanism of disease diagnosis, the types of arrhythmia, and the characteristics of the typical signal are studied, and the rhythm performance, individual variability, and neurophysiological basis of electrical signals in intensive cardiac monitoring are researched. Finally, the automatic signal recognition technology is studied. In order to improve the training speed and generalization ability, a multiclassification model based on Least Squares Twin Support Vector Machine (LS-TWIN-SVM) is proposed. The computational complexity of the classification model algorithm is compared, and intelligence is adopted. The optimization algorithm selects the parameters of the classifier and uses the EEG signal to simulate the model. Support Vector Machines and their improved algorithms have achieved the ultimum in shallow neural networks and have achieved good results in the classification and recognition of bioelectric signals. The LS-TWIN-SVM algorithm proposed in this paper has achieved good results in the classification and recognition of bioelectric signals. It can perform bioinformatics processing on intensive cardiac care EEG signals, systematically biometric information, diagnose diseases, the real-time detection, auxiliary diagnosis, and rehabilitation of patients.

An Improvement Of Least Square - Twin Support Vector Machine

n binary classification problems, two classes of data seem tobe different from each other. It is expected to be more complicated dueto the number of data points of clusters in each class also be different.Traditional algorithms as Support Vector Machine (SVM), Twin Support Vector Machine (TSVM), or Least Square Twin Support VectorMachine (LSTSVM) cannot sufficiently exploit information about thenumber of data points in each cluster of the data. Which may be effectto the accuracy of classification problems. In this paper, we proposes anew Improved Least Square - Support Vector Machine (called ILS-SVM)for binary classification problems with a class-vs-clusters strategy. Experimental results show that the ILS-SVM training time is faster thanthat of TSVM, and the ILS-SVM accuracy is better than LSTSVM andTSVM in most cases.

Universum based Lagrangian twin bounded support vector machine to classify EEG signals

Background and objectiveThe detection of brain-related problems and neurological disorders like epilepsy, sleep disorder, and so on is done by using electroencephalogram (EEG) signals which contain noisy signals and outliers. Universum data contains a set of a sample that does not belong to any of the concerned classes and serves as the advanced knowledge about the data distribution. Earlier information has been utilized viably in improving classification performance. Recently a novel universum support vector machine (USVM) was proposed for EEG signal classification and further, a universum twin support vector machine (UTWSVM) was proposed based on USVM to improve the performance. Inspired by USVM and UTWSVM, this paper suggests a novel method called universum based Lagrangian twin bounded support vector machine (ULTBSVM), where universum data is utilized to incorporate the prior information about the data distribution to classify healthy and seizure EEG signals. MethodsIn the proposed ULTBSVM the square of the 2-norm of the slack variables is used to formulate the objective function strongly convex; hence it always gives unique solutions. Unlike twin support vector machine (TWSVM) and universum twin support vector machine (UTWSVM), the proposed ULTBSVM is having regularization terms that follow the structural risk minimization (SRM) principle and enhance the stability in the dual formulations, make the model well-posed and prevents the overfitting problem. Here, interracial EEG data have been considered as universum data to classify healthy and seizure signals. Several feature extraction techniques have been implemented to get important noiseless features. ResultsSeveral EEG datasets, as well as publicly available UCI datasets, are utilized to assess the performance of the proposed method. An analytical comparison has been performed of the proposed method with USVM and UTWSVM to detect seizure and healthy signals and for real-world data, the ULTBSVM is compared with the universum based models as well as TWSVM and the proposed method gives better results in most of the cases as compared to the other methods. ConclusionThe results clearly show that ULTBSVM is a potential method for the classification of EEG signals as well as real-world datasets having interracial data as universum data. Here we have used universum points for the binary class classification problem, but one can extend and use it for multi-class classification problems as well.

Efficient heart disease diagnosis based on twin support vector machine

Efficient heart disease diagnosis based on twin support vector machine

Intuitionistic Fuzzy Laplacian Twin Support Vector Machine for Semi-supervised Classification

In general, data contain noises which come from faulty instruments, flawed measurements or faulty communication. Learning with data in the context of classification or regression is inevitably affected by noises in the data. In order to remove or greatly reduce the impact of noises, we introduce the ideas of fuzzy membership functions and the Laplacian twin support vector machine (Lap-TSVM). A formulation of the linear intuitionistic fuzzy Laplacian twin support vector machine (IFLap-TSVM) is presented. Moreover, we extend the linear IFLap-TSVM to the nonlinear case by kernel function. The proposed IFLap-TSVM resolves the negative impact of noises and outliers by using fuzzy membership functions and is a more accurate reasonable classifier by using the geometric distribution information of labeled data and unlabeled data based on manifold regularization. Experiments with constructed artificial datasets, several UCI benchmark datasets and MNIST dataset show that the IFLap-TSVM has better classification accuracy than other state-of-the-art twin support vector machine (TSVM), intuitionistic fuzzy twin support vector machine (IFTSVM) and Lap-TSVM.

Pinball Loss Twin Support Vector Clustering

Twin Support Vector Clustering (TWSVC) is a clustering algorithm inspired by the principles of Twin Support Vector Machine (TWSVM). TWSVC has already outperformed other traditional plane based clustering algorithms. However, TWSVC uses hinge loss, which maximizes shortest distance between clusters and hence suffers from noise-sensitivity and low re-sampling stability. In this article, we propose Pinball loss Twin Support Vector Clustering (pinTSVC) as a clustering algorithm. The proposed pinTSVC model incorporates the pinball loss function in the plane clustering formulation. Pinball loss function introduces favorable properties such as noise-insensitivity and re-sampling stability. The time complexity of the proposed pinTSVC remains equivalent to that of TWSVC. Extensive numerical experiments on noise-corrupted benchmark UCI and artificial datasets have been provided. Results of the proposed pinTSVC model are compared with TWSVC, Twin Bounded Support Vector Clustering (TBSVC) and Fuzzy c-means clustering (FCM). Detailed and exhaustive comparisons demonstrate the better performance and generalization of the proposed pinTSVC for noise-corrupted datasets. Further experiments and analysis on the performance of the above-mentioned clustering algorithms on structural MRI (sMRI) images taken from the ADNI database, face clustering, and facial expression clustering have been done to demonstrate the effectiveness and feasibility of the proposed pinTSVC model.

An Improvement Of Least Square - Twin Support Vector Machine

In binary classification problems, two classes of data seem to be different from each other. It is expected to bemore complicated due to the number of data points of clusters in each class also be different. Traditional algorithmsas Support Vector Machine (SVM), Twin Support Vector Machine (TSVM), or Least Square Twin Support VectorMachine (LSTSVM) cannot sufficiently exploit information about the number of data points in each cluster of the data.Which may be effect to the accuracy of classification problems. In this paper, we propose a new Improvement LeastSquare - Support Vector Machine (called ILS-SVM) for binary classification problems with a class-vs-clusters strategy.Experimental results show that the ILS-SVM training time is faster than that of TSVM, and the ILS-SVM accuracy isbetter than LSTSVM and TSVM in most cases.

Pinball loss-based multi-task twin support vector machine and its safe acceleration method

Pinball loss-based multi-task twin support vector machine and its safe acceleration method

R-CTSVM+: Robust capped L1-norm twin support vector machine with privileged information

In the new paradigm, learning using privileged information (LUPI) creates a more informative strategy for tasks to achieve better prediction. SVM based methods including SVM+ and TSVM+, have achieved considerable success in LUPI on the clean data. However, these methods typically suffer from the noise and outliers contained in the data, which leads to larger fluctuations in performance. To handle this problem, in this paper, we propose a novel Robust Capped L1-norm Twin Support Vector Machine with Privileged Information (R-CTSVM+). The proposed pair of regularization functions (up- and down-bound) can definitely help to increase the learning model’s tolerance to disturbance, because the up-bound function aims to maximize the lower bound of the perturbation of both main feature and privilege feature, meanwhile, the down-bound function aims to minimize the upper bound of the perturbation of both main feature and privilege feature. Moreover, as the widely employed squared L2-norm distance typically exaggerates the impact of outliers, we adopt the capped L1 regularized distance to further guarantee the robustness of the model. We present that the resulted optimization problem is theoretically converged and can be solved using an effective alternating optimization procedure. Experimental results on benchmark datasets indicate that the proposed robust model can produce superior performance in the case where data samples contain much noise and outliers.

A robust projection twin support vector machine with a generalized correntropy-based loss

A robust projection twin support vector machine with a generalized correntropy-based loss

Large-Scale Least Squares Twin SVMs

In the last decade, twin support vector machine (TWSVM) classifiers have achieved considerable emphasis on pattern classification tasks. However, the TWSVM formulation still suffers from the following two shortcomings: (1) TWSVM deals with the inverse matrix calculation in the Wolfe-dual problems, which is intractable for large-scale datasets with numerous features and samples, and (2) TWSVM minimizes the empirical risk instead of the structural risk in its formulation. With the advent of huge amounts of data today, these disadvantages render TWSVM an ineffective choice for pattern classification tasks. In this article, we propose an efficient large-scale least squares twin support vector machine (LS-LSTSVM) for pattern classification that rectifies all the aforementioned shortcomings. The proposed LS-LSTSVM introduces different Lagrangian functions to eliminate the need for calculating inverse matrices. The proposed LS-LSTSVM also does not employ kernel-generated surfaces for the non-linear case, and thus uses the kernel trick directly. This ensures that the proposed LS-LSTSVM model is superior to the original TWSVM and LSTSVM. Lastly, the structural risk is minimized in LS-LSTSVM. This exhibits the essence of statistical learning theory, and consequently, classification accuracy on datasets can be improved due to this change. The proposed LS-LSTSVM is solved using the sequential minimal optimization (SMO) technique, making it more suitable for large-scale problems. We further proved the convergence of the proposed LS-LSTSVM. Exhaustive experiments on several real-world benchmarks and NDC-based large-scale datasets demonstrate that the proposed LS-LSTSVM is feasible for large datasets and, in most cases, performed better than existing algorithms.

Automated Diagnosis system for detection of the pathological brain using Fast version of Simplified Pulse-Coupled Neural Network and Twin Support Vector Machine

Brain abnormalities are neurological disorders of the human nervous system that contain biochemical, electrical, and structural changes in the brain and spinal cord. However, such changes produce diverse symptoms like paralysis, amnesia, and muscle weakness. The diagnosis of these abnormalities is crucial for treatment planning in the early stage to limit the progression of diseases. The brain Magnetic Resonance (MR) images are extensively used for treatment planning, but manually diagnosis of MR images is a time-consuming, expensive, and cumbersome task. Hence, in this paper, we have proposed the automated Computer-Aided Diagnosis (CAD) system for classification of brain MR images. These images are skill-stripped for removing the irrelevant tissues that improve the quality of images. We have developed the Fast version of Simplified Pulse-Coupled Neural Network (F-SPCNN) to segment the region of interest. Further, the features are extracted from the segmented images by using the Ripplet Transform (RT). Subsequently, Probabilistic Principal Component Analysis (PPCA) is employed for reducing the dimensionality of features. Finally, Twin Support Vector Machine (TWSVM) is applied for classification of brain MR images. The extensive simulation results on three standard datasets, e.g., DS-66, DS-160, and DS-255, demonstrate that the proposed method achieves better performance than the state-of-the-art methods.

Stellar spectra classification with twin hypersphere model

With the increase of stellar spectra, how to automatically classify these spectra have attracted astronomer's attention. Support Vector Machine (SVM), as a typical classifier, has widely used in stellar spectra classification. Due to its limited performance in various classification problems and higher training time, a model with a pair of hyperspheres named Twin Hypersphere Model (THM), proposed by Peng and Xu, is utilized for stellar spectra classification in this paper. In THM, the samples in one hypersphere is far from another according to the Euclidean distance. The comparative experiments with SVM and Twin Support Vector Machine (TWSVM) on the SDSS datasets shows that the THM model gives the best classification accuracy of 0.8836 for type F, 0.9446 for type G, and 0.9509 for type K, which are better than the classification accuracies of 0.8000, 0.8484, 0.8911 obtained by SVM and 0.8413, 0.8699, 0.9109 obtained by TWSVM. It can be concluded that THM perform better than traditional techniques such as SVM and TWSVM on the K-, F-, G- type stellar spectra classification.

A new method of roundness error evaluation based on twin support vector machines

The geometric error determines the product quality and function to a certain extent. Among them, the roundness error is one of the important indicators for evaluating the geometric error of shaft parts. With the increase of industrial requirements, there are higher requirements for the accuracy and efficiency of roundness error evaluation. However, most of the traditional roundness error evaluation models used in the industry can no longer meet the needs of current industrial processing in terms of efficiency and accuracy. This paper proposes a new roundness error evaluation method based on twin support vector machines (TWSVMs). First, according to the roundness error evaluation and the TWSVMs theory, the roundness error evaluation model with the TWSVMs is obtained. Then, experimental research and analysis are carried out, and the accuracy and efficiency of the traditional roundness evaluation method and the new method are compared. The research results show that the new roundness evaluation method based on the TWSVMs proposed in this paper can efficiently and accurately evaluate the roundness error, and can be applied to the online evaluation of the roundness error in industrial processing to improve the processing efficiency.