Twin Support Vector Machine Research Articles

Capped Linex Metric Twin Support Vector Machine for Robust Classification

In this paper, a novel robust loss function is designed, namely, capped linear loss function . Simultaneously, we give some ideal and important properties of , such as boundedness, nonconvexity and robustness. Furthermore, a new binary classification learning method is proposed via introducing , which is called the robust twin support vector machine (Linex-TSVM). Linex-TSVM can not only reduce the influence of outliers on Linex-SVM, but also improve the classification performance and robustness of Linex-SVM. Moreover, the effect of outliers on the model can be greatly reduced by introducing two regularization terms to realize the structural risk minimization principle. Finally, a simple and efficient iterative algorithm is designed to solve the non-convex optimization problem Linex-TSVM, and the time complexity of the algorithm is analyzed, which proves that the model satisfies the Bayes rule. Experimental results on multiple datasets demonstrate that the proposed Linex-TSVM can compete with the existing methods in terms of robustness and feasibility.

A least squares twin support vector machine method with uncertain data

A least squares twin support vector machine method with uncertain data

EEG signal classification via pinball universum twin support vector machine

EEG signal classification via pinball universum twin support vector machine

TSVM-M[formula omitted]: Twin support vector machine based on multi-order moment matching for large-scale multi-class classification

For multi-class classification, many existing methods, such as multiple weighted linear loss twin support vector machine (MWLTSVM), construct multiple decision hyperplanes by minimizing the positive points loss’s first-order moment (mean), which may lead to sensitivity to outliers. Also, when faced with a large-scale classification problem, how to speed up the process of solving the optimization model is also a challenge. An alternative is to use rectangular kernel technology (RKT) to reduce computational complexity. However, RKT is based on the uniform point selection method, which can be proven to be ineffective in improving classifier performance. To address these problems, a novel classifier under the structure of “one-versus-rest” for multi-class classification is proposed in this paper, named twin support vector machine based on multi-order moment matching (TSVM-M3). When constructing the decision hyperplanes, TSVM-M3 takes the first-order and second-order moments (mean and variance) of positive points loss into consideration and implements this by introducing an adjusting factor into the objective function. A theoretical analysis of the robustness of the proposed TSVM-M3 is also provided. Meanwhile, a novel RKT based on the density-dependent data selection method is proposed for large-scale classification. We demonstrate that the proposed RKT can benefit from reducing modeling error. Experimental results show the effectiveness of the proposed TSVM-M3.

Image splicing detection system using intensity-level multi-fractal dimension feature engineering and twin support vector machine based classifier

Image splicing detection system using intensity-level multi-fractal dimension feature engineering and twin support vector machine based classifier

EEG signal classification using improved intuitionistic fuzzy twin support vector machines

EEG signal classification using improved intuitionistic fuzzy twin support vector machines

Solution path algorithm for twin multi-class support vector machine

The twin support vector machine and its extensions have made great achievements in dealing with binary classification problems. However, it suffers from difficulties in effective solution of multi-classification and fast model selection. This work devotes to the fast regularization parameter tuning algorithm for the twin multi-class support vector machine. Specifically, a novel sample data set partition strategy is first adopted, which is the basis for the model construction. Then, combining the linear equations and block matrix theory, the Lagrangian multipliers are proved to be piecewise linear w.r.t. the regularization parameters, so that the regularization parameters are continuously updated by only solving the break points. Next, Lagrangian multipliers are proved to be 1 as the regularization parameter approaches infinity, thus, a simple yet effective initialization algorithm is devised. Finally, eight kinds of events are defined to seek for the starting event for the next iteration. Extensive experimental results on nine UCI data sets show that the proposed method can achieve comparable classification performance without solving any quadratic programming problem.

Intuitionistic fuzzy twin support vector machines for imbalanced data

The classification of imbalanced datasets is one of the main challenges in machine learning techniques. Support vector machine (SVM), which generates a model biased within the majority class, usually has bad performance on the minority class because this class may be considered incorrectly as noises. Moreover, datasets often include noises and outliers, and SVM cannot effectively deal with those datasets. In this paper, to defeat the aforementioned challenges, we propose intuitionistic fuzzy twin support vector machines for imbalanced data (IFTSVM-ID). The proposed method can easily handle imbalanced datasets in the presence of noises and outliers. A reasonable weighting strategy is offered to deal with imbalanced classes, and a margin-based technique is assigned to reduce the impact of noise and outliers. We formulate the linear and non-linear kernel functions to find two non-parallel hyperplanes. One real-world and thirty-two imbalanced datasets are selected to validate the performance of IFTSVM-ID. The Friedman test and the bootstrap technique with 95% confidence interval are applied to quantify the results statistically. The experimental results show that our proposed method has much better performance in comparison with other similar techniques.

Adaptive robust Adaboost-based twin support vector machine with universum data

Universum, as third class that does not belong to the positive class and negative class, allows to incorporate the prior knowledge into the learning process. A lot of reaserchers confirmed that Universum is helpful in the supervised and semi-supervised learning. Moreover, Universum has already been introduced into the support vector machine (SVM) to enhance the generalisation performance. The twin support vector machine (TSVM) serves as an updated classification algorithm based on SVM, having a fast calculation speed. So we introduce Universum into TWSVM to improve the generalisation performance in the same way. Furthermore, in order to make the generalisation performance better in complex environment, an adaptive robust Adaboost-based twin support vector machine with universum learning (ARABUTWSVM) is put up in this paper. First of all, universum learning is used in TSVM to settle a matter of universum data. In order to make our method more robust, we use various loss functions selected by adaptive parameter θ and an update metric induced by correntropy method for distance measure. Further, so as to make further improvement of the learning effect of our method, the Adaboost method is embeded to ARABUTWSVM. The comprehensive experimentation has been performed on various datasets compared with previous universum learning methods, which show that ARABUTWSVM is an adaptive and robust method.

Nonintrusive Load Monitoring Method Based on Color Encoding and Improved Twin Support Vector Machine

In the process of traditional power load identification, the load information of V-I track is missing, the image similarity of V-I track of some power loads is high and the recognition effect is not good, and the training time of recognition model is too long. In view of the abovementioned situation, this study proposes a power load recognition method based on color image coding and the improved twin support vector machine (ITWSVM). First, based on the traditional voltage–current gray trajectory method, the bilinear interpolation technique is used to solve the pixel discontinuity problem effectively. Considering the complementarity of features, the numerical features are embedded into the gray V-I trajectory by constructing three channels, namely, current (R), voltage (G), and phase (B), so the color V-I image with rich electrical features is obtained. Second, the two-dimension Gabor wavelet is used to extract the texture features of the image, and the dimension is reduced by means of local linear embedding (LLE). Finally, the artificial fish swarm algorithm (AFSA) is used to optimize the twin support vector machine (TWSVM), and the ITWSM is used to train the load recognition model, which greatly enhances the model training speed. Experimental results show that the proposed color V-I image coding method and the ITWSVM classification method, compared with the traditional V-I track image construction method and image classification algorithm, improve the accuracy by 6.12% and reduce the model training time by 1071.23 s.

Hydrological time series forecasting via signal decomposition and twin support vector machine using cooperation search algorithm for parameter identification

Accurate hydrologic forecasting is of vital importance for water resource management and reservoir operation under the changing environments. However, it is difficult for a standalone method to track the complex hydrological time series with highly nonlinear and nonstationary features. To improve the forecasting accuracy, this study proposes a hybrid model using compete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and twin support vector machine (TSVM) optimized by metaheuristic algorithm. Firstly, the CEEMDAN tool is chosen to divide the original runoff data into several relatively stable subcomponents with various resolutions and frequencies. Then, the TSVM forecasting model is established to extract the deep features of each subseries, while the emerging cooperation search algorithm is used to search for satisfying parameter combination. The final forecasting value for the original hydrological time series is formulated by aggregating the outputs of all TSVM models. Long-term streamflow data from several hydrological stations in China’s Yangtze River is used for simulation. Experimental results reveal that the hybrid model significantly outperforms several conventional forecasting models in prediction accuracy. For instance, the proposed method improves the root mean squared error and mean absolute error value with about 36.5% and 41.6% reductions compared with the traditional support vector machine at station B. Thus, a novel and effective evolutionary artificial intelligence model is developed for hydrological forecasting.

Research on Risk Features and Prediction of China’s Crude Oil Futures Market Based on Machine Learning

Facing the rapidly changing domestic and foreign futures markets, how to accurately and immediately predict the price trend of crude oil futures in order to avoid the risks caused by price fluctuations is very important for all participants in the crude oil futures market. Based on the 5-min high-frequency trading data of China’s crude oil futures market in recent 3 years, this paper uses the EMD-MFDFA model combined with multifractal detrended fluctuation analysis (MF-DFA) and empirical mode decomposition unsupervised K-means clustering and Gaussian mixture model (GMM) to identify the risk status of each trading day. Further, Support vector machine (SVM), extreme gradient lifting (XGBoost) and their improved algorithms are used to predict the risk state of China’s crude oil futures market. The empirical results are as follows: first, There are obvious multifractal features in the return rate series of China’s crude oil futures market and its single trading day; Second, compared with the traditional SVM model, the improved Twin Support Vector Machine (TWSVM) based on solving the sample imbalance issue has better prediction ability for China’s crude oil futures risk.; Third, The XGBoost has a great impact on the prediction of China’s crude oil risk, and the Focal-XGBoost with focal loss function performs the best in predicting the risk of China’s crude oil futures market.

Multiclass recognition of AD neurological diseases using a bag of deep reduced features coupled with gradient descent optimized twin support vector machine classifier for early diagnosis

SummaryAlzheimer's disease (AD) is an advanced neurodegenerative disease of the brain that affects the nerve system of brain. Previously, several feature extraction and classification methods were discussed, but that methods provide high over fitting problem, which leads to minimization of detection accuracy. To overcome these issues, the multi class classification of AD diseases using bag of deep feature reduction technique and twin support vector machine classifier (TSVM) optimized with gradient decent optimizer is proposed in this manuscript for classifying the AD disease as severe AD, mild cognitive impairment, healthy control. At first, the input EEG signals are pre‐processed. To decrease the execution time and processing time with feature size, a bag of deep features reduction technique is used. The reduced feature signals are classified by optimized TSVM. The simulation process is implemented in MATLAB environment. The proposed model achieves higher accuracy 33.84%, 28.93%, 33.03%, 27.93%, higher precision 22.87%, 16.97%, 16.97%, and 36.97%, compared with the existing methods, such as piecewise aggregate approximation support vector machine (MCC‐EEG‐PAA‐SVM), convolutional neural network (MCC‐EEG‐CNN), conformal kernel‐based fuzzy support vector machine (MCC‐EEG‐CKF‐SVM), Pearson correlation coefficient‐based feature selection strategy with linear discriminant analysis classifier (MCC‐EEG‐ PCC‐LDA).

Learning Pinball TWSVM efficiently using Privileged Information and its Applications

Expert knowledge plays a vital role in any learning framework. However, in the field of machine learning, an expert's knowledge is rarely utilised. Furthermore, most machine learning methods (support vector machine, SVM-based) use a hinge loss function that is sensitive to noise. Thus, in order to benefit from expert knowledge while reducing noise sensitivity, we propose in this paper a fast and novel Twin Support Vector Machine classifier based on privileged information with a pinball loss function, dubbed Pin-TWSVMPI, where expert knowledge is in the form of privileged information. The proposed Pin-TWSVMPI incorporates privileged information into two nonparallel decision hyperplanes by employing a correction function. Furthermore, we employ the Sequential Minimal Optimization (SMO) technique to get the classifier in order to make computations more efficient and faster, and we have demonstrated its applicability for pedestrian detection and handwritten digit recognition. Furthermore, for UCI datasets, we first construct a process that retrieves privileged information from the dataset's features, which is then used by Pin-TWSVMPI, resulting in improved classification accuracy with a reduced computing time.

Learning Pinball TWSVM efficiently using Privileged Information and their applications

In any learning framework, an expert knowledge always plays a crucial role. But, in the field of machine learning, the knowledge offered by an expert is rarely used. Moreover, machine learning algorithms (SVM based) generally use hinge loss function which is sensitive towards the noise. Thus, in order to get the advantage from an expert knowledge and to reduce the sensitivity towards the noise, in this paper, we propose a fast and novel Twin Support Vector Machine classifier based on privileged information with pinball loss function which has been termed as Pin-TWSVMPI where expert's knowledge is in the form of privileged information. The proposed Pin-TWSVMPI incorporates privileged information by using correcting function so as to obtain two nonparallel decision hyperplanes. Further, in order to make computations more efficient and fast, we use Sequential Minimal Optimization (SMO) technique for obtaining the classifier and have also shown its application for Pedestrian detection and Handwritten digit recognition. Further, for UCI datasets, we first implement a procedure which extracts privileged information from the features of the dataset which are then further utilized by Pin-TWSVMPI to which lead to enhancement in classification accuracy with comparatively lesser computational time.

Multi-view learning with privileged weighted twin support vector machine

By using inter-class and intra-class K-Nearest Neighbors (KNNs), weighted twin support vector machine (WLTSVM) mines as much potential similarity information in samples as possible to improve the common short-coming of nonparallel hyperplane classifiers. Multi-view learning (MVL) has a lot of potential due to the multi-modal datasets that are becoming available. In this paper, we propose a new multi-view learning with privileged weighted twin support vector machine (MPWTSVM). It not only inherits the advantages of WLTSVM but also has its characteristics. Firstly, it enhances generalization ability by mining intra-class information from the same perspective. Secondly, it reduces the redundant constraints with the help of inter-class information, thus improving the running speed. Most importantly, it can follow both the consensus and the complementary principles simultaneously. The consensus principle is realized by minimizing the coupling items of different views in the original objective function. The complementary principle is achieved by establishing privileged information paradigms and MVL. Compared with state-of-the-art MVL methods: SVM-2K, MVTSVM, MCPK, PSVM-2V, MVRDTSVM and MVTHSVM-2C, our model has better accuracy and classification efficiency. Experimental results on numerous datasets prove the effectiveness of the proposed algorithm.

Fast Detection of Diarrhetic Shellfish Poisoning Toxins in Mussels Using NIR Spectroscopy and Improved Twin Support Vector Machines

Diarrhetic shellfish poisoning (DSP) toxins are potent marine biotoxins. It can cause a severe gastrointestinal illness by the consumption of mussels contaminated by DSP toxins. New methods for effectively and rapidly detecting DSP toxins-contaminated mussels are required. In this study, we used near-infrared (NIR) reflection spectroscopy combined with pattern recognition methods to detect DSP toxins. In the range of 950-1700 nm, the spectral data of healthy mussels and DSP toxins-contaminated mussels were acquired. To select optimal waveband subsets, a waveband selection algorithm with a Gaussian membership function based on fuzzy rough set theory was applied. Considering that detecting DSP toxins-contaminated mussels from healthy mussels was an imbalanced classification problem, an improved approach of twin support vector machines (TWSVM) was explored, which is based on a centered kernel alignment. The influences of parameters of the waveband selection algorithm and regularization hyperparameters of the improved TWSVM (ITWSVM) on the performance of models were analyzed. Compared to conventional SVM, TWSVM, and other state-of-the-art algorithms (such as multi-layer perceptron, extreme gradient boosting and adaptive boosting), our proposed model exhibited better performance in detecting DSP toxins and was little affected by the imbalance ratio. For the proposed model, the F-measure reached 0.9886, and detection accuracy reached 98.83%. We explored the physical basis for the detection model by analyzing the relationship between the occurrence of overtone and combination bands and selected wavebands. This study supports NIR spectroscopy as an innovative, rapid, and convenient analytical method to detect DSP toxins in mussels.

Fruit fly optimization algorithm based modified least squares twin SVM for grading of metacarpophalangeal rheumatoid arthritis

In the past few decades, ultrasound (US) has witnessed a drastic progress in the inflammatory joint diseases field. RA (Rheumatoid arthritis) is possibly the pathology, which has gained the most advantages due to this progress, both with respect to timely diagnosis and monitoring of disease course. Training a professional sonographer is costly and time- constrained whereas some studies pay attention towards automated RA grading techniques and itis possible using machine learning (DL) technique, to yield a highly objective, automated and rapid means of RA diagnosis in clinical environments. Hence, effective MLTs (machine learning techniques) that are less time consuming while achieving better performances in quick recognition of RA diseases should be proposed by combining soft computing methods. With this motivation, this work proposes FFO-MLSTSVM (Fruit Fly Optimization Algorithm Based Modified Least Squares Twin Support Vector Machines) method for Grading of MRA (Metacarpophalangeal RA). This technical work makes use of the parameters and texture characteristics pertaining to ROIs (regions of interest).

Fruit fly optimization algorithm based modified least squares twin SVM for grading of metacarpophalangeal rheumatoid arthritis

In the past few decades, ultrasound (US) has witnessed a drastic progress in the inflammatory joint diseases field. RA (Rheumatoid arthritis) is possibly the pathology, which has gained the most advantages due to this progress, both with respect to timely diagnosis and monitoring of disease course. Training a professional sonographer is costly and time- constrained whereas some studies pay attention towards automated RA grading techniques and itis possible using machine learning (DL) technique, to yield a highly objective, automated and rapid means of RA diagnosis in clinical environments. Hence, effective MLTs (machine learning techniques) that are less time consuming while achieving better performances in quick recognition of RA diseases should be proposed by combining soft computing methods. With this motivation, this work proposes FFO-MLSTSVM (Fruit Fly Optimization Algorithm Based Modified Least Squares Twin Support Vector Machines) method for Grading of MRA (Metacarpophalangeal RA). This technical work makes use of the parameters and texture characteristics pertaining to ROIs (regions of interest).

Multi-task nonparallel support vector machine for classification

Direct multi-task twin support vector machine (DMTSVM) explores the shared information between multiple correlated tasks, then it produces better generalization performance. However, it contains matrix inversion operation when solving the dual problems, so it costs much running time. Moreover, kernel trick cannot be directly utilized in the nonlinear case. To effectively avoid above problems, a novel multi-task nonparallel support vector machine (MTNPSVM) including linear and nonlinear cases is proposed in this paper. By introducing ε-insensitive loss instead of square loss in DMTSVM, MTNPSVM effectively avoids matrix inversion operation and takes full advantage of the kernel trick. Theoretical implication of the model is further discussed. To further improve the computational efficiency, the alternating direction method of multipliers (ADMM) is employed when solving the dual problem. The computational complexity and convergence of the algorithm are provided. In addition, the property and sensitivity of the parameter in model are further explored. The experimental results on fifteen benchmark datasets and twelve image datasets demonstrate the validity of MTNPSVM in comparison with the state-of-the-art algorithms. Finally, it is applied to real Chinese Wine dataset, and also verifies its effectiveness.