Multi-class Least Square Support Vector Machine Research Articles

Improved fuzzy sparse multi-class least squares support vector machine

The classification problem is a key area of research in machine learning. The Least Squares Support Vector Machine (LSSVM) is an important classifier that is commonly used to solve classification problems. Its widespread use stems from its replacement of the inequality constraint in the Support Vector Machine (SVM) with the equality constraint, which transforms the convex quadratic programming (QP) problem of SVM into the solution of linear equations. However, when dealing with multi-class classification problems, LSSVM faces the challenges of lack of sparsity and sample noises, which can negatively impact its performance. Based on the modeling characteristics and data distribution of the multi-class LSSVM model, this paper proposes two improvements and establishes an improved fuzzy sparse multi-class least squares support vector machine (IF-S-M-LSSVM). The first improvement adopts a non-iterative sparse algorithm, which can delete training sample points to different degrees by adjusting the sparse ratio. The second improvement addresses the impact of sample noise on determining the optimal hyperplane by adding a fuzzy membership degree based on sample density. The advantages of the new model, in terms of training speed and classification accuracy, are verified through UCI machine learning standard data set experiments. Finally, the statistical significance of the IF-S-M-LSSVM model is tested using the Friedman and Bonferroni-Dunn tests.

Incipient Chiller Fault Diagnosis Using an Optimized Least Squares Support Vector Machine With Gravitational Search Algorithm

Operational faults in centrifugal chillers will lead to high energy consumption, poor indoor thermal comfort, and low operational safety, and thus it is of significance to detect and diagnose the anomalies timely and effectively, especially for those at their incipient stages. The least squares support vector machine (LSSVM) has been regarded as an effective algorithm for multiclass classification. One of the most difficult issues in LSSVM is parameter tuning. Therefore, this paper reports a development of a gravitational search algorithm (GSA) optimized LSSVM method for incipient fault diagnosis in centrifugal chillers. Considering the inadequacies of conventional principle component analysis (PCA) algorithm for nonlinear data transformation, kernel principle component analysis (KPCA) was firstly employed to reduce the dimensionality of the original input data. Secondly, an optimized “one against one” multi-class LSSVM classifier was developed and its penalty constant and kernel bandwidth were tuned by GSA. Based on the fault samples of seven typical faults at their incipient stages in chillers from ASHRAE RP 1043, the proposed GSA optimized LSSVM fault diagnostic model was trained and validated. For the purpose of demonstrating the priority of the proposed fault diagnosis method, the obtained results were compared to that of using the LSSVM classifier optimized by another two algorithms, namely, the conventional cross-validation method and particle swarm optimizer. Results showed that the best fault diagnosis performance could be achieved using the proposed GSA-LSSVM classifier. The overall average fault diagnosis accuracy for the least severity faults was reported over 95%.

2-D Compact Variational Mode Decomposition- Based Automatic Classification of Glaucoma Stages From Fundus Images

Glaucoma is one of the leading causes of vision loss worldwide. It leads to reduced quality of life for individuals and substantial economic loss for society. This problem can be reduced by the early and reliable diagnosis of glaucoma. The traditional instrument-based methods are nonautomated and laborious. Recently, many computer-based approaches have been proposed for glaucoma detection. However, none of the existing approaches can be efficiently used for the classification of glaucoma stages. In this study, we proposed a novel method to classify the glaucoma stages (healthy, early-stage, and advanced-stage) using a 2-D compact variational mode decomposition (2-D-C-VMD) algorithm. In this work, the preprocessed input images are first decomposed into several variational modes (VMs) employing 2-D-C-VMD. Next, various features, namely, Kapur entropy (KE), Renyi entropy (RE), Shannon entropy (SE), Yager entropy (YE), energy (En), and fractal dimension (FD) features, which are extracted from the first VM. Then, linear discriminant analysis (LDA) has been used for dimensionality reduction. Finally, a trained multiclass least-squares-support vector machine (MC-LS-SVM) classifier has been utilized for classification purpose. The proposed approach has been tested on two different public glaucoma database. Our method achieved the highest classification accuracy of 98.11% with tenfold cross-validation. The experimental results show that the proposed approach performed far better as compared to state-of-the-art approaches.

Automatic Classification of Glaucoma Stages Using Two-Dimensional Tensor Empirical Wavelet Transform

Glaucoma is a chronic eye disease, causing damage to the optic nerve; it may cause permanent vision loss. The conventional instrument methods for glaucoma detection are manual and time-consuming. Many approaches have recently been proposed for automatic glaucoma classification using retinal fundus images. However, none of the existing methods can efficiently use for early-stage glaucoma detection. In this letter, we proposed a novel method for glaucoma classification based on the newly introduced two-dimensional tensor empirical wavelet transform (2D-T-EWT). In this study, the pre-processed images are decomposed into sub-band images (SBIs) using 2D-T-EWT. Then, texture-based grey level co-occurrence matrix (GLCM), chip histogram, and moment invariant features have been extracted from decomposed SBIs. Afore, robust features have been selects and ranked using the student's t -test algorithm. Finally, trained multi-class least squares-support vector machine (MC-LS-SVM) classifier has been used for the classification. The experimental results show that our method outperformed state-of-the-art approaches for glaucoma classification. The proposed method achieved the highest classification accuracy of 93.65% using tenfold cross-validation.

A Method for Diagnosing the State of Insulation Paper in Traction Transformer Based on FDS Test and CS-DQ Algorithm

Traction transformer is vital equipment in high-speed railway. Insulation status decides its safety and reliability, and the frequency-domain dielectric spectrum (FDS) test is one of the most effective methods reflecting the changing of insulation status. For field applications, the following problems should be addressed: 1) how to obtain the result of paper insulation from the combined result of the oil–paper insulation system and 2) how to discriminate the defects of insulation paper between aging and damp. In this article, the first problem was transferred to a nonlinear equation set, and a cuckoo search algorithm optimized by the differential evolution algorithm and the quadratic interpolation (QI) method (CS-DQ algorithm) was proposed to solve it. Then, the insulation states were discriminated by establishing a multiclass least-squares support vector machine (LS-SVM) model, in which the CS-DQ algorithm was also used. Finally, a diagnostic approach for the insulation paper in the traction transformer was proposed. The results in the laboratory show that the pure result of insulation paper can be obtained, and the insulation states can be discriminated effectively by using the proposed approach. Meanwhile, the proposed CS-DQ algorithm has a better performance than the conventional CS algorithm. The results of field testing also verify the proposed approach.

Adaptive Tunable Q Wavelet Transform-Based Emotion Identification

Emotion is a neuronic transient that drives a person to a certain action. Emotion recognition from electroencephalogram (EEG) signals plays a vital role in the development of a brain-computer interface (BCI). Extracting the important information from raw EEG signals is difficult due to its nonstationary nature. Fixing a factual predefined basis function for efficient decomposition using a tunable Q wavelet transform is an arduous task. In this article, an adaptive tunable Q wavelet transform is proposed for the automatic selection of tuning parameters. Optimum tuning parameters are obtained using gray wolf optimization (GWO). Tuning parameters obtained by GWO are used to decompose the EEG signals into subbands (SBs). The set of time-domain features elicited from the SBs are used as an input to multiclass least-squares support vector machine. Classification accuracy of four basic emotions, namely, happy, fear, sad, and relax, is tested and compared with existing methods. An accuracy of 95.70% is achieved with a radial basis function kernel that is about 5% more than the existing methods using the same data set. This article proposes the development of a nonparameterized decomposition method for efficient decomposition of EEG signals. This method can be used with machine learning algorithms to take a step forward in the development of BCI systems.

Environmental sound classification using optimum allocation sampling based empirical mode decomposition

Automatic environmental sound classification (ESC) is prominent in various fields like robotics, security, and crime investigation. In this paper, optimum allocation sampling (OAS)-based empirical mode method (EMD) is proposed for automatic ESC. The OAS provides the reduced homogeneous length sequence of each long length sound signal, which is further decomposed into band-limited intrinsic mode functions (IMFs) using EMD. The features namely approximate entropy (AE), permutation entropy (PE), log energy entropy (LE), interquartile range (IQR), and zero cross rate (ZCR) are extracted from the IMFs. The OAS-EMD based features used as input to multi-class least squares support vector machine (MC-LS-SVM) and extreme learning machine (ELM) classifiers for evaluation the performance of proposed method. Experimental results show an accuracy of 87.25% and 77.61% with MC-LS-SVM and ELM classifiers, respectively.

Emotion recognition from single-channel EEG signals using a two-stage correlation and instantaneous frequency-based filtering method

Background and objective:The recognition of emotional states is a crucial step in the development of a brain-computer interface (BCI) system. Emotion recognition system finds applications in medical science for the impaired and disabled people. Electroencephalography assesses the neurophysiology of the brain for recognition of different emotional states.Methods: The audio-video stimulus based experimental setup is arranged for the electroencephalogram (EEG) recordings of happy, fear, sad, and relax emotions and a two-stage filtering method is proposed for the recognition of emotion EEG signals. At the first stage, a correlation-criterion is suggested for removal of noisy intrinsic mode functions (IMFs) from the IMFs obtained by applying the empirical mode decomposition on the raw EEG signal. The noise-free IMFs are used to reconstruct the denoised EEG signal with improved stationarity characteristics. The denoised EEG signal is further decomposed into modes using the variational mode decomposition (VMD). At the second stage, the instantaneous-frequency based filtering of VMD modes is performed and filtered modes are retained for the reconstruction of denoised EEG signal with the desired frequency range. After two-stage filtering, the non-linear measures of filtered EEG signals are used as input features to multi-class least squares support vector machine (MC-LS-SVM) classifier for emotion recognition.Results:The different kernel functions are tested in MC-LS-SVM classifier for emotion recognition. The Morlet wavelet (MW) kernel function provides the best individual classification accuracies for happy, fear, sad, and relax emotions as 92.79%, 87.62%, 88.98%, and 93.13%, respectively. The MW-kernel function also obtained the best overall accuracy of 90.63%, F1-score 0.9064, and kappa value 0.8751.Conclusions: The Audio-video stimulus based emotion EEG-dataset is recorded. A new filtering method is proposed for EEG signals. The proposed method provides better emotion recognition performance as compared to the state-of-the-art methods and classifies emotions using single-bipolar EEG channel, which can greatly reduce the complexity of emotion-recognition based BCI systems.

LS-SVM for large data sets

In this paper we propose multiclassification method for large data sets by ensembling least squares support vector machines (LS-SVM) with principal components instead of raw input vector. We use the revised one-vs-all method for multiclassification, which is one of voting scheme based on combining several binary classifications. The revised one-vs-all method is performed by using the hat matrix of LS-SVM ensemble, which is obtained by ensembling LS-SVMs trained using each random sample from the whole large training data. The leave-one-out cross validation (CV) function is used for the optimal values of hyper-parameters which affect the performance of multiclass LS-SVM ensemble. We present the generalized cross validation function to reduce computational burden of leave-one-out CV functions. Experimental results from real data sets are then obtained to illustrate the performance of the proposed multiclass LS-SVM ensemble.

Multiclass LS-SVM ensemble for large data

Multiclass classification is typically performed using the voting scheme method based on combining binary classifications. In this paper we propose multiclass classification method for large data, which can be regarded as the revised one-vs-all method. The multiclass classification is performed by using the hat matrix of least squares support vector machine (LS-SVM) ensemble, which is obtained by aggregating individual LS-SVM trained on each subset of whole large data. The cross validation function is defined to select the optimal values of hyperparameters which affect the performance of multiclass LS-SVM proposed. We obtain the generalized cross validation function to reduce computational burden of cross validation function. Experimental results are then presented which indicate the performance of the proposed method.

Facial neuromuscular signal classification by means of least square support vector machine for MuCI

Facial neuromuscular signal has recently drawn the researchers’ attention to its outstanding potential as an efficient medium for Muscle Computer Interface (MuCI) applications. The proper analysis of such electromyogram (EMG) signals is essential in designing the interfaces. In this article, a multiclass least-square support vector machine (LS-SVM) is proposed for classification of different facial gestures EMG signals. EMG signals were captured through three bi-polar electrodes from ten participants while gesturing ten different facial states. EMGs were filtered and segmented into non-overlapped windows from which root mean square (RMS) features were extracted and then fed to the classifier. For the purpose of classification, different models of LS-SVM were constructed while tuning the kernel parameters automatically and manually. In the automatic mode, 48 models were formed while parameters of linear and radial basis function (RBF) kernels were tuned using different optimization techniques, cost functions and encoding schemes. In the manual mode, 8 models were shaped by means of the considered kernel functions and encoding schemes. In order to find the best model with a reliable performance, constructed models were evaluated and compared in terms of classification accuracy and computational cost. Results reported that the model including RBF kernel which was tuned manually and encoded by one-versus-all scheme provided the highest classification accuracy (93.10%) and consumed 0.98s for training. It was indicated that automatic models were outperformed since they required too much time for tuning the parameters without any meaningful improvement in the final classification accuracy. The robustness of the selected LS-SVM model was evaluated through comparison with Support Vector Machine, fuzzy C-Means and fuzzy Gath-Geva clustering techniques.

The Hybrid KICA-GDA-LSSVM Method Research on Rolling Bearing Fault Feature Extraction and Classification

Rolling element bearings are widely used in high-speed rotating machinery; thus proper monitoring and fault diagnosis procedure to avoid major machine failures is necessary. As feature extraction and classification based on vibration signals are important in condition monitoring technique, and superfluous features may degrade the classification performance, it is needed to extract independent features, so LSSVM (least square support vector machine) based on hybrid KICA-GDA (kernel independent component analysis-generalized discriminate analysis) is presented in this study. A new method named sensitive subband feature set design (SSFD) based on wavelet packet is also presented; using proposed variance differential spectrum method, the sensitive subbands are selected. Firstly, independent features are obtained by KICA; the feature redundancy is reduced. Secondly, feature dimension is reduced by GDA. Finally, the projected feature is classified by LSSVM. The whole paper aims to classify the feature vectors extracted from the time series and magnitude of spectral analysis and to discriminate the state of the rolling element bearings by virtue of multiclass LSSVM. Experimental results from two different fault-seeded bearing tests show good performance of the proposed method.

A novel statistical algorithm for multiclass EEG signal classification

This paper presents a new algorithm for the classification of multiclass EEG signals. This algorithm involves applying the optimum allocation technique to select representative samples that reflect an entire database. This research investigates whether the optimum allocation is suitable to extract representative samples depending on their variability within the groups in the input EEG data. It also assesses whether these samples are efficient for the multiclass least square support vector machine (MLS-SVM) to classify EEG signals. The performances of the MLS-SVM with four different output coding approaches: minimum output codes (MOC), error correcting output codes (ECOC), One vs One (1vs1) and One vs All (1vsA), are evaluated with a benchmark epileptic EEG database. To test the consistency, all experiments are repeated ten times with the same classifying parameters in each classification process. The results show very high classification performances for each class, and also confirm the consistency of the proposed method in each repeated experiment. In addition, the performances by the optimum allocation based MLS-SVM method are compared with the four existing reference methods using the same database. The outcomes of this research demonstrate that the optimum allocation is very effective and efficient for extracting the representative patterns from the multiclass EEG data, and the MLS-SVM is also very well fitted with the optimum allocation technique for the EEG classification.

Categorization of fabric design using multi-class least-square support vector machine

Purpose – The purpose of this paper is to give an approach for categorization of diverse textile designs using their textural features as extracted from their gray images by means of multi-class least-square support vector machines (LS-SVM). Design/methodology/approach – In this work, the authors endeavor to devise a pattern recognition system based on LS-SVM which performs a multi-class categorization of three basic woven designs namely plain, twill and sateen after analyzing their features. Findings – The result establishes that LS-SVM is able to classify the fabric design with a reasonable degree of accuracy and it outperforms the standard SVM. Originality/value – The algorithmic simplicity of LS-SVM resulting from replacement of inequality constraints by equality ones and ability of handling noisy data by accommodating an error variable in its algorithm make it eminently suitable for textile pattern recognition. This paper offers a maiden application of LS-SVM in textile pattern recognition.

Automatic classification of sleep stages based on the time-frequency image of EEG signals

In this paper, a new method for automatic sleep stage classification based on time-frequency image (TFI) of electroencephalogram (EEG) signals is proposed. Automatic classification of sleep stages is an important part for diagnosis and treatment of sleep disorders. The smoothed pseudo Wigner–Ville distribution (SPWVD) based time-frequency representation (TFR) of EEG signal has been used to obtain the time-frequency image (TFI). The segmentation of TFI has been performed based on the frequency-bands of the rhythms of EEG signals. The features derived from the histogram of segmented TFI have been used as an input feature set to multiclass least squares support vector machines (MC-LS-SVM) together with the radial basis function (RBF), Mexican hat wavelet, and Morlet wavelet kernel functions for automatic classification of sleep stages from EEG signals. The experimental results are presented to show the effectiveness of the proposed method for classification of sleep stages from EEG signals.

Safety Region Estimation and State Identification of Rolling Bearing Based on Statistical Feature Extraction

The idea of safety region was introduced into the rolling bearing condition monitoring. The safety region estimation and the state identification of the rolling bearing operational were performed by the comprehensive utilization of Empirical Mode Decomposition (EMD), Principal Component Analysis (PCA), and the Least Square Support Vector Machine (LSSVM). The collected vibration data was segmented according to a certain time interval, and then the Intrinsic Mode Functions (IMFs) of each piece of the data were obtained by EMD. The control limits of two statistical variables extracted by PCA were presented as state characteristics. The safety region estimation for the rolling bearing operational status was performed by two-class LSSVM. The states of normal bearing, ball fault, inner race fault, and outer race fault were identified by the multi-class LSSVM. The results show that the estimation accuracy for both the safety region and the states identification reached 95%, and that the validity of the proposed method was verified.

Safety Region Estimation and State Identification of Rolling Bearing Based on Statistical Feature Extraction

The idea of safety region was introduced into the rolling bearing condition monitoring. The safety region estimation and the state identification of the rolling bearing operational were performed by the comprehensive utilization of Empirical Mode Decomposition (EMD), Principal Component Analysis (PCA), and the Least Square Support Vector Machine (LSSVM). The collected vibration data was segmented according to a certain time interval, and then the Intrinsic Mode Functions (IMFs) of each piece of the data were obtained by EMD. The control limits of two statistical variables extracted by PCA were presented as state characteristics. The safety region estimation for the rolling bearing operational status was performed by two-class LSSVM. The states of normal bearing, ball fault, inner race fault, and outer race fault were identified by the multi-class LSSVM. The results show that the estimation accuracy for both the safety region and the states identification reached 95%, and that the validity of the proposed method was verified.

Roller bearing safety region estimation and state identification based on LMD–PCA–LSSVM

The basic idea of safety region is introduced into roller bearing condition monitoring. Local mean decomposition (LMD), principal component analysis (PCA) and least square support vector machine (LSSVM) are used comprehensively for the estimation of the safety region and the identification of normal state and faulty state for the roller bearing operational status. First, the vibration acceleration data was segmented according to a certain time interval and then Product Functions (PFs) of each piece of the data were obtained by LMD. Based on this, statistics control limits T2 and SPE were extracted by PCA as roller bearings’ state characteristics. Finally, LSSVM was used for the estimation of the safety region of the roller bearing operation state, and multi-class LSSVM was used for the identification of the four normal, ball fault, inner race fault and outer race fault states. The results show that both the safety region estimation and state identification are accurate, and confirm the validity of the LMD–PCA–LSSVM method.

A NOVEL FIVE-CATEGORY LOAN-RISK EVALUATION MODEL USING MULTICLASS LS-SVM BY PSO

Five-category loan classification (FCLC) is an international financial regulation approach. Recently, the application and implementation of FCLC in the Chinese microfinance bank has mostly relied on subjective judgment, and it is difficult to control and lower loan risk. In view of this, this paper is dedicated to researching and solving this problem by constructing the FCLC model based on improved particle-swarm optimization (PSO) and the multiclass, least-square, support-vector machine (LS-SVM). First, LS-SVM is the extension of SVM, which is proposed to achieve multiclass classification. Then, improved PSO is employed to determine the parameters of multiclass LS-SVM for improving classification accuracy. Finally, some experiments are carried out based on rural credit cooperative data to demonstrate the performance of our proposed model. The results show that the proposed model makes a distinct improvement in the accuracy rate compared with one-vs.-one (1-v-1) LS-SVM, one-vs.-rest (1-v-r) LS-SVM, 1-v-1 SVM, and 1-v-r SVM. In addition, it is an effective tool in solving the problem of loan-risk rating.

Identification of thermal error in a feed system based on multi-class LS-SVM

Research of thermal characteristics has been a key issue in the development of high-speed feed system. The thermal positioning error of a ball-screw is one of the most important objects to consider for high-accuracy and high-speed machine tools. The research work undertaken herein ultimately aims at the development of a comprehensive thermal error identification model with high accuracy and robust. Using multi-class least squares support vector machines (LS-SVM), the thermal positioning error of the feed system is identified with the variance and mean square value of the temperatures of supporting bearings and screw-nut as feature vector. A series of experiments were carried out on a self-made quasi highspeed feed system experimental bench HUST-FS-001 to verify the identification capacity of the presented method. The results show that the recommended model can be used to predict the thermal error of a feed system with good accuracy, which is better than the ordinary BP and RBF neural network. The work described in this paper lays a solid foundation of thermal error prediction and compensation in a feed system.