Multi-class Support Vector Machine Method Research Articles

Classifying Coal Mine Pillar Stability Areas with Multiclass SVM on Ensemble Learning Models

Pillars are key structural components in coal mining. The safety requirements of underground coal mines are non-negotiable. Accurately classifying the areas of pillar stability helps ensure safety in coal mines. This study aimed to classify new pillar stability categories and their stability areas. The multiclass support vector machine (SVM) method was implemented with two types of kernel functions (polynomial and radial basis function (RBF) kernels) on pillar stability data with four new categories: failed or intact, either with or without an appropriate safety factor. This classification uses three basic ensemble learning models: Artificial Neural Network-Backpropagation Rectified Linear Unit, Artificial Neural Network-Backpropagation Exponential Linear Unit, and Artificial Neural Network-Backpropagation Gaussian Error Linear Unit. The results with four data proportions and ten experiments had an average accuracy and standard deviation of 92.98% and 0.56%-1.64% respectively. The accuracies of the multiclass SVM method using the polynomial kernel and the RBF kernel with Bayesian parameter optimization to classify the areas of pillar stability were 91% and 92%, respectively. The multiclass SVM method with the RBF kernel captured 96.6% of potentially dangerous pillars. The visualization of classification areas showed that areas with intact pillars may also have failed pillars.

A Hybrid EEG-based Emotion Recognition Approach Using Wavelet Convolutional Neural Networks and Support Vector Machine.

Nowadays, deep learning and convolutional neural networks (CNNs) have become widespread tools in many biomedical engineering studies. CNN is an end-to-end tool, which makes the processing procedure integrated, but in some situations, this processing tool requires to be fused with machine learning methods to be more accurate. In this paper, a hybrid approach based on deep features extracted from wavelet CNNs (WCNNs) weighted layers and multiclass support vector machine (MSVM) was proposed to improve the recognition of emotional states from electroencephalogram (EEG) signals. First, EEG signals were preprocessed and converted to Time-Frequency (T-F) color representation or scalogram using the continuous wavelet transform (CWT) method. Then, scalograms were fed into four popular pre-trained CNNs, AlexNet, ResNet-18, VGG-19, and Inception-v3 to fine-tune them. Then, the best feature layer from each one was used as input to the MSVM method to classify four quarters of the valence-arousal model. Finally, the subject-independent leave-one-subject-out criterion was used to evaluate the proposed method on DEAP and MAHNOB-HCI databases. Results showed that extracting deep features from the earlier convolutional layer of ResNet-18 (Res2a) and classifying using the MSVM increased the average accuracy, precision, and recall by about 20% and 12% for MAHNOB-HCI and DEAP databases, respectively. Also, combining scalograms from four regions of pre-frontal, frontal, parietal, and parietal-occipital and two regions of frontal and parietal achieved the higher average accuracy of 77.47% and 87.45% for MAHNOB-HCI and DEAP databases, respectively. Combining CNN and MSVM increased the recognition of emotion from EEG signals and the results were comparable to state-of-the art studies.

Prediction of factors for Controlling of Green House Farming with Fuzzy based multiclass Support Vector Machine

A smart greenhouse is a scheme with the aim of allowing the plant growth optimally due to the control in temperature, soil moisture, and humidity. It is necessary to go for best possible smart greenhouse controlling system to maneuver the parameters in accordance with the climatic and seasonal changes plant. In this proposed work, the appropriate setting, monitoring of the model parameters like optimal temperature, humidity, and soil moisture in the greenhouse farm available at Modakkurichi near Erode is carried out using fuzzy logic and multiclass Support Vector Machine (SVM) techniques. It is efficiently implemented as the fuzzy trapezoidal membership function for each sample within the hyper-sphere as a linear function of the selected sample’s distance in the non-linear SVM hyperplane. Based on the simulation and experimental results, the fuzzy logic with Multi-class Support Vector Machine method is effective in selection of the rules to make decision. Simulation of the work was carried out in Python using the open source tool COLAB. Statistical parameters are evaluated for the proposed Fuzzy based SVM method. Root Mean Square Error of the proposed controller has the minimum value as 0.962E-02 when compared to other controllers which vary from 1.04 E-02 to 8.27 E-02.

Pattern Recognition using Multiclass Support Vector Machine Method with Local Binary Pattern as Feature Extraction

Corn is an essential agricultural commodity since it is used in animal feed, biofuel, industrial processing, and the manufacture of non-food industrial commodities such as starch, acid, and alcohol. Early detection of diseases and pests of corn aims to reduce the possibility of crop failure and maintain the quality and quantity of crop yields. A decision tree is a nonparametric classification model in statistical machine learning that predicts target variables using tree-structured decisions. The performance of this model can increase significantly if the continuous predictor variables are discretized into valid categories. However, in some cases, the result does not provide satisfactory performance. The possible cause is the ambiguity in discretizing predictor variables. The incorporation of fuzzy membership functions into the model to resolve discretization ambiguity issues. This work aims to classify diseases and pests of corn plants using the decision tree model and improve the model’s performance by implementing fuzzy membership functions. The main contribution of this work is that we have shown a significant improvement in the decision tree model performance by implementing fuzzy membership functions; S-growth, triangle, and S-shrinkage curves. The proposed fuzzy model is better than the decision tree model, with an average performance increase from the largest to the smallest; kappa (12.16%), recall (11.8%), F-score (9.71%), precision (5.08%), accuracy (3.23%), specificity (1.94%), and AUC (0.49%). The combination of bias and variance generated by the proposed model is quite small, indicating that the model is able to capture data trends well.

Sentiment Analysis on KAI Twitter Post Using Multiclass Support Vector Machine (SVM)

Information in form of unstructured texts is increasing and becoming commonplace for its existence on the internet. This information is easily found and utilized by business people or companies through social media. One of them is Twitter. Twitter is ranked 6th as a social media that is widely accessed today. The use of Twitter has the disadvantage of unstructured and large data. Consequently, it is difficult for business people or companies to know opinion towards service with limited resources. To Make it easier for businesses know the public's sentiment for better service in the future, public sentiment on Twitter needs to be classified as positive, neutral, and negative. The Multiclass Support Vector Machine (SVM) method is a supervised learning classification method that handles three classes classification. This paper uses One Against All (OAA) approach as a method to determine the class. This paper contains the results of classifying OAA Multiclass SVM methods with five different weighting features unigram, bigram, trigram, unigram+ bigram, and word cloud for analyzing tweet data, finding the best accuracy and important feature when processed with large data. The highest accuracy is the unigram TF-IDF model combined with the OAA Multiclass SVM with gamma 0.7 is 80.59.

Effective plant discrimination based on the combination of local binary pattern operators and multiclass support vector machine methods

Accurate crop and weed discrimination plays a critical role in addressing the challenges of weed management in agriculture. The use of herbicides is currently the most common approach to weed control. However, herbicide resistant plants have long been recognised as a major concern due to the excessive use of herbicides. Effective weed detection techniques can reduce the cost of weed management and improve crop quality and yield. A computationally efficient and robust plant classification algorithm is developed and applied to the classification of three crops: Brassica napus (canola), Zea mays (maize/corn), and radish. The developed algorithm is based on the combination of Local Binary Pattern (LBP) operators, for the extraction of crop leaf textural features and Support vector machine (SVM) method, for multiclass plant classification. This paper presents the first investigation of the accuracy of the combined LBP algorithms, trained using a large dataset of canola, radish and barley leaf images captured by a testing facility under simulated field conditions. The dataset has four subclasses, background, canola, corn, and radish, with 24,000 images used for training and 6000 images, for validation. The dataset is referred herein as “bccr-segset” and published online. In each subclass, plant images are collected at four crop growth stages. Experimentally, the algorithm demonstrates plant classification accuracy as high as 91.85%, for the four classes.

Complex power quality disturbances classification via curvelet transform and deep learning

This paper presents a novel approach to detect and classify the power quality disturbance (PQD) signals based on singular spectrum analysis (SSA), curvelet transform (CT) and deep convolutional neural networks (DCNNs). SSA is a non-parametric technique, does not require any supposition to generate the observed signal, and provides an effective way to recognize weak transient PQ signal. Fast discrete curvelet transform (FDCT) is an efficient method compared to wavelet transforms. Firstly, PQD signals are decomposed using SSA and FDCT methods. Initial six and three levels decomposition of the SSA and FDCT are used as features of PQD respectively. Finally, DCNNs based classifier and multiclass support vector machines (SVMs) classifier are used for classification of single and complex PQDs. For validation of the proposed algorithm, thirty-one categories of real and synthetic PQD waveforms are considered. The proposed scheme is tested and the results are recorded. The results of proposed SSA-FDCT-DCNN (SFD) based classifier are compared to the results of multiclass SVM based and other existing methods The achieved results show that the SFD classifier is more proficient than the multiclass SVM and other present methods. In addition, the proposed SFD based classifier can be efficiently used to classify the single and complex PQ disturbances.

RETRACTED ARTICLE: Research on the intelligent judgment of traffic congestion in intelligent traffic based on pattern recognition technology

Traffic congestion is becoming more and more frequent with the increase of city vehicles. There are still some problems in data processing and real-time traffic state identification for the intelligent judgment of road congestion. Based on this, a multi-class support vector machine method for pattern recognition was proposed, which was an improvement of the traditional support vector machine. Firstly, the road situation was divided into three kinds: “traffic”, “congestion” and “traffic paralysis” by using pattern recognition technology, and the road traffic situation was divided into “traffic” and “congestion” by using support vector machine, on the basis of this, the quadratic discriminant of “congestion” and “traffic paralysis” were carried out to “congestion” state, so that the intelligent judgment of three kinds of traffic state was met. Then combined with the actual road sections and real-time monitoring of road data, the simulation experiment of the pattern recognition was carried out to show that the pattern recognition method can effectively divide and analyze the road traffic situation, and realize the function of intelligent judgment, which could promote the intelligent management of the road, improve the urban road planning and improve the service quality of the traffic system.

Multiple-Fault Diagnosis Method Based on Multiscale Feature Extraction and MSVM_PPA

Identification of rolling bearing fault patterns, especially for the compound faults, has attracted notable attention and is still a challenge in fault diagnosis. In this paper, a novel method called multiscale feature extraction (MFE) and multiclass support vector machine (MSVM) with particle parameter adaptive (PPA) is proposed. MFE is used to preprocess the process signals, which decomposes the data into intrinsic mode function by empirical mode decomposition method, and instantaneous frequency of decomposed components was obtained by Hilbert transformation. Then, statistical features and principal component analysis are utilized to extract significant information from the features, to get effective data from multiple faults. MSVM method with PPA parameters optimization will classify the fault patterns. The results of a case study of the rolling bearings faults data from Case Western Reserve University show that (1) the proposed intelligent method (MFE_PPA_MSVM) improves the classification recognition rate; (2) the accuracy will decline when the number of fault patterns increases; (3) prediction accuracy can be the best when the training set size is increased to 70% of the total sample set. It verifies the method is feasible and efficient for fault diagnosis.

One-Class Classification of Airborne LiDAR Data in Urban Areas Using a Presence and Background Learning Algorithm

Automatic classification of light detection and ranging (LiDAR) data in urban areas is of great importance for many applications such as generating three-dimensional (3D) building models and monitoring power lines. Traditional supervised classification methods require training samples of all classes to construct a reliable classifier. However, complete training samples are normally hard and costly to collect, and a common circumstance is that only training samples for a class of interest are available, in which traditional supervised classification methods may be inappropriate. In this study, we investigated the possibility of using a novel one-class classification algorithm, i.e., the presence and background learning (PBL) algorithm, to classify LiDAR data in an urban scenario. The results demonstrated that the PBL algorithm implemented by back propagation (BP) neural network (PBL-BP) could effectively classify a single class (e.g., building, tree, terrain, power line, and others) from airborne LiDAR point cloud with very high accuracy. The mean F-score for all of the classes from the PBL-BP classification results was 0.94, which was higher than those from one-class support vector machine (SVM), biased SVM, and maximum entropy methods (0.68, 0.82 and 0.93, respectively). Moreover, the PBL-BP algorithm yielded a comparable overall accuracy to the multi-class SVM method. Therefore, this method is very promising in the classification of the LiDAR point cloud.

A multi-class SVM approach based on the l1-norm minimization of the distances between the reduced convex hulls

Multi-class classification is an important pattern recognition task that can be addressed accurately and efficiently by Support Vector Machine (SVM). In this work we present a novel SVM-based multi-class classification approach based on the center of the configuration, a point which is equidistant to all classes. The center of the configuration is obtained from the dual formulation by minimizing the distances between the reduced convex hulls using the l1-norm, while the decision functions are subsequently constructed from this point. This work also extends the ideas of Zhou et al. (2002) [37] to multi-class classification. The use of l1-norm provides a single linear programming formulation, which reduces the complexity and confers scalability compared with other multi-class SVM methods based on quadratic programming formulations. Experiments on benchmark datasets demonstrate the virtues of our approach in terms of classification performance and running times compared with various other multi-class SVM methods.

Feature-Selected Tree-Based Classification

Feature selection can decrease classifier size and improve accuracy by removing noisy and/or redundant features. However, it is possible for feature selection to yield features that are only partially informative about the classes in the set. These features are beneficial for distinguishing between some classes but not others. In these cases, it is beneficial to divide the large classification problem into a set of smaller problems, where a more specific set of features can be used to classify different classes. Dividing a problem this way is also common when the base classifier is binary, and the problem needs to be reformulated as a set of two-class problems so it can be handled by the classifier. This paper presents a method for multiclass classification that simultaneously formulates a binary tree of simpler classification subproblems and performs feature selection for the individual classifiers. The feature selected hierarchical classifier (FSHC) is tested against several well-known techniques for multiclass division. Tests are run on nine different real data sets and one artificial data set using a support vector machine (SVM) classifier. The results show that the accuracy obtained by the FSHC is comparable with other common multiclass SVM methods. Furthermore, the results demonstrate that the algorithm creates solutions with fewer classifiers, fewer features, and a shorter testing time than the other SVM multiclass extensions.

Multiclass classification of n-butanol concentrations with k-nearest neighbor algorithm and support vector machine in an electronic nose

An electronic nose (e-nose) is a machine used for sensing and recognizing odors by using chemical sensors. The performance of e-nose depends on choosing correct sensor and correct pattern recognition algorithm according to application fields and kinds of the odors. In this study, different n-butanol concentrations sensed by 12 metal oxide gas sensors are classified by using multiclass support vector machine methods (SVM) and k-nearest neighbor (k-NN) algorithm. Focus in this paper is that the performances of these algorithms are increased with a decision tree structure. Therefore the proposed decision tree structure is applied to the electronic nose data for sensor subset selection and classification of the n-butanol concentrations. SVM and k-NN algorithms are tested for classification of different concentrations in this decision tree structure and ordinary structure. In addition to these, cross-validation technique is used for both increasing success of classification algorithms and assessing the results objectively. This study shows that the success of classification algorithms increase from 87% to 93% and 86% to 96% by using data of two sensors selected with the proposed decision tree structure for the k-NN and the SVM methods, respectively.

Image Classification for Steel Strip Surface Defects Based on Support Vector Machines

In order to realize less time consuming and on-line image classification for steel strip surface defects, an improved multiclass support vector machine (SVM) was proposed. The SVM used a novel algorithm and only constructed (k-1) two-class SVMs where K is the number of classes. In the testing phase, to identify the surface defects it used a new unidirectional acyclic graph which had internal (k-1) nodes and k leaves. Its testing time is less than traditional multiclass SVM method. The experiment results shows that this method is simple and less time consuming while preserving generalization ability and recognition accuracy toward steel strip surface defects.

Handwritten Digit Recognition Based on A Neural – SVM Combination

In this paper, we present a new multi-class method for handwritten digit recognition that is based on cascade combination of neural networks and support vector machines (SVMs). Binary SVMs are used to provide high linear separation between classes while the neural network generates automatic multi-class decision. In addition, we introduce Jaccard distance into negative distance (ND) SVM kernel. The performance evaluation of the proposed method is conducted on the well-known US Postal Service database. Experimental results indicate that it can significantly reduce the runtime while giving at least the same performance as conventional multi-class SVM methods. Besides, the use of Jaccard distance improves significantly the performance of the standard ND kernel.

Robust and efficient multiclass SVM models for phrase pattern recognition

Phrase pattern recognition (phrase chunking) refers to automatic approaches for identifying predefined phrase structures in a stream of text. Support vector machines (SVMs)-based methods had shown excellent performance in many sequential text pattern recognition tasks such as protein name finding, and noun phrase (NP)-chunking. Even though they yield very accurate results, they are not efficient for online applications, which need to handle hundreds of thousand words in a limited time. In this paper, we firstly re-examine five typical multiclass SVM methods and the adaptation to phrase chunking. However, most of them were inefficient when the number of phrase types scales. We thus introduce the proposed two new multiclass SVM models that make the system substantially faster in terms of training and testing while keeps the SVM accurate. The two methods can also be applied to similar tasks such as named entity recognition and Chinese word segmentation. Experiments on CoNLL-2000 chunking and Chinese base-chunking tasks showed that our method can achieve very competitive accuracy and at least 100 times faster than the state-of-the-art SVM-based phrase chunking method. Besides, the computational time complexity and the time cost analysis of our methods were also given in this paper.

Comparison of Supervised Clustering Methods for the Analysis of DNA Microarray Expression Data

Several typical supervised clustering methods such as Gaussian mixture model-based supervised clustering (GMM), k-nearest-neighbor (KNN), binary support vector machines (SVMs) and multiclass support vector machines (MC-SVMs) were employed to classify the computer simulation data and two real microarray expression datasets. False positive, false negative, true positive, true negative, clustering accuracy and Matthews' correlation coefficient (MCC) were compared among these methods. The results are as follows: (1) In classifying thousands of gene expression data, the performances of two GMM methods have the maximal clustering accuracy and the least overall FP+FN error numbers on the basis of the assumption that the whole set of microarray data are a finite mixture of multivariate Gaussian distributions. Furthermore, when the number of training sample is very small, the clustering accuracy of GMM-II method has superiority over GMM-I method. (2) In general, the superior classification performance of the MC-SVMs are more robust and more practical, which are less sensitive to the curse of dimensionality, and not only next to GMM method in clustering accuracy to thousands of gene expression data, but also more robust to a small number of high-dimensional gene expression samples than other techniques. (3) Of the MC-SVMs, OVO and DAGSVM perform better on the large sample sizes, whereas five MC-SVMs methods have very similar performance on moderate sample sizes. In other cases, OVR, WW and CS yield better results when sample sizes are small. So, it is recommended that at least two candidate methods, choosing on the basis of the real data features and experimental conditions, should be performed and compared to obtain better clustering result.

A decision based one-against-one method for multi-class support vector machine

The support vector machine (SVM) has a high generalisation ability to solve binary classification problems, but its extension to multi-class problems is still an ongoing research issue. Among the existing multi-class SVM methods, the one-against-one method is one of the most suitable methods for practical use. This paper presents a new multi-class SVM method that can reduce the number of hyperplanes of the one-against-one method and thus it returns fewer support vectors. The proposed algorithm works as follows. While producing the boundary of a class, no more hyperplanes are constructed if the discriminating hyperplanes of neighbouring classes happen to separate the rest of the classes. We present a large number of experiments that show that the training time of the proposed method is the least among the existing multi-class SVM methods. The experimental results also show that the testing time of the proposed method is less than that of the one-against-one method because of the reduction of hyperplanes and support vectors. The proposed method can resolve unclassifiable regions and alleviate the over-fitting problem in a much better way than the one-against-one method by reducing the number of hyperplanes. We also present a direct acyclic graph SVM (DAGSVM) based testing methodology that improves the testing time of the DAGSVM method.

Multi-class support vector machines for protein secondary structure prediction.

The solution of binary classification problems using the Support Vector Machine (SVM) method has been well developed. Though multi-class classification is typically solved by combining several binary classifiers, recently, several multi-class methods that consider all classes at once have been proposed. However, these methods require resolving a much larger optimization problem and are applicable to small datasets. Three methods based on binary classifications: one-against-all (OAA), one-against-one (OAO), and directed acyclic graph (DAG), and two approaches for multi-class problem by solving one single optimization problem, are implemented to predict protein secondary structure. Our experiments indicate that multi-class SVM methods are more suitable for protein secondary structure (PSS) prediction than the other methods, including binary SVMs, because their capacity to solve an optimization problem in one step. Furthermore, in this paper, we argue that it is feasible to extend the prediction accuracy by adding a second-stage multi-class SVM to capture the contextual information among secondary structural elements and thereby further improving the accuracies. We demonstrate that two-stage SVMs perform better than single-stage SVM techniques for PSS prediction using two datasets and report a maximum accuracy of 79.5%.