Generalization Ability Of Support Vector Machine Research Articles

Condensing the solution of support vector machines via radius-margin bound

The paper deals with the problem of how to condense efficiently the used basis vectors in the decision function of support vector machines (SVM). Most existing methods learn the basis vectors and the corresponding coefficients by maximizing the margin between different classes. They ignore the key fact that condensing the solution of SVM is equivalent to constructing the SVM model in the transformation space determined by the basis vectors. Thus, the radius-margin bound, which is related with the generalization ability of SVM, changes with them. In the paper, we propose a novel method called sparse support vector machine guided by radius-margin bound (RMB-SSVM) to learn the condensed solution for SVM. The key characteristic of RMB-SSVM is that it employs a criterion that exploits integratedly the margin and the radius of the minimum hypersphere enclosing the data to guide the selecting of the basis vectors and the learning of the corresponding coefficients. Thus, the learning criterion of RMB-SSVM is directly related with the generalization ability of SVM and so it can yield better performance. We develop an effective method to handle the proposed optimization model, and propose a heuristic scheme to select the basis vectors used in the decision function. Further, we explore how to use the maximum pairwise distance over the pairs of data points to approximate the radius in our methodology. This can simplify the proposed model and reduce the training time while not drastically impairing the performance. Finally, we conduct the comprehensive experiments to demonstrate the effectiveness and superiority of the proposed methods by comparing them with the counterparts.

Fault classification in smart distribution network using support vector machine

<span>Machine learning application have been widely used in various sector as part of reducing work load and creating an automated decision making tool. This has gain the interest of power industries and utilities to apply machine learning as part of the operation. Fault identification and classification based machine learning application in power industries have gain significant accreditation due to its great capability and performance. In this paper, a machine-learning algorithm known as Support Vector Machine (SVM) for fault type classification in distribution system has been developed. Eleven different types of faults are generated with respect to actual network. A wide range of simulation condition in terms of different fault impedance value as well as fault types are considered in training and testing data. Right setting parameters are important to learning results and generalization ability of SVM. Gaussian radial basis function (RBF) kernel function has been used for training of SVM to accomplish the most optimized classifier. Initial finding from simulation result indicates that the proposed method is quick in learning and shows good accuracy values on faults type classification in distribution system. The developed algorithm is tested on IEEE 34 bus and IEEE 123 bus test distribution system. </span>

An improved support vector machine classifier based on artificial bee colony algorithm

Support vector machine (SVM) has unique advantages in the classification of small sample data. The selection of parameters has an important impact on the classification accuracy and generalization ability of SVM. Since the selection of parameters of SVM is usually based on experience, an improved SVM classification model based on artificial bee colony algorithm (GOABC-SVM) is proposed in this paper. In this model, firstly, the traditional artificial bee colony algorithm is optimized using the ideas of global optimal solution guidance and opposite learning. Secondly, we set the reciprocal of classification error rate as the fitness function, and use the improved artificial bee colony algorithm to obtain the optimal parameter combination of SVM. Experiments on a set of datasets of UCI show that the proposed model has higher classification accuracy and better generalization ability.

Multi-Scale Mahalanobis Kernel-Based Support Vector Machine for Classification of High-Resolution Remote Sensing Images

Support vector machine (SVM) is a powerful cognitive and learning algorithm in the domain of pattern recognition and image classification. However, the generalization ability of SVM is limited when processing classification of high-resolution remote sensing images. One chief reason for this is that the Euclidean distance-based distance matrix in traditional SVM treats different samples equally and overlooks the global distribution of samples. To construct a more effective SVM-based classification method, this paper proposes a multi-scale Mahalanobis kernel-based SVM classifier. In this new method, we first introduce a Mahalanobis distance kernel to improve the global cognitive learning ability of SVM. Then, the Mahalanobis distance kernel is embedded to the multi-scale kernel learning (MSKL) to construct a novel multi-scale Mahalanobis kernel, in which the parameters are optimized by a bio-inspired algorithm, named differential evolution. Finally, the new method is extended to the classification of high-resolution remote sensing images based on the spatial-spectral features. The comparison experiments of five public UCI datasets and two high-resolution remote sensing images verify that the Mahalanobis distance-based method can obtain more accurate classification results than that of the Euclidean distance-based method. In addition, the proposed method produced the best classification results in all the experiments. The global cognitive learning ability of Mahalanobis distance-based method is stronger than that of the Euclidean distance-based method. In addition, this study indicates that the optimized MSKL are potential for the interpretation and understanding of complicated high-resolution remote sensing scene.

A novel adaptive differential evolution SVM model for predicting coal and gas outbursts

Parameter selection is a key factor affecting the performance of support vector machines (SVMs). To further improve the classification accuracy and generalization ability of SVMs, a parameter selection model for SVMs with RBF kernel is proposed based on adaptive differential evolution (ADE) algorithm, and is applied to predict coal and gas outbursts. The function of each parameter and its adjustment scheme of differential evolution (DE) algorithm are analyzed, and the algorithm is improved by using the decision error rate of samples as the objective function. Adaptive calculation equations for variability factor and crossover factor are designed. The variability and crossover factors are automatically adjusted in the execution process of the algorithm, so that the population diversity is maintained in the early stages of algorithm execution to enhance the ability of searching global optimal values, while the stability of the algorithm is guaranteed in the late stages by promoting the searching ability for local optimal values. A novel ADESVM model for predicting coal and gas outbursts is established by using ADE algorithm to select SVM parameters, which is applied to predict the coal and gas outbursts. Experimental results show that the designed ADE algorithm has high convergence speed and high computational accuracy. The proposed ADESVM model has higher training speed and is more robust compared with other similar SVM models. It also has higher prediction accuracy and shorter training time, compared with back propagation neural networks, providing a new method for the intelligent prediction of coal and gas outbursts.

An Experimental Study of the Hyper-parameters Distribution Region and Its Optimization Method for Support Vector Machine with Gaussian Kernel

Support vector machine (SVM) is a kind of machine learning method, but the selection of parameters has important effects on the generalization ability of SVMs. In this study, the relation between the error penalty parameter C, kernel parameter σ and the generalization ability of SVMs is discussed. Parameter C adjusts the similarity among within-class members, while parameter σ adjusts the similarity between classes. Moreover, C and σ balances each other mutually within a certain range, which forms a fan-shaped optional parameter distribution region. The optimal parameter area should be located near the center of the sector where both C and σ are small. According to this, a method is suggested to first search a suitable area with coarse grids, and then determine the optimal parameter within the area with a fine bilinear grid. Experimental results show that the new parameter selection method can not only avoid local optima, and thus excluding the cases in which C and σ are big and unstable, but also can be extremely fast in searching process. Compared with other parameter selection methods, the performance of SVMs cannot be influenced, or even better in some cases.

Hidden Defects Diagnosis Using Parameter Optimization

In this paper, the issue of composite defects diagnosis by applying the support vector machine (SVM) was addressed. The component analysis was performed initially to extract the features and to reduce the dimensionality of original data features. Kernel parameters selection of support vector machine which has great influence on the performance of defects classification has been discussed in this work. Precisely, we focus on wavelet transform to extract the feature from the original signals, adopt component analysis to do feature selection and apply support vector machine to classify the defects. This paper exploits the parameter optimization procedure to ensure the generalization ability of SVM. The result shows that multi-class SVM produces promising results and has the potential for use in fault diagnosis.

Structure of Mixture Kernel Function and its Application in Soft Measurement about Beating Degree

Kernel function is dominant in regression process of Support Vector Machine (SVM), it influences the prediction performance of SVM directly. Single local or global kernel function has limitation in generalization or learning capacity. The mixture kernel function with two good capacities had been structured, and it had been used in soft-sensor modeling of refining process. The contrast of simulation results show: the generalization ability of SVM based on mixture kernel function is better than the ones based on the single local or global kernel function, the algorithm had been applied in the refining process of HETAO wood industry factory, and the prediction result meets the requirement of craft production.

Runway Incursion Event Forecast Model Based on LS-SVR with Multi-kernel

Forecasting of runway incursion event is very significant to guide the job of civil aviation safety management. It is an important part of the runway incursion early warning management. However, prediction of runway incursion event is a complicated problem due to its non-linearity and the small quantity of training data. As a novel type of learning machine, support vector machine (SVM) has been gaining popularity due to their promising performance, such as dealing with the data of small sample, the high dimension and the excellent generalization ability. However, the generalization ability of SVM often relies on whether the selected kernel function is suitable for real data. To lessen the sensitivity of different kernels and improve generalization ability, least square support vector regression (LS-SVR) with multi-kernel is proposed to forecast the runway incursion event in this paper. The two experimental results indicate that LS-SVR with multi-kernel model is better than LS-SVR with individual kernel model and generalized regression neural network (GRNN) model. Consequently, multi-kernel LS-SVR model is a proper alternative for forecasting of the runway incursion event.

Modeling of SiC MESFETs by Using Support Vector Machine Regression

In this paper, a support vector machine (SVM) regression approach is introduced for modeling of field effect transistors (FETs). Benefits to the good generalization ability of SVM, a SVM regression (SVR) model is established using a set of training and testing data, which is produced by simulation using an available empirical model of SiC MESFETs. Experimental results show the SVR model has good ability in predicting electrical performance of FETs.

ALL/AML Cancer Classification by Gene Expression Data Using SVM and CSVM Approach

Cancer classification plays an important role in cancer treatment. There has been no general approach for this problem now. The tasks for cancer classification are of two aspects: identifying new cancer classes and assigning tumors to known classes, which are called class discovery and class prediction by Golub et al. [1]. From mathematical point of view, class discovery is a cluster analysis problem, while class prediction is usually called classification problem (we’ll use the later name to keep consist with pattern recognition literatures). Until now, cancer classification has been based primarily on morphological appearance of tumor [1]. This has serious limitations because of ambiguity. Golub et al. presented a new approach to cancer classification based on gene expression monitoring by DNA microarrays in [1]. They chose acute leukemia as a test case, and the target is to distinguish between ALL (acute lymphoblastic leukemia) and AML (acute myeloid leukemia), which is a typical cancer classification problem not well solved despite many years of efforts. This paper is a report of our work on the classification (prediction) part of this problem following their original work. Golub et al. adopted a feature selection (gene selection) procedure before classification. A metric was defined to evaluate the correlation of each gene to the classification. After some “good” genes were selected from all the 6817 genes, the classification is done by a weighted voting scheme. The classifier was trained on a 38-sample training set, and another 34-sample set was used for testing. With leave-one-out cross-validation on the training set with 50 selected genes, 36 out of 38 samples were correctly classified and 2 were rejected (no-call). The performance on the test set was that 29 samples out of 34 were correctly classified and the other 5 were rejected. If the classifier were compelled to give these 5 no-calls a prediction, the prediction would be wrong. Since the feature selection procedure is of single selection type, and the classification method is also an intuitive one, we believe that there is still much space for the performance to be improved. In our approach to the problem, we took all the genes for the classification (the selection problem will be discussed in another paper), and applied the support vector machine(SVM) method and one of its improved version CSVM as the classifier. Thanks to the better generalization ability of SVM and CSVM, much better performance was obtained.

10.1109/ETFA.2010.5641303

In this paper, the issue of composite defects diagnosi s by applying the support vector machine (SVM) was addressed. The component analysis was performed initially to extract the features and to reduce the dimensionality of original data features. Kernel parameters selection of support vector machine which has great influence on the performance of defects classification has been discussed in this work. Precisely, we focus extracting the feature from the original signals, adopt component analysis to do feature selection and apply support vector machine to classify the defects. This paper exploits the parameter optimization procedure to ensure the generalization ability of SVM. The result shows that SVM produces promising results and has the potential for use in fault diagnosis.