Support Vector Machine Parameter Optimization Research Articles

SVM Parameter Optimization using Grid Search and Genetic Algorithm to Improve Classification Performance

Machine Learning algorithms have been widely used to solve various kinds of data classification problems. Classification problem especially for high dimensional datasets have attracted many researchers in order to find efficient approaches to address them. However, the classification problem has become very complicated and computationally expensive, especially when the number of possible different combinations of variables is so high. Support Vector Machine (SVM) has been proven to perform much better when dealing with high dimensional datasets and numerical features. Although SVM works well with default value, the performance of SVM can be improved significantly using parameter optimization. We applied two methods which are Grid Search and Genetic Algorithm (GA) to optimize the SVM parameters. Our experiment showed that SVM parameter optimization using grid search always finds near optimal parameter combination within the given ranges. However, grid search was very slow; therefore it was very reliable only in low dimensional datasets with few parameters. SVM parameter optimization using GA can be used to solve the problem of grid search. GA has proven to be more stable than grid search. Based on average running time on 9 datasets, GA was almost 16 times faster than grid search. Futhermore, the GA’s results were slighlty better than the grid search in 8 of 9 datasets.

Biomedical classification application and parameters optimization of mixed kernel SVM based on the information entropy particle swarm optimization

The types of kernel function and relevant parameters’ selection in support vector machine (SVM) have a major impact on the performance of the classifier. In order to improve the accuracy and generalization ability of the model, we used mixed kernel function SVM classification algorithm based on the information entropy particle swarm optimization (PSO): on the one hand, the generalization ability of classifier is effectively enhanced by constructing a mixed kernel function with global kernel function and local kernel function; on the other hand, the accuracy of classification is improved through optimization for related kernel parameters based on information entropy PSO. Compared with PSO-RBF kernel and PSO-mixed kernel, the improved PSO-mixed kernel SVM can effectively improve the classification accuracy through the classification experiment on biomedical datasets, which would not only prove the efficiency of this algorithm, but also show that the algorithm has good practical application value in biomedicine prediction.

Traffic fatalities prediction based on support vector machine

To effectively predict traffic fatalities and promote the friendly development of transportation, a prediction model of traffic fatalities is established based on support vector machine (SVM). As the prediction accuracy of SVM largely depends on the selection of parameters, Particle Swarm Optimization (PSO) is introduced to find the optimal parameters. In this paper, small sample and nonlinear data are used to predict fatalities of traffic accident. Traffic accident statistics data of China from 1981 to 2012 are chosen as experimental data. The input variables for predicting accident are highway mileage, vehicle number and population size while the output variables are traffic fatality. To verify the validity of the proposed prediction method, the back-propagation neural network (BPNN) prediction model and SVM prediction model are also used to predict the traffic fatalities. The results show that compared with BPNN prediction model and SVM model, the prediction model of traffic fatalities based on PSO-SVM has higher prediction precision and smaller errors. The model can be more effective to forecast the traffic fatalities. And the method using particle swarm optimization algorithm for parameter optimization of SVM is feasible and effective. In addition, this method avoids overcomes the problem of “over learning” in neural network training progress

Development of KSVGRNN: A hybrid soft computing technique for estimation of boiler flue gas components

Abstract In this research paper, a novel hybrid technique named KSVGRNN, which combines a multi-class support vector machine (SVM) and a generalized regression neural network (GRNN), has been developed for obtaining the composition of boiler flue gas mixtures. This hybridization was made by the support of K-means clustering algorithm and grid search technique. In the first phase, K-Means clustering technique has been utilized and the size of the training vectors has been reduced by employing a multiclass SVM. In the second, a GRNN has been trained for estimating the individual gas concentration in the flue gas mixture. The reduction of training vectors through SVM has been shown to improve the generalization capability of GRNN. Grid search has been utilized to obtain the optimal parameters of SVM. This hybrid technique has been validated by measuring its performance by processing volatile organic component (VOC) data acquired from quartz crystal microbalance (QCM) and SnO2 semiconductor type sensors utilized by other researchers in this domain. Further studies have been carried out to assess the discriminating and estimation capability of the proposed hybrid technique for real-time flue gas data obtained from two different analyzers namely ORSAT® and KANE®. The outcome of these studies, observations and analysis clearly indicate the exceptional performance of the proposed hybrid model in classifying and estimating the flue gas components in the machine (Analyzer) independent manner.

PSO-SVM-Based Online Locomotion Mode Identification for Rehabilitation Robotic Exoskeletons.

Locomotion mode identification is essential for the control of a robotic rehabilitation exoskeletons. This paper proposes an online support vector machine (SVM) optimized by particle swarm optimization (PSO) to identify different locomotion modes to realize a smooth and automatic locomotion transition. A PSO algorithm is used to obtain the optimal parameters of SVM for a better overall performance. Signals measured by the foot pressure sensors integrated in the insoles of wearable shoes and the MEMS-based attitude and heading reference systems (AHRS) attached on the shoes and shanks of leg segments are fused together as the input information of SVM. Based on the chosen window whose size is 200 ms (with sampling frequency of 40 Hz), a three-layer wavelet packet analysis (WPA) is used for feature extraction, after which, the kernel principal component analysis (kPCA) is utilized to reduce the dimension of the feature set to reduce computation cost of the SVM. Since the signals are from two types of different sensors, the normalization is conducted to scale the input into the interval of [0, 1]. Five-fold cross validation is adapted to train the classifier, which prevents the classifier over-fitting. Based on the SVM model obtained offline in MATLAB, an online SVM algorithm is constructed for locomotion mode identification. Experiments are performed for different locomotion modes and experimental results show the effectiveness of the proposed algorithm with an accuracy of 96.00% ± 2.45%. To improve its accuracy, majority vote algorithm (MVA) is used for post-processing, with which the identification accuracy is better than 98.35% ± 1.65%. The proposed algorithm can be extended and employed in the field of robotic rehabilitation and assistance.

Forecasting Range Volatility Using Support Vector Machines with Improved PSO Algorithms

Financial volatility forecasting is an important content in financial derivatives pricing, financial risk management, portfolio allocation. Support vector machines (SVM) has been successfully used in modeling and forecasting volatility. To enhance the volatility forecasting ability of SVM, this paper introduced three improved particle swarm optimization (IPSO) algorithms into SVM for forecasting range volatility. SVM was used to construct the volatility forecasting model. And, the three IPSO algorithms: stochastic inertia weight PSO (SIWPSO), time-varying acceleration coefficients PSO (TVACPSO) and two-order oscillating particle swarm optimization (TOOPSO) were applied to finding the optimal parameters in SVM, respectively. The accuracy of the proposed models in forecasting range volatility was demonstrated by using three stock indices in China stock market. The empirical results imply that compared to SVM with SIWPSO and TVACPSO algorithms, SVM with TOOPSO algorithm produces higher accuracy in forecasting range volatility and faster speed on searching for the optimal parameters of SVM.

Novel hybrid SVM-TLBO forecasting model incorporating dimensionality reduction techniques

In this paper, we present a highly accurate forecasting method that supports improved investment decisions. The proposed method extends the novel hybrid SVM-TLBO model consisting of a support vector machine (SVM) and a teaching-learning-based optimization (TLBO) method that determines the optimal SVM parameters, by combining it with dimensional reduction techniques (DR-SVM-TLBO). The dimension reduction techniques (feature extraction approach) extract critical, non-collinear, relevant, and de-noised information from the input variables (features), and reduce the time complexity. We investigated three different feature extraction techniques: principal component analysis, kernel principal component analysis, and independent component analysis. The feasibility and effectiveness of this proposed ensemble model were examined using a case study, predicting the daily closing prices of the COMDEX commodity futures index traded in the Multi Commodity Exchange of India Limited. In this study, we assessed the performance of the new ensemble model with the three feature extraction techniques, using different performance metrics and statistical measures. We compared our results with results from a standard SVM model and an SVM-TLBO hybrid model. Our experimental results show that the new ensemble model is viable and effective, and provides better predictions. This proposed model can provide technical support for better financial investment decisions and can be used as an alternative model for forecasting tasks that require more accurate predictions.

OF GENES AND MACHINES: APPLICATION OF A COMBINATION OF MACHINE LEARNING TOOLS TO ASTRONOMY DATA SETS

ABSTRACT We apply a combination of genetic algorithm (GA) and support vector machine (SVM) machine learning algorithms to solve two important problems faced by the astronomical community: star–galaxy separation and photometric redshift estimation of galaxies in survey catalogs. We use the GA to select the relevant features in the first step, followed by optimization of SVM parameters in the second step to obtain an optimal set of parameters to classify or regress, in the process of which we avoid overfitting. We apply our method to star–galaxy separation in Pan-STARRS1 data. We show that our method correctly classifies 98% of objects down to i P1 = 24.5 , with a completeness (or true positive rate) of 99% for galaxies and 88% for stars. By combining colors with morphology, our star–galaxy separation method yields better results than the new SExtractor classifier spread_model, in particular at the faint end ( i P1 > 22 ). We also use our method to derive photometric redshifts for galaxies in the COSMOS bright multiwavelength data set down to an error in ( 1 + z ) of σ = 0.013 , which compares well with estimates from spectral energy distribution fitting on the same data ( σ = 0.007 ) while making a significantly smaller number of assumptions.

Optimal dissolved gas ratios selected by genetic algorithm for power transformer fault diagnosis based on support vector machine

Dissolved gas analysis (DGA) of oil is used to detect the incipient fault of power transformers. This paper presents a new approach for transformer fault diagnosis based on selected gas ratios concentrated in oil and support vector machine (SVM). Firstly, based on IEC TC 10 database, the optimal dissolved gas ratios (ODGR) are obtained by genetic algorithm (GA) that is designed for simultaneous DGA ratios selection and SVM parameters optimization. Three traditional methods, namely, DGA data with SVM and back propagation neural network (BPNN), IEC criteria, and IEC three-key gas ratios with SVM and BPNN are employed for effectiveness comparison. The fault diagnosis results of IEC TC 10 database show that the proposed ODGR with SVM may be used as an alternative tool for transformer fault diagnosis. In addition, the robustness and generalization ability of ODGR is confirmed by the diagnosis accuracy of 87.18% of China DGA samples. The obtained results illustrate that it is preferable to apply the proposed ODGR to transformer fault diagnosis with the assistance of SVM.

Prediction of Daily Dewpoint Temperature Using a Model Combining the Support Vector Machine with Firefly Algorithm

AbstractIn this research work, a hybrid approach of integrating a support vector machine (SVM) with firefly algorithm (FFA) is proposed to predict daily dewpoint temperature (Tdew). The main aim of employing FFA is to identify the optimal SVM parameters and provide the possibility of enhancing the SVM’s capability. The weather data sets including 10 years of measured-daily average air temperature (Tavg), relative humidity (Rh), atmospheric pressure (P), and Tdew for an Iranian city have been utilized. Seven different sets of parameters with one, two, and three of the considered parameters serve as inputs to establish seven models. The capability of the SVM-FFA method is compared against SVM, artificial neural network (ANN), and genetic programming (GP) to demonstrate its efficiency and viability. It is found that further precision is achieved for Model 7 established based on all approaches utilizing three inputs of Tavg, Rh, and P. The obtained results clearly indicate that the SVM-FFA method, by providin...

Network Traffic Classification using Genetic Algorithms based on Support Vector Machine

In recent years，machine learning method has been applied to the extensive research on traffic classification. In these methods, SVM (Support vector machine) is a supervised learning which can improve generalization ability of learning machine effectively. However, the penalty parameter C and kernel function parameter  are generally given by test experience during training of SVM. How to determine the optimal parameters of SVM is a problem to be solved. We proposed a method to deriving the optimal parameters of SVM based on GA (Genetic algorithm).This method does not need to traverse all the parameter points. The method extracts a certain number population from random solutions, and ultimately produces SVM optimal parameters according to the specific rules of operation. Through the method, we derived the optimal parameters combination C and  of SVM. The accuracy of network traffic classification is improved greatly.

Combatting nonlinear phase noise in coherent optical systems with an optimized decision processor based on machine learning

An effective machine learning algorithm, the support vector machine (SVM), is presented in the context of a coherent optical transmission system. As a classifier, the SVM can create nonlinear decision boundaries to mitigate the distortions caused by nonlinear phase noise (NLPN). Without any prior information or heuristic assumptions, the SVM can learn and capture the link properties from only a few training data. Compared with the maximum likelihood estimation (MLE) algorithm, a lower bit-error rate (BER) is achieved by the SVM for a given launch power; moreover, the launch power dynamic range (LPDR) is increased by 3.3dBm for 8 phase-shift keying (8PSK), 1.2dBm for QPSK, and 0.3dBm for BPSK. The maximum transmission distance corresponding to a BER of 1×10−3 is increased by 480km for the case of 8PSK. The larger launch power range and longer transmission distance improve the tolerance to amplitude and phase noise, which demonstrates the feasibility of the SVM in digital signal processing for M-PSK formats. Meanwhile, in order to apply the SVM method to 16 quadratic amplitude modulation (16QAM) detection, we propose a parameter optimization scheme. By utilizing a cross-validation and grid-search techniques, the optimal parameters of SVM can be selected, thus leading to the LPDR improvement by 2.8dBm. Additionally, we demonstrate that the SVM is also effective in combating the laser phase noise combined with the inphase and quadrature (I/Q) modulator imperfections, but the improvement is insignificant for the linear noise and separate I/Q imbalance. The computational complexity of SVM is also discussed. The relatively low complexity makes it possible for SVM to implement the real-time processing.

Intelligent Analysis Method of Gear Faults Based on FRWT and SVM

An intelligent analysis method for gear faults based on fractional wavelet transform (FRWT) and support vector machine (SVM) is proposed. Based on this method, FRWT is used to eliminate noise from the gear vibration signal, and the vibration signal after noise elimination is carried thought wavelet packet decomposition and reconstruction. A sequence corresponding to the signal is constructed consisting of the module with the highest level wavelet coefficients after decomposition and feature vectors corresponding to the energy sequence which were obtained by calculation. Then, a particle optimization method is used to optimize SVM parameters, and the feature vectors as training samples are input into SVM for training while the test samples are input for fault recognition. Experimental results show that the gear fault analysis method proposed in this paper is able to effectively extract the weak fault signal. The accuracy rate for identification of the type of gear fault reached 96.7%.

SVM based zonal setting of Mho relay for shunt compensated transmission line

Abstract Presence of shunt FACTS (Flexible Alternating Current Transmission System) device significantly affect the performance of protection system and may create security and reliability issues. This paper introduces a novel approach for zone identification in transmission line employing shunt FACTS devices such as Static Var Compensator (SVC) and Static Synchronous Compensator (STATCOM). The technique depends on the modified apparent impedance seen by the impedance relay for all possible operating conditions. In this technique the Support Vector Machine (SVM) is used for discriminating faulty zone (zone 1 or zone 2). In addition an optimization technique viz. Genetic Algorithm (GA) is used to optimize SVM parameters. A typical 230 kV system was simulated in PSCAD/EMTP software and the results show that the proposed scheme is secure, accurate and reliable under the wide variation in power system parameters like load angle, fault resistance, fault location, inception angle and compensation level.

Rough set-support vector machine-based real-time monitoring model of safety status during dangerous dam reinforcement

There are usually a small amount and short sequence of prototype monitoring data on structural behavior during dangerous dam reinforcement. According to above data characteristics, some methods, such as support vector machine, particle swarm optimization, genetic algorithm, rough set, are combined to build the real-time monitoring model of safety status during structural reinforcement of dangerous dam. Firstly, the construction principle on standard support vector machine-based monitoring model of safety status is demonstrated. To improve the modeling accuracy and efficiency, particle swarm optimization and genetic algorithm are introduced to implement the support vector machine parameters optimization. Secondly, the ability on data mining in rough set is developed to determine the input vector of support vector machine. An approach is presented to obtain the simplified non-linear mapping relationship between dam behavior (effect-quantity) and its cause (influence-quantity). An index and its calculating formula are proposed to measure the influence-quantity importance. A real-time monitoring model based on rough set-support vector machine is established to describe reasonably the working mechanism during dangerous dam reinforcement. Lastly, the fitting and prediction capability of above monitoring model is demonstrated with an actual case.

A peer to peer traffic identification method based on wavelet and particle swarm optimization algorithm

With the rapid development of network, the peer-to-peer (P2P) traffic has become one of the most important traffics on the Internet; meanwhile, it also brings many security problems to the network management. Thus, nowadays, P2P traffic identification is the hottest topic of P2P traffic management. Much effort has been made on this topic, however, effectiveness remains an issue and the classification performance needs to be further improved. Support vector machine (SVM) has advantages with resolving small samples and high dimension for P2P classification problems. However, the performance of SVM is largely dependent on its kernel and parameters. The traditional kernels are hard to map complicated function with high precision and the traditional parameters tuning methods are of low efficiency and difficult to obtain good parameters. As wavelet kernel function is able to approximate a function with high precision and Particle Swarm Optimization algorithm could tune the optimal parameters for SVM. Hence, in the paper, a novel SVM method based on wavelet kernel and particle swarm optimization algorithm (PSO) is proposed for P2P identification. First, the proposed approach tunes the best parameters for SVM with PSO on training data. Subsequently, the wavelet SVM configured with the best parameters is conducted to identify P2P traffic. Experimental results on campus traffic traces indicate that the proposed method is able to identify popular P2P applications with very high accuracy.

An improved SVM integrated GS-PCA fault diagnosis approach of Tennessee Eastman process

In modern industry, fault diagnosis and process supervision are very important in detecting machinery failures and keeping the stability of production systems. In this paper, a multi-class support vector machine (SVM) based process supervision and fault diagnosis scheme is proposed to predict the status of the Tennessee Eastman (TE) Process. After preprocessing the collected data, principal component analysis (PCA) is firstly used to reduce the feature dimension. Then, to increase prediction accuracy and reduce computation load, the optimization of SVM parameters is accomplished with the grid search (GS) method, which generates comparable classification accuracy to genetic algorithm (GA) and particle swarm optimization (PSO) while being more efficient than the latter two algorithms. Finally, to demonstrate the effectiveness of the proposed SVM integrated GS-PCA fault diagnosis approach, a comparison is made with other related fault diagnosis methods.

A survey of water level fluctuation predicting in Urmia Lake using support vector machine with firefly algorithm

Forecasting lake level at various horizons is a critical issue in navigation, water resource planning and catchment management. In this article, multistep ahead predictive models of predicting daily lake levels for three prediction horizons were created. The models were developed using a novel method based on support vector machine (SVM) coupled with firefly algorithm (FA). The FA was applied to estimate the optimal SVM parameters. Daily water-level data from Urmia Lake in northwestern Iran were used to train, test and validate the used technique. The prediction results of the SVM–FA models were compared to the genetic programming (GP) and artificial neural networks (ANNs) models. The experimental results showed that an improvement in the predictive accuracy and capability of generalization can be achieved by the SVM–FA approach in comparison to the GP and ANN in 1 day ahead lake level forecast. Moreover, the findings indicated that the developed SVM–FA models can be used with confidence for further work on formulating a novel model of predictive strategy for lake level prediction.

Automatic target recognition of moving target based on empirical mode decomposition and genetic algorithm support vector machine

In order to improve measurement accuracy of moving target signals, an automatic target recognition model of moving target signals was established based on empirical mode decomposition (EMD) and support vector machine (SVM). Automatic target recognition process on the nonlinear and non-stationary of Doppler signals of military target by using automatic target recognition model can be expressed as follows. Firstly, the nonlinearity and non-stationary of Doppler signals were decomposed into a set of intrinsic mode functions (IMFs) using EMD. After the Hilbert transform of IMF, the energy ratio of each IMF to the total IMFs can be extracted as the features of military target. Then, the SVM was trained through using the energy ratio to classify the military targets, and genetic algorithm (GA) was used to optimize SVM parameters in the solution space. The experimental results show that this algorithm can achieve the recognition accuracies of 86.15%, 87.93%, and 82.28% for tank, vehicle and soldier, respectively.

Parameters Optimization of SVM Based on Improved FOA and Its Application in Fault Diagnosis

In most cases, fault diagnosis is essentially a pattern recognition problem and support vector machine (SVM) provides a new solution for the diagnosis problem of systems in which the fault samples are few. However, the parameters selection in SVM has significant influence on the diagnosis performance. In this paper, improved fruit fly optimization algorithm (IFOA), which is basically the standard fruit fly optimization algorithm (FOA) combined with Levy flight search strategy, is proposed to determine the SVM parameters. Some benchmark datasets are used to evaluate the proposed algorithm. Furthermore, the proposed method is used to diagnose the faults of hydraulic pump. Experiments and engineering application show that the proposed method outperforms standard FOA, genetic algorithm (GA) and particle swarm optimization (PSO) methods.