Support Vector Regression Hyperparameters Research Articles

Fast grid search: A grid search-inspired algorithm for optimizing hyperparameters of support vector regression

Fast grid search: A grid search-inspired algorithm for optimizing hyperparameters of support vector regression

Support vector regression based on improved Harris Hawk optimization algorithm for power load forecasting

Electricity load forecasting is an important part of power system operation, planning and management. Support vector regression (SVR) is a commonly used and efficient method for medium and long-term load forecasting. The hyperparameters of SVR have a very high influence on its performance and are not easily determined. In this paper, an improved Harris Hawk optimization algorithm (TAHHO) is proposed for optimizing the hyperparameters of SVR, and a TAHHO-SVR medium- and long-term prediction model is developed. TAHHHO enhances the convergence and stability of the original algorithm by introducing the survival of the fittest principle and the crossover operator of the artificial tree (AT) algorithm, which is validated in 13 benchmark test functions. The proposed TAHHO-SVR model is used for forecasting the UK National Grid dataset and demonstrates the feasibility and competitiveness of the model, which effectively improves the forecasting accuracy compared to HHO-SVR.

Parameter optimization of SVM algorithm for predicting physical parameters of SF6–Cu mixture plasma under local thermodynamic equilibrium

The physical parameters of SF6–Cu mixture plasma are necessary for arc calculation simulation. The calculation of these parameters is very difficult, but the prediction of the corresponding parameters using the existing database is one of the methods used to solve this difficult problem. The support vector regression (SVR) algorithm can effectively deal with the high-dimensional input vector problem and is widely used in the prediction field. To address the problem that the RBF kernel parameters gamma and penalty coefficient C are difficult to be obtained using the SVR algorithm, which leads to some data not satisfying the prediction accuracy, this paper first uses particle swarm optimization and the gray wolf optimizer to optimize the parameters of SVR, then uses the optimized hyperparameters to predict the data, and finally compares and analyzes the predicted data before and after the optimization. The results show that the optimized SVR parameters obtained using the optimization-seeking algorithm can fit the data better, which verifies the feasibility of the optimization of SVR hyperparameters by the optimization-seeking algorithm.

SVR-TO-APMA: Hybrid efficient modelling and topology framework for stable topology optimization with accelerated performance measure approach

The topology design under uncertainties is an acceptable framework for providing a safe design and optimum configuration of structures. In this regard, the accuracy, efficiency, and robustness of the analytical topology optimization coupled with reliability analysis are the main efforts for reliability-based topology optimization (RBTO) of continuous structures. In the present investigation, a hybrid RBTO approach is developed for robust iterative formulation of reliability loop and an effective and simple optimization approach for topology loop. A multi-level set analytical framework is proposed based on the performance measure approach to seek the optimum results under multi uncertainties in size, mechanical properties, and loads of structures. The considered topology optimization procedure to derive the optimal layout of the structure which is called moving iso-surface threshold (MIST) is a powerful and enhanced topology optimization method in which, unlike other optimization methods, it has the capability of having no direct sensitivity analysis making it easier and simple. The accelerated first-order reliability method is introduced for searching the most probability points in the reliability loop of the RBTO model. The proposed reliability method is developed based on a stable iterative formulation. For reducing the computational burden of the topology optimization loop, the machine learning approach given from the support vector regression (SVR) is applied for estimating the probabilistic constraints in the RBTO model. Four examples of 2D, and 3D compliance, 2D complaint mechanism problem, and frequency maximization are presented for validation and highlighting the abilities in having the most accuracy and safety levels of the proposed hybrid framework named SVR-TO-APMA. The results indicate that the hyperparameters of SVR are directly affected by the accuracy of RBTO results. The stability of the RBTO model is shown by the accelerated PMA and the optimum layout is different from topology optimization in comparison with the RBTO results for 3D compliance and frequency maximization problems. The optimum design with uncertainties is increased by about 40%–90% with a reliability index of 3 compared to TO results for the studied example.

Hybrid modeling of the nonlinear behaviors for magnetorheological energy absorber

Precise forecast of strong nonlinear behaviors of magnetorheological energy absorber (MREA) plays a significant role in its semi-active control and engineering applications. This study proposes a hybrid model based on the support vector regression (SVR) and adaptive differential evolution (i.e., JADE) algorithm for MREA, called the JADE-SVR model. The JADE algorithm is employed to find the optimal hyperparameters of SVR to improve prediction accuracy and generalization ability. The JADE-SVR model is developed using the training data randomly partitioned from an impact test database, and then applied to predict the nonlinear behaviors of MREA in the test set, including force–time, force–displacement and force–velocity characteristics. It is demonstrated that the JADE-SVR model is capable of forecasting strong nonlinear behaviors of MREA under various impact conditions. Moreover, the JADE-SVR model is compared with other SVR-based models to validate its prediction accuracy and convergence rate further. In addition to four statistical indicators, a novel desirability function is used to assess the overall performance of these models. The results show that the JADE-SVR model has the optimal values for all the performance indices, and the largest overall desirability of 0.6274. Therefore, it can be concluded that the JADE-SVR model is the best one for forecasting the dynamic behaviors of MREA under high-impact loadings.

Support vector regression with heuristic optimization algorithms for predicting the ground surface displacement induced by EPB shield tunneling

Machine learning method with heuristic optimization algorithms is proposed to predict the stratum displacement induced by earth pressure balanced shield tunneling. Support vector regression is used as the machine learning method. Four heuristic intelligent optimization algorithms, namely, genetic algorithm, particle swarm optimization, grey wolf optimizer and sparrow search algorithm, are applied to optimize the two hyperparameters of support vector regression model, namely, penalty factor and bandwidth term. Simulated annealing algorithm is introduced to show the necessity of using heuristic algorithms. Mean square error of k-fold cross validation is considered as the fitness function for optimization algorithms. Normalization method and dummy variables are used for data preprocessing. For 115 samples from field measurement, 92 samples are used as the training set, and 23 samples are used as the test set. Three categories of parameters, namely, shield tunneling parameters, tunnel geometrical parameters and stratum types, are used as input parameters for the proposed method. Correlations among parameters are analyzed by Pearson correlation coefficient. The prediction results show that grey wolf optimizer and sparrow search algorithm are suitable methods for determining hyperparameters of support vector regression due to higher accuracy, efficiency, and stability.

Remaining Useful Life Prediction for Fuel Cell Based on Support Vector Regression and Grey Wolf Optimizer Algorithm

Remaining useful life prediction is an important way to improve the durability and reduce the cost of the proton exchange membrane fuel cell. This paper presents a novel method to predict the remaining useful life of proton exchange membrane fuel cell under different load currents based on support vector regression and grey wolf optimizer algorithm. The proposed method considers the influence of 17 operating conditions and historical voltage. Firstly, the measured data are reconstructed through robust locally weighted smoothing method to reduce the calculation amount and filter disturbances. Then, support vector regression with fewer hyperparameters is used to establish the degradation model. Finally, the hyperparameters of support vector regression are optimized through grey wolf optimizer algorithm to improve the accuracy of degradation prediction. The proposed method is validated by two degradation experiments under different load currents. The test results show that grey wolf optimizer algorithm can effectively improve the accuracy of degradation prediction based on support vector regression. Compared with other methods, the proposed method has the highest accuracy. The proposed method can predict the fuel cell degradation with a mean absolute percentage error of less than 0.3%. The proposed method can predict the remaining useful life of 492 hours.

The potential of a novel support vector machine trained with modified mayfly optimization algorithm for streamflow prediction

ABSTRACT This paper focuses on the development of a robust accurate streamflow prediction model by balancing the abilities of exploitation and exploration to find the best parameters of a machine learning model. To do so, the simulated annealing (SA) algorithm is integrated with the mayfly optimization algorithm (MOA) as SAMOA to determine the optimal hyper-parameters of support vector regression (SVR) to overcome the exploration weakness of the MOA method. The proposed method is compared with the classical SVR and hybrid SVR-MOA. To examine the accuracy of the selected methods, monthly hydroclimatic data from Jhelum River Basin is used to predict the monthly streamflow on the basis of RMSE, MAE, NSE, and R2 indices. Test results show that the SVR-SAMOA outperformed the SVR-MOA and SVR models. SVR-SAMOA reduced the prediction errors of the SVR-MOA and SVR models by decreasing the RMSE and the MSE from 21.4% to 14.7% and from 21.7% to 15.1%, respectively, in the test stage.

Machine learning modeling to identify affinity improved biobetter anticancer drug trastuzumab and the insight of molecular recognition of trastuzumab towards its antigen HER2

In the present study, a machine learning (ML) model was developed to predict the epistatic phenomena of combination mutants to improve the anticancer antibody-drug trastuzumab's binding affinity towards its antigen human epidermal growth factor receptor 2 (HER2). An ML algorithm, Support Vector Regression (SVR) was used to develop ML models with a data set consists of 193 affinity values of single mutants of trastuzumab and its associated various amino acid sequence derived descriptors. The subset selection of descriptors and SVR hyperparameters were done using the Genetic Algorithm (GA) within the SVR and the wrapper approach called GA-SVR. A 100 evolutionary cycles of GA produced the best 100 probable GA-SVR models based on their fitness score estimated using a stratified 5 fold cross-validation procedure. The final ML model found to be highly predictive of test data set of six combination mutants and one single mutant with = 0.71. The analysis of descriptors in the ML model highlighted the importance of mutant induced secondary structural variation causes the binding affinity variation of the trastuzumab. The same was verified using a short 20 ns and a long 100 ns in duplicate molecular dynamics simulation of a wild and mutant variant of trastuzumab. The secondary structure induced affinity change due to mutations in the CDR-H3 is a novel insight that came out of this study. That should help rational mutant selection to develop a biobetter trastuzumab with a multifold improved binding affinity into the market quickly. Communicated by Ramaswamy H. Sarma

Evolving support vector regression using Grey Wolf optimization; forecasting the geomechanical properties of rock

The geomechanical properties of rock, including shear strength (SS) and uniaxial compressive strength (UCS), are very important parameters in designing rock structures. To improve the accuracy of SS and UCS prediction, this study presented an evolving support vector regression (SVR) using Grey Wolf optimization (GWO). To examine the feasibility and applicability of the SVR-GWO model, the differential evolution (DE) and artificial bee colony (ABC) algorithms were also used. In other words, the SVR hyperparameters were tuned using the GWO, DE, and ABC algorithms. To implement the proposed models, a comprehensive database accessible in an open-source was used in this study. Finally, the comparative experiments such as root mean square error (RMSE) were conducted to show the superiority of the proposed models. The SVR-GWO model predicted the SS and UCS with RMSE of 0.460 and 3.208, respectively, while, the SVR-DE model predicted the SS and UCS with RMSE of 0.542 and 5.4, respectively. Furthermore, the SVR-ABC model predicted the SS and UCS with RMSE of 0.855 and 5.033, respectively. The aforementioned results clearly exhibited the applicability as well as the usability of the proposed SVR-GWO model in the prediction of both SS and UCS parameters. Accordingly, the SVR-GWO model can be also applied to solving various complex systems, especially in geotechnical and mining fields.

AGA-SVR-based selection of feature subsets and optimization of parameter in regional soil salinization monitoring

ABSTRACTSalinization is a major eco-environmental threat in arid and semi-arid regions. Machine learning combined with salinization-related factors extracted from satellite images and Digital Elevation Model (DEM) data to monitor salinization is popular in recent years. The hyper-parameters referring to the parameters prior to fitting the model to the data and the features referring to the factors used for model establishment are vital to modelling accuracy, while the optimization of the above two was not given enough attention. In this study, we proposed a novel approach for simultaneously identifying the input features and hyper-parameters of Support Vector Regression (SVR) based on Adaptive Genetic Algorithm (AGA) for quantitative assessment of salinization in Weigan-Kuqa river delta oasis (Wei-Ku oasis), Sangong River Basin, and Qitai oasis of Xinjiang. First, a total of 41 salinization-related factors of 7 categories were extracted from Landsat 5 TM and DEM data. In each sub-region, the Pearson’s correlation analysis was developed between Soil Salt Content (SSC) and salinization-related factors, and the factors significantly correlated with SSC were arranged in descending order of absolute correlation coefficient to form the Candidate Feature Variables (CFVs). The ration of Coefficient of Determination (R2) and Root Mean Square Error (RMSE) multiplied by 1000 was considered as the fitness function of Genetic Algorithm (GA) and AGA. The CFVs and hyper-parameters were combined together and binary coded, then brought into the AGA and GA for simultaneous feature selection and hyper-parameters optimization of SVR, and established salinization monitoring models (AGA-SVR, GA-SVR). In order to highlight the importance of identifying the input features and hyper-parameter to modelling accuracy, the GS-SVR with all the CFVs as input was established using Grid Search (GS) algorithm to optimize the hyper-parameters. Finally, the salinization maps predicted by three models were compared. The results showed that the sensitivity of salinization-related factors to SSC varied with regions, and 25, 16, 24 salinization-related factors were selected as CFVs in Wei-Ku oasis, Sangong River Basin, and Qitai oasis, respectively. Compared with GS-SVR, the GA-SVR and AGA-SVR got more accurate salinization monitoring with fewer features as input, and fitness generated by AGA-SVR increased by 25.968% in Wei-Ku oasis, 25.159% in Sangong River Basin, 27.568% in Qitai oasis, respectively. Both nature and human factors lead to salinization. The difference of Land Surface Temperature (LST) was the main contributor of different salinization between Wei-Ku oasis of southern Xinjiang and the other two sub-regions of northern Xinjiang. The differences in soil texture, irrigation methods, and livestock carrying capacity were the main factors resulting in differences of salinization in Qitai oasis and Sangong River Basin. Our study shows that the proposed approach can provide technical support for accurate salinization monitoring.

Short-Term Forecasting of Natural Gas Consumption Using Factor Selection Algorithm and Optimized Support Vector Regression

Forecasting of natural gas consumption has been essential for natural gas companies, customers, and governments. However, accurate forecasting of natural gas consumption is difficult, due to the cyclical change of the consumption and the complexity of the factors that influence the consumption. In this work, we constructed a hybrid artificial intelligence (AI) model to predict the short-term natural gas consumption and examine the effects of the factors in the consumption cycle. The proposed model combines factor selection algorithm (FSA), life genetic algorithm (LGA), and support vector regression (SVR), namely, as FSA-LGA-SVR. FSA is used to select factors automatically for different period based on correlation analysis. The LGA optimized SVR is utilized to provide the prediction of time series data. To avoid being trapped in local minima, the hyper-parameters of SVR are determined by LGA, which is enhanced due to newly added “learning” and “death” operations in conventional genetic algorithm. Additionally, in order to examine the effects of the factors in different period, we utilized the recent data of three big cities in Greece and divided the data into 12 subseries. The prediction results demonstrated that the proposed model can give a better performance of short-term natural gas consumption forecasting compared to the estimation value of existing models. Particularly, the mean absolute range normalized errors of the proposed model in Athens, Thessaloniki, and Larisa are 1.90%, 2.26%, and 2.12%, respectively.

Application of hybrid support vector regression artificial bee colony for prediction of MMP in CO2-EOR process

Minimum miscibility pressure (MMP) is a key parameter in the successful design of miscible gases injection such as CO2 flooding for enhanced oil recovery process (EOR). MMP is generally determined through experimental tests such as slim tube and rising bubble apparatus (RBA). As these tests are time-consuming and their cost is very expensive, several correlations have been developed. However, and although the simplicity of these correlations, they suffer from inaccuracies and bad generalization due to the limitation of their ranges of application. This paper aims to establish a global model to predict MMP in both pure and impure CO2-crude oil in EOR process by combining support vector regression (SVR) with artificial bee colony (ABC). ABC is used to find best SVR hyper-parameters. 201 data collected from authenticated published literature and covering a wide range of variables are considered to develop SVR-ABC pure/impure CO2-crude oil MMP model with following inputs: reservoir temperature (TR), critical temperature of the injection gas (Tc), molecular weight of pentane plus fraction of crude oil (MWC5+) and the ratio of volatile components to intermediate components in crude oil (xvol/xint). Statistical indicators and graphical error analyses show that SVR-ABC MMP model yields excellent results with a low mean absolute percentage error (3.24%) and root mean square error (0.79) and a high coefficient of determination (0.9868). Furthermore, the results reveal that SVR-ABC outperforms either ordinary SVR with trial and error approach or all existing methods considered in this work in the prediction of pure and impure CO2-crude oil MMP. Finally, the Leverage approach (Williams plot) is done to investigate the realm of prediction capability of the new model and to detect any probable erroneous data points.

Prediction Modeling Framework With Bootstrap Aggregating for Noisy Resistance Spot Welding Data

In resistance spot welding (RSW), data inconsistency is a well-known issue. Such inconsistent data are usually treated as noise and removed from the original dataset before conducting analyses or constructing prediction models. This may not be desirable for all design and manufacturing applications since data that are often considered noise can contain important information in determining weldment design, and proper welding conditions. In this paper, we present the Meta2 prediction framework to provide cost-effective opportunities for proper welding material and condition selection from the noisy RSW quality data. The Meta2 framework employs bootstrap aggregating with support vector regression (SVR) to improve the prediction accuracy on the noisy RSW data with computational efficiency. Hyper-parameters for SVR are selected by particle swarm optimization (PSO) with meta-modeling to reduce the computational cost. Experiments on three artificially generated noisy datasets and a real RSW dataset indicate that Meta2 is capable of providing satisfactory solutions with a noticeably reduced computational cost. The authors find Meta2 promising as a potential prediction model algorithm for this type of noisy data.