Optimized Support Vector Regression Model Research Articles

Prediction of landslide step-like displacement using factor preprocessing-based hybrid optimized SVR model in the Three Gorges Reservoir, China

Artificial intelligence (AI) is gaining popularity in landslide displacement modeling due to its strong generalization capacity and ability to accurately represent complex, nonlinear laws. However, few studies have examined the validity of factor selection and the impact of data noise on the model. To address these issues, we employ various analyses, including Pearson correlation coefficient and grey correlation degree analyses to diagnose redundant and highly correlated factors, respectively. Singular spectrum analysis (SSA) is employed for denoising the data, leveraging the benefits of the hankel matrix and minimal parameters. This technique facilitates the decomposition of displacement into trend and periodic terms. The trend term displacement is fitted with a polynomial, and the periodic term displacement is predicted using a support vector regression (SVR) model. The hybrid-optimized SVR by particle swarm optimization-gravitational search algorithm (PSO-GSA) is used to improve the model's performance and prediction ability. We apply the proposed method to the Bazimen and Baishuihe landslides in Zigui County for model validation. The results show that denoising the input factor data reduces the root mean square error (RMSE) and mean absolute error (MAE) and increases the correlation coefficient (R). Furthermore, the PSO-GSA algorithm exhibits excellent generalization ability and fast optimization speed in optimizing parameters. The correlation coefficient increase to 0.99 when using the hybrid-optimized SVR model for displacement prediction.

Support Vector Regression Model for Determining Optimal Parameters of HfAlO-Based Charge Trapping Memory Devices

The production and optimization of HfAlO-based charge trapping memory devices is central to our research. Current optimization methods, based largely on experimental experience, are tedious and time-consuming. We examine various fabrication parameters and use the resulting memory window data to train machine learning algorithms. An optimized Support Vector Regression model, processed using the Swarm algorithm, is applied for data prediction and process optimization. Our model achieves a MSE of 0.47, an R2 of 0.98856, and a recognition accuracy of 90.3% under cross-validation. The findings underscore the effectiveness of machine learning algorithms in non-volatile memory fabrication process optimization, enabling efficient parameter selection or outcome prediction.

Prediction using multi-objective slime mould algorithm optimized support vector regression model

Different from the traditional neural network models based on empirical risk minimization principle, support vector regression (SVR) minimizes the upper limit of generalization error by the structural risk minimization principle. SVR is suitable for solving nonlinear, small-sample, high-dimensional modelling, and pattern recognition problems. However, the selection of hyperparameters of SVR can significantly affect the prediction accuracy and computational time. This paper proposes a method to optimize the SVR hyperparameters through an intelligent optimization algorithm, specifically multi-objective slime mould algorithm (MOSMA). MOSMA is a new multi-objective optimization algorithm with strong global search ability and fast convergence. The principle and calculation process of MOSMA are first described in detail. The analytical method for transforming the SVR hyperparameter optimization problem into a multi-objective problem is then presented. The process of optimizing two SVR hyperparameters (the penalty coefficient and radial basis function kernel parameter) based on MOSMA is described in steps. Meanwhile, the procedures of algorithm parameter setting, fitness function design, and population update are given. To investigate the characteristics and performance of MOSMA-SVR, two datasets are used to predict the vibration trend of the spindle of CNC milling machine in one and the maximum bending normal stress in the cutting process in the other. The performance of MOSMA-SVR is evaluated by multiple statistical indexes and compared with seven other prediction models (GA-SVR, PSO-SVR, ABC-SVR, GWO-SVR, SSA-SVR, SMA-SVR and MOWOA-SVR).

Predicting the energy consumption in buildings using the optimized support vector regression model

Predicting the energy consumption in buildings using the optimized support vector regression model

A consumer emotion analysis system based on support vector regression model.

The effective means to stimulate economic growth is to enhance consumers' consumption capacity. Because many consumers have different consumption habits, they will pay different attention to products. Even the same consumer will have different shopping experiences when buying the same product at different times. By mining the online comments of consumers on the online fitness platform, we can find the characteristics of fitness projects that consumers care about. Analyzing consumers' emotional tendencies towards the characteristics of fitness programs will help online fitness platforms adjust the quality and service direction of fitness programs in a timely manner. At the same time, it can also provide purchase advice and suggestions for other consumers. Based on this goal, this study uses an optimized support vector regression (SVR) model to build a consumer sentiment analysis system, so as to predict the consumer's willingness to pay. The optimized SVR model uses the region convolution neural network (RCNN) to extract features from the dataset, and uses feature data to train the SVR model. The experimental results show that the SVR model optimized by RCNN is more accurate. The improvement of the accuracy of consumer sentiment analysis can accurately help businesses promote and publicize, and increase sales. On the other hand, the increase in the accuracy of emotion analysis can also help users quickly locate their favorite fitness projects, saving browsing time. To sum up, the emotional analysis system for consumers in this paper has good practical value.

Real-time evaluation of the dynamic Young’s modulus for composite formations based on the drilling parameters using different machine learning algorithms

The dynamic Young’s modulus (Edyn) is a parameter needed for optimizing different aspects related to oil well designing. Currently, Edyn is determined from the knowledge of the formation bulk density, in addition to the shear and compressional velocities, which are not always available. This study introduces three machine learning (ML) models, namely, random forest (RF), adaptive neuro-fuzzy inference system with subtractive clustering (ANFIS-SC), and support vector regression (SVR), for estimation of the Edyn from only the real-time available drilling parameters. The ML models were learned on 2054 datasets collected from Well-A and then tested and validated on 871 and 2912 datasets from Well-B and Well-C, respectively. The results showed that the three optimized ML models accurately predicted the Edyn in the three oil wells considered in this study. The optimized SVR model outperformed both the RF and ANFIS-SC models in evaluating the Edyn in all three wells. For the validation data, the Edyn was assessed accurately with low average absolute percentage errors of 3.64%, 6.74%, and 1.03% using the optimized RF, ANFIS-SC, and SVR models, respectively.

Bayesian information criterion based data-driven state of charge estimation for lithium-ion battery

Accurate state of charge (SOC) estimation is essential for the safe and reliable operation of Li-ion batteries. To solve the problem of poor generalisation caused by over-fitting, this paper presents a combination algorithm based on feature selection to estimate battery SOC. Firstly, a portion of the features is extracted from the extended Kalman filtering (EKF) results. It forms the set of features to be selected with four other measured features. Secondly, the optimal feature subset is adopted by designing a wrapped feature screening framework based on the Bayesian information criterion (BIC). Finally, the selected combination of features is adopted to train the support vector regression (SVR) model, which is applied to the battery SOC estimation. The experimental results reveal that the combination strategy of EKF and SVR improves the accuracy of SOC estimation. The optimal SVR model based on the feature selection criterion shows better generalisation. Better estimation results in four driving conditions are achieved, and the root-mean-square error of the battery SOC estimation is decreased by at least 64.1 % and 56.5 % compared to the EKF algorithm and SVR algorithm driven by full feature, respectively.

Icing Time Prediction Model of Pavement Based on an Improved SVR Model with Response Surface Approach

Pavement icing imposes a great threat to driving safety and impacts the efficiency of the road transportation system in cold regions. This has attracted research predicting pavement icing time to solve the problems brought about by icing. Different models have been proposed in the past decades to predict pavement icing, within which support vector regression (SVR) is a widely used algorithm for calibrating highly nonlinear relationships. This paper presents a hybrid improved SVR algorithm to predict the time of pavement icing with an enhancement operation by response surface method (RSM) and particle swarm optimization (PSO). RSM is used to increase the number of input data collected onsite. Based on that, the optimal SVR model is established by optimizing the kernel function parameters and penalty coefficient with the particle swarm optimization (PSO) algorithm. The hybrid improved SVR is compared with SVR, PSO-SVR, and RSM-PSO for coefficient of determination (R2), mean absolute error, mean absolute percentage error, and root mean square error to check the effectiveness of PSO and RSM in optimizing SVR. The results show that the combination of two methods in the hybrid improved algorithm has a better optimization capability with R2 of 0.9655 and 0.9318 in a train set and test set, respectively, which outperforms PSO-SVR, RSM-SVR, and SVR. In addition, the R2 of the hybrid improved SVR and PSO-SVR both reach the optimal fitness value approximately at the iteration of 20, which suggests that convergence capacity remains relatively constant with the predictive accuracy being improved.

An optimized support vector regression for prediction of bearing degradation

Machine learning and deep learning models are gradually being applied to predict the remaining useful life of bearings. But, still extracting effective features from the bearing signals and enhancing the prediction accuracy is a major concern. Therefore, an optimized support vector regression (SVR) model for prediction of bearing degradation is proposed. Firstly, the original time domain and frequency domain features extracted from the bearing vibration signal are learned by using the deep neural network (DNN) to improve the quality of degradation features. Secondly, a novel multi-population fruit fly optimization algorithm (MPFOA) is proposed by introducing multi-population mechanism. Thirdly, MPFOA is employed to choose the parameters of SVR, then we use MPFOA-SVR to predict the bearing remaining useful life through the degradation features learned by DNN. At last, CEC 2017 unconstrained benchmark functions and a real bearing dataset (IEEE 2012 PHM) are used to verify the performance of MPFOA and optimized SVR models respectively. Numerical experimental results show that MPFOA has a better optimization ability than the compared meta-heuristic algorithms. The optimized SVR model has a higher prediction accuracy in predicting the bearing remaining useful life.

Prediction of House Price Index Based on Bagging Integrated WOA-SVR Model

Aiming at the shortcomings of a single machine learning model with low model prediction accuracy and insufficient generalization ability in house price index prediction, a whale algorithm optimized support vector regression model based on bagging ensemble learning method is proposed. Firstly, gray correlation analysis is used to obtain the main influencing factors of house prices, and the segmentation forecasting method is used to divide the data set and forecast the house prices in the coming year using the data of the past ten years. Secondly, the whale optimization algorithm is used to find the optimal parameters of the penalty factor and kernel function in the SVR model, and then, the WOA-SVR model is established. Finally, in order to further improve the model generalization capability, a bagging integration strategy is used to further integrate and optimize the WOA-SVR model. The experiments are conducted to forecast the house price indices of four regions, Beijing, Shanghai, Tianjin, and Chongqing, respectively, and the results show that the prediction accuracy of the proposed integrated model is better than the comparison model in all cases.

Intelligently Predict the Rock Joint Shear Strength Using the Support Vector Regression and Firefly Algorithm

AbstractTo propose an effective and reasonable excavation plan for rock joints to control the overall stability of the surrounding rock mass and predict and prevent engineering disasters, this study is aimed at predicting the rock joint shear strength using the combined algorithm by the support vector regression (SVR) and firefly algorithm (FA). The dataset of rock joint shear strength collected was employed as the output of the prediction, using the joint roughness coefficient (JRC), uniaxial compressive strength (σc), normal stress (σn), and basic friction angle (φb) as the input for the machine learning. Based on the database of rock joint shear strength, the training subset and test subset for machine learning processes are developed to realize the prediction and evaluation processes. The results showed that the FA algorithm can adjust the hyperparameters effectively and accurately, obtaining the optimized SVR model to complete the prediction of rock joint shear strength. For the testing results, the developed model was able to obtain values of 0.9825 and 0.2334 for the coefficient of determination and root-mean-square error, showing the good applicability of the SVR-FA model to establish the nonlinear relationship between the input variables and the rock joint shear strength. Results of the importance scores showed that σn is the most important factor that affects the rock joint shear strength while σc has the least significant effect. As a factor influencing the shear stiffness from the perspective of physical appearance, the change of the JRC value has a significant impact on the rock joint shear strength.

A support vector regression model to predict nitrate-nitrogen isotopic composition using hydro-chemical variables

Nitrate is a prominent pollutant in surface and groundwater bodies worldwide. Isotopes in nitrate provide a powerful approach for tracing nitrate sources and transformations in waters. Given that analytical techniques for determining isotopic compositions are generally time-consuming, laborious and expensive, alternative methods are warranted to supplement and enhance existing approaches. Hence, we developed a support vector regression (SVR) model and explored its feasibility to predict nitrogen isotopic composition of nitrate (δ15N–NO3−) in a rural-urban river system in Southeastern China. A total of 16 easily obtained hydro-chemical variables were measured in the wet season (September 2019) and dry season (January 2020) and used to develop the SVR prediction model. The grading method utilized ~75% (35) of the samples for model building while the remaining 11 samples assessed model performance. Principal component analysis (PCA) extracted 7 principal components for SVR model inputs as PCA reduces superfluous variables. We optimized tuning parameters in the SVR model using a grid search technique coupled with V-fold cross-validation. The optimized SVR model provided accurate δ15N–NO3− predictions with a determination coefficient (R2) of 0.88, Nash-Sutcliffe (NS) of 0.87, and mean square error (MSE) of 0.53‰ in the testing step, and performed much better than the corresponding multivariate linear regression model (R2 = 0.60, NS = 0.58 and MSE = 1.76‰) and general regression neural network model (R2 = 0.66, NS = 0.65 and MSE = 1.45‰). Overall, the SVR model provides a potential indirect method to predict environmental isotope values for water quality management that will complement and enhance the interpretation of direct measurements of δ15N–NO3−.

Comparative performance analysis of support vector regression and artificial neural network for prediction of municipal solid waste generation.

The evolution of machine learning (ML) algorithms provides researchers and engineers with state-of-the-art tools to dynamically model complex relationships. The design and operation of municipal solid waste (MSW) management systems require accurate estimation of generation rates. In this study, we applied rapid, non-linear and non-parametric data driven ML algorithms independently, multi-layer perceptron artificial neural network (MLP-ANN) and support vector regression (SVR) models to predict annual MSW generation rates in Bahrain. Models were trained and tested with MSW generation data for period of 1997-2019. The population, gross domestic product, annual tourist numbers, annual electricity consumption and total annual CO2 emissions were selected as explanatory variables and incorporated into developed models. The zero score normalization (ZSN) and minimum maximum normalization (MMN) methods were utilized to improve the quality of data and subsequently enhances the performance of ML algorithms. Statistical metrics were employed to discriminate performance of MLP-ANN and SVR models. The linear, polynomial, radial basis function (RBF) and sigmoid kernel functions were investigated to find the optimal SVR model. Results showed that RBF-SVR model with R2 value of 0.97% and 4.82% and absolute forecasting error (AFE) for the period of 2008 and 2019 exhibits superior prediction and robustness in comparison to MLP-ANN. The efficacy of MLP-ANN model was also reasonably successful with R2 value of 0.94. It was shown that MMN pre-processing generated optimal MLP-ANN model while ZSN pre-processing produced optimal RBF-SVR model. This work also highlights the importance of application of ML modelling approaches to plan and implement their roadmap for waste management systems by policymakers.

Flood spatial prediction modeling using a hybrid of meta-optimization and support vector regression modeling

Flood spatial susceptibility prediction is the first essential step in developing flood mitigation strategies and reducing flood damage. Flood occurrence is a complex process that is not easily predicted through simple methods. This study describes optimization of support vector regression (SVR) using meta-optimization algorithms including the grasshopper optimization algorithm (GOA) and particle swarm optimization (PSO) for flood modeling at Qazvin Plain, Iran. Geospatial data including nine readily available geo-environmental flood conditioning factors (i.e., ground slope, aspect, elevation, planform curvature, profile curvature, proximity to a river, land use, lithology and rainfall) were derived. The information gain ratio (IGR) method was used to determine the relative importance of input variables. A historical flood inventory map for 43 locations was created from existing reports. The geospatial data and historical flood levels were used to construct the training and testing datasets. Then, the training dataset was used to generate flood-susceptibility maps using the optimized SVR model with the GOA and PSO algorithms. Finally, the predictive accuracy of the models was quantified using the statistical measures of root mean square error (RMSE), mean absolute error (MAE), and area under the receiver operating characteristic (ROC) curve (AUC). Although both the GOA and PSO algorithms improved SVR performance, the SVR-GOA model performed best (AUC = 0.959, RMSE = 0.31 and MSE = 0.098), followed by the SVR-PSO model (AUC = 0.959, RMSE = 0.33 and MSE = 0.11) and standalone SVR model (AUC = 0.87, RMSE = 0.35 and MSE = 0.12). Elevation, lithology and aspect had the highest IGR values and were identified as the most effective predictors of flood susceptibility.

Prediction of Formation Quality of Inconel 625 Clads Using Support Vector Regression

The process parameters of pulsed tungsten inert gas (PTIG) have a significant influence on the formation quality, mechanical properties and corrosion resistance of the weld overlay. The PTIG was utilized to deposit Inconel 625 clads with various combinations of the process parameters, which were determined by the central composite design (CCD) method. Based on the experimental results, the relationship between process parameters of PTIG and formation quality of the Inconel 625 clads was established using support vector regression (SVR) with different kernel functions, including polynomial kernel function, radial basis function (RBF) kernel function, and sigmoid kernel function. The results indicate that the kernel functions have a great influence on the prediction of height, width and dilution. The models with RBF kernel function feature the best goodness of fitting and the most accurate against the other SVR models for estimating the height and the dilution. However, the model with polynomial kernel function is superior to the other SVR models for predicting the width. Meanwhile, the prediction performance of the SVR models was compared with the general regression analysis. The results demonstrate that the optimized SVR model is much better than the general regression model in the prediction performance.

Support Vector Regression for the Relationships between Ground Motion Parameters and Macroseismic Intensity in the Sichuan–Yunnan Region

In this paper, a nonlinear regression method called a support vector regression (SVR) is presented to establish the relationship between engineering ground motion parameters and macroseismic intensity (MSI). Sixteen ground motion parameters, including peak ground acceleration (PGA), peak ground velocity (PGV), Arias intensity, Housner intensity, acceleration spectrum intensity, velocity spectrum intensity, and others, are considered as candidates for feature selection to generate optimal SVR models. The datasets with both useable strong ground motion records and corresponding investigated MSIs in the Sichuan–Yunnan region, China, are all collected, and these 125 pairs of datasets are used for selecting features and comparing regression results. Nine ground motion parameters are selected as the most relevant features: PGA is the first fundamental one and PGV is the fifth relevant feature. Based on performance measures on the testing dataset, the best SVR model is given when the number of features is one all the way up to nine. According to predicted accuracy, SVR models with Gaussian kernel give much better MSI prediction than linear kernel SVR models and linear regression models. In particular, the Gaussian kernel SVR of PGA gives much higher MSI prediction accuracy than the linear regression model of PGV and PGA. The proposed SVR models are valid for MSI values from VI to IX, and they can be used for rapid mapping damage potential and reporting seismic intensity for this high-seismic-activity region.

Porosity estimation of abradable seal coating with an optimized support vector regression model based on multi-scale ultrasonic attenuation coefficient

Accurate porosity characterization is of great significance to evaluate abradable seal coating comprehensive performance. In this paper, a support vector regression model optimized by particle swarm optimization algorithm, termed PSO-SVR, is proposed to predict coating porosity based on multi-scale ultrasonic attenuation coefficient. To decouple the ‘multi-scale scattering effect’ of ultrasonic propagation in abradable seal coating, the echo signals are decomposed using Continuous Wavelet Transform (CWT). The ultrasonic responses in different frequency bands could be sufficiently extracted through the multi-scale ultrasonic attenuation coefficient obtained by CWT. Subsequently, taking the coefficients as input vectors, the SVR model is established. The parameters of SVR, including the penalty factor C, kernel function parameter γ, and insensitive loss ε, are optimized through PSO algorithm. Finally, the optimized SVR model is applied to predict the porosity of the AlSi-polyester abradable seal coating prepared by plasma spraying. The normalized mean squared error MSE of the validation set is 0.067 with a determination coefficient R2 of 0.947. The prediction results show that the PSO-SVR model has higher accuracy, better generalization ability, and stronger robustness in the case with limited experimental data, compared with classical Artificial Neural Network (ANN) models, e.g. Back Propagation (BP), Radial Basis Function (RBF), and General Regression Neural Network (GRNN).

Assessment of agricultural energy consumption of Turkey by MLR and Bayesian optimized SVR and GPR models

AbstractAgricultural productivity highly depends on the cost of energy required for cultivation. Thus prior knowledge of energy consumption is an important step for energy planning and policy development in agriculture. The aim of the present study is to evaluate the application potential of multiple linear regression (MLR) and machine learning tools such as support vector regression (SVR) and Gaussian process regression (GPR) to forecast the agricultural energy consumption of Turkey. In the development of the models, widespread indicators such as agricultural value‐added, total arable land, gross domestic product share of agriculture, and population data were used as input parameters. Twenty‐eight‐year historical data from 1990 to 2017 were utilized for the training and testing stages of the models. A Bayesian optimization method was applied to improve the prediction capability of SVR and GPR models. The performance of the models was measured by various statistical tools. The results indicated that the Bayesian optimized GPR (BGPR) model with exponential kernel function showed a superior prediction capability over MLR and Bayesian optimized SVR model. The root mean square error, mean absolute deviation, mean absolute percentage error, and coefficient of determination (R2) values for the BGPR model were determined as 0.0022, 0.0005, 0.2041, and 0.9999 in the training phase and 0.0452, 0.0310, 7.7152, and 0.9677 in the testing phase, respectively. As a result, it can be concluded that the proposed BGPR model is an efficient technique and has the potential to predict agricultural energy consumption with high accuracy.

Prediction of specific cutting forces and maximum tool temperatures in orthogonal machining by Support Vector and Gaussian Process Regression Methods

In this paper, machine learning (ML) models, namely Support Vector Regression (SVR) and Gaussian Process Regression (GPR), are presented for the prediction of specific cutting forces and maximum tool temperatures during orthogonal machining processes. The training/test data for building the ML models is generated from finite element (FE) simulations. The simulations are performed using the commercial FE package Abaqus/Explicit and validated using experimental results. The FE generated data consists of cutting speed, uncut chip thickness, and rake angle as the input parameters. The response variables are the cutting force and maximum tool temperature. The data is split into training and test sets using 80-20 split. The optimal SVR and GPR models are selected using grid search on the training data. The predictions on the test data sets show that both the models perform well with high accuracy in predicting cutting force and maximum tool temperature. Between the two models, the mean square errors (MSE) for SVR are less than those for GPR.

IGBT junction temperature prediction method based on improved artificial bee colony algorithm for optimizing SVR

Aiming at the low precision of IGBT thermal parameter junction temperature prediction method and the need to extract multiple parameters and being vulnerable to load, the improved artificial bee colony algorithm optimized support vector regression machine (ABC- SVR) is used to predict the IGBT junction temperature. Firstly, the formula for updating the honey source in the artificial bee colony algorithm (ABC) is improved, and the fitness probability of Follower bee select honey source is introduced to construct the weight function; Then the problem of support vector regression (SVR) parameter selection is transformed into the parameter combination optimization problem, then establish an optimal SVR model; Finally, the data from the IGBT accelerated aging test provided by the National Aeronautics and Space Administration (NASA) is taken as a sample, the predicted results of the improved ABC-SVR model and the common ABC-SVR model were compared and analyzed. Through the simulation results, the prediction effect of improved ABC-SVR model is better than the common model, and the running time is greatly reduced, it has greater precision in junction temperature prediction.