Least Squares Support Vector Regression Model Research Articles

Whole flow field performance prediction by impeller parameters of centrifugal pumps using support vector regression

The relationship of multiple impeller parameters and performance indices is difficult to describe because of some unknown hydrodynamic phenomena. Modeling of performance indices of the whole flow field from impeller parameters often encounters some challenges, especially lower prediction accuracy in relatively small and large flow points, dependence on designers’ experience and time-consuming designing process. In this work, the least squares support vector regression (LSSVR) method is proposed to predict multiple pump performance indices of the whole flow field. To describe the performance more completely, the powder, the head, and the efficiency indices are chosen as the model outputs. Additionally, to improve the prediction accuracy and reduce the manufacture difficulty, nine impeller parameters and the flow rate are selected as the model inputs. With the LSSVR model, the complex nonlinearity relationship between multiple impeller parameters and performance indices can be described approximately. Moreover, the LSSVR model and the computational fluid dynamics numerical simulation model are applied to predict the powder, the head, and the efficiency of an actual centrifugal mine pump in the whole flow field. Compared with the performance test results, the superiority of the proposed method is demonstrated in terms of more accurate prediction performance and faster designing process.

Pan evaporation modeling using four different heuristic approaches

Evaporation plays important roles in regional water resources management, climate change and agricultural production. This study investigates the abilities of fuzzy genetic (FG), least square support vector regression (LSSVR), multivariate adaptive regression spline (MARS), M5 model tree (M5Tree) and multiple linear regression (MLR) in estimating daily pan evaporation (Ep). Daily climatic data, air temperature (Ta), surface temperature (Ts), wind speed (Ws), relative humidity (RH) and sunshine hours (Hs) at eight stations in the Dongting Lake Basin, China are used for model development and validation. The first part of this study focuses on testing the model accuracies at each station using local input and output data. The results show that LSSVR and FG models with more input variables perform better than the MARS, M5Tree and MLR models in predicting daily Ep at most stations with respect to mean absolute errors (MAE), root mean square errors (RMSE) and determination coefficient (R2). In the second part of this study, the models are tested using cross-validation method in two different applications. The daily Ep of Yueyang station is estimated using the input and output data of Jingzhou and Changsha, respectively. Comparisons of the models indicate that the FG, LSSVR and MARS models outperform the M5Tree model, Ts, Hs and Ta are major influencing factors and adding Ws or RH into model inputs significantly improve the model performances. The overall results indicate that above models can be successfully used for estimating daily Ep using local input and output data while the FG and LSSVR generally perform better than the other models without local input and outputs.

Novel sparse LSSVR models in primal weight space for robust system identification with outliers

Recent demands from big data applications have strongly motivated a successful sparse formulation of the least squares support vector regression (LSSVR) model in primal weight space. Such an approach, which has been called fixed-size LSSVR (FS-LSSVR), is built upon an approximation of the nonlinear feature mapping based on Nyström method to overcome the usual memory constraints and high computational costs of the standard non-sparse LSSVR model. Despite this advance, an important modeling issue remains unaddressed by the FS-LSSVR model. As in the standard LSSVR model, the performance of FS-LSSVR model is greatly degraded when estimation data is corrupted with non-Gaussian noise or outliers. Bearing this major issue in mind, we introduce two robust variants of the FS-LSSVR model based on M-estimation framework and the weighted least squares method. The proposed approaches, henceforth called Robust FS-LSSVR (RFS-LSSVR) and Reweighted Robust FS-LSSVR (R2FS-LSSVR) models, produce solutions that are simultaneously robust to outliers and sparse, making use of only a small sample of training patterns as prototype vectors. We evaluate the performances of both algorithms in benchmarking nonlinear system identification problems with synthetic and real-world datasets (including a large scale dataset) corresponding to SISO and MIMO systems, whose estimation outputs are contaminated with outliers. The obtained results indicate that our proposed approaches consistently outperforms existing robust models designed in dual space (e.g. W-LSSVR and IR-LSSVR models), specially as the amount of outliers in the data increases.

Modelling of runoff and sediment yield using ANN, LS-SVR, REPTree and M5 models

AbstractIn this study, the performance evaluation of five machine learning models, namely, ANNLM, ANNSCG, least square-support vector regression (LS-SVR), reduced error pruning tree (REPTree) and M5, was carried out for predicting runoff and sediment in the Pokhariya watershed, India using hydro-meteorological variables as input. The input variables were selected using the trial-and-error procedure which represents the hydrological process in the watershed. The seven input variables to all the models comprised a combination of rainfall, average temperature, relative humidity, pan evaporation, sunshine duration, solar radiation and wind speed. The monthly runoff and sediment yield data were used to calibrate and validate all models for the years 2000 to 2008. Evaluation of models' performances were carried out using four statistical indices, i.e., Nash–Sutcliffe coefficient (NSE), coefficient of determination (R2), percent bias (PBIAS) and RMSE-observations standard deviation ratio (RSR). Comparative analysis showed that the ANNLM model marginally outperformed the LS-SVR model and all the other models investigated during calibration and validation for runoff modelling whereas the LS-SVR model surpassed the artificial neural networks (ANN) model and other models for sediment yield modelling. Moreover, M5 model tree is better in simulating sediment yield and runoff than its near counterpart, the REPTree model, and marginally inferior when compared to LS-SVR and ANN models.

An enhanced chiller FDD strategy based on the combination of the LSSVR-DE model and EWMA control charts

Development of the fault detection and diagnosis (FDD) for chiller systems is very important for improving the equipment reliability and saving energy consumption. The results of FDD performance are strongly dependent on the accuracy of chiller models. Since the accuracy of the chiller models depends on the indefinite model parameters which are normally chosen by experiments or experiences, an accurate chiller model is difficult to build. Therefore, optimization of model parameters is very useful to increase the accuracy of chiller models. This paper presents a new FDD strategy for centrifugal chillers of building air-conditioning systems, which is the combination between the nonlinear least squares support vector regression (LSSVR) based on the differential evolution (DE) algorithm and the exponentially weighted moving average (EWMA) control charts. In this strategy, the nonlinear LSSVR, which is a reformulation of SVR model with better generalization performances, is adopted to develop the reference feature parameter models in a typical non-linear chiller system. The DE algorithm which is a real-coding optimal algorithm with powerful global searching capacity is employed to enhance the accuracy of LSSVR models. The exponentially weighted moving average (EWMA) control charts are introduced to improve the fault detection capability as well as to reduce the Type II errors in a t-statistics-based way. Six typical faults of the chiller from the real-time experimental data of ASHRAE RP-1043 project are chosen to validate proposed FDD methods. Comprehensive comparisons between the proposed method and two similarly previous studies are performed. The comparison results show that the proposed method has achieved significant improvement in accuracy and reliability, especially at low severity levels. The proposed DE-LSSVR-EWMA strategy is robust for fault detection and diagnosis in centrifugal chiller systems.

Enhancing Long-Term Streamflow Forecasting and Predicting using Periodicity Data Component: Application of Artificial Intelligence

Streamflow forecasting and predicting are significant concern for several applications of water resources and management including flood management, determination of river water potentials, environmental flow analysis, and agriculture and hydro-power generation. Forecasting and predicting of monthly streamflows are investigated by using three heuristic regression techniques, least square support vector regression (LSSVR), multivariate adaptive regression splines (MARS) and M5 Model Tree (M5-Tree). Data from four different stations, Besiri and Malabadi located in Turkey, Hit and Baghdad located in Iraq, are used in the analysis. Cross validation method is employed in the applications. In the first stage of the study, the heuristic regression models are compared with each other and multiple linear regression (MLR) in forecasting one month ahead streamflow of each station, individually. In the second stage, the models are evaluated and compared in predicting streamflow of one station using data of nearby station. The research investigated also the influence of the periodicity component (month number of the year) as an external sub-set in modeling long-term streamflow. In both stages, the comparison results indicate that the LSSVR model generally performs superior to the MARS, M5-Tree and MLR models. In addition, it is seen that adding periodicity as input to the models significantly increase their accuracy in forecasting and predicting monthly streamflows in both stages of the study.

Modeling reference evapotranspiration using three different heuristic regression approaches

Modeling reference evapotranspiration (ET0) is important in reservoir management, planning regional water resources and evaluation of drinking-water supplies. The study investigates the ability of three different heuristic regression approaches, least square support vector regression (LSSVR), multivariate adaptive regression splines (MARS) and M5 Model Tree (M5Tree) in modeling ET0. The first part of the study focused on testing the accuracy of the LSSVR, MARS and M5Tree models in estimating the ET0 data of Antalya and Isparta stations located in Mediterranean Region of Turkey. Cross-validation method was utilized in the applications. The LSSVR models were observed to be better than the MARS and M5Tree models in estimating ET0 of Antalya and Isparta stations with local input and output data. The accuracy of the applied methods was investigated in estimation of ET0 using air temperature, solar radiation, relative humidity and wind speed inputs from nearby station in the second part of the study (cross-station application without local input data). The results showed that the MARS models provided better accuracy than the LSSVR and M5Tree models with respect to SI, mean absolute error (MAE) and determination coefficient (R2). In the third part of the study, the accuracy of the applied models was investigated in ET0 estimation using input and output data from nearby station. The results showed that the M5Tree models outperformed the other models with respect to SI, MAE and R2. The overall results showed that the LSSVR could be successfully used in estimating ET0 by using local input and output data. In case of without local inputs, however, the MARS model performed better than the LSSVR and M5Tree models while the M5Tree was observed to be the best alternative for estimating ET0 in the absence of local input and output data.

On conditional risk estimation considering model risk

Usually, parametric procedures used for conditional variance modelling are associated with model risk. Model risk may affect the volatility and conditional value at risk estimation process either due to estimation or misspecification risks. Hence, non-parametric artificial intelligence models can be considered as alternative models given that they do not rely on an explicit form of the volatility. In this paper, we consider the least-squares support vector regression (LS-SVR), weighted LS-SVR and Fixed size LS-SVR models in order to handle the problem of conditional risk estimation taking into account issues of model risk. A simulation study and a real application show the performance of proposed volatility and VaR models.

Object tracking based on Kalman particle filter with LSSVR

In visual tracking field, traditional Kalman particle filter often suffers from the accuracy loss when estimating the target. To alleviate this problem, we propose a novel object tracking method with the fusion of the extended Kalman particle filter (EKPF) and the least squares support vector regression (LSSVR). First, the observation value of Kalman filter is acquired with the cues of color and motion features. The importance probability density function is generated by extended Kalman filter (EKF), which makes the distribution of particles approximately to the posterior probability. And then, a weighted plan is used to determine the weighted coefficient of LSSVR model, the robustness and sparseness of LSSVR modeling will thereby be enhanced. The updated EKF features of tested samples served as training samples to establish the dynamic LSSVR model real-time in the next frame. Finally, the LSSVR is used to calibrate the tracking results of Kalman particle filter, such that the tracking object will always follow the correct motion trajectory. The experimental results show that our method performs favorably against traditional Kalman particle filter with real-time performance and strong robustness.

Application of least squares support vector regression and linear multiple regression for modeling removal of methyl orange onto tin oxide nanoparticles loaded on activated carbon and activated carbon prepared from Pistacia atlantica wood

Two novel and eco friendly adsorbents namely tin oxide nanoparticles loaded on activated carbon (SnO2-NP-AC) and activated carbon prepared from wood tree Pistacia atlantica (AC-PAW) were used for the rapid removal and fast adsorption of methyl orange (MO) from the aqueous phase. The dependency of MO removal with various adsorption influential parameters was well modeled and optimized using multiple linear regressions (MLR) and least squares support vector regression (LSSVR). The optimal parameters for the LSSVR model were found based on γ value of 0.76 and σ2 of 0.15. For testing the data set, the mean square error (MSE) values of 0.0010 and the coefficient of determination (R2) values of 0.976 were obtained for LSSVR model, and the MSE value of 0.0037 and the R2 value of 0.897 were obtained for the MLR model. The adsorption equilibrium and kinetic data was found to be well fitted and in good agreement with Langmuir isotherm model and second-order equation and intra-particle diffusion models respectively. The small amount of the proposed SnO2-NP-AC and AC-PAW (0.015g and 0.08g) is applicable for successful rapid removal of methyl orange (>95%). The maximum adsorption capacity for SnO2-NP-AC and AC-PAW was 250mgg−1 and 125mgg−1 respectively.

Solar power output forecasting using evolutionary seasonal decomposition least-square support vector regression

Renewable power output is an important factor in scheduling and for improving balanced area control performance. This investigation develops an evolutionary seasonal decomposition least-square support vector regression (ESDLS-SVR) to forecast monthly solar power output. The construction of the ESDLS-SVR uses seasonal decomposition and least-square support vector regression (LS-SVR). Genetic algorithms (GA) are used simultaneously to select the parameters of the LS-SVR. Monthly solar power output data from Taiwan Power Company are used. Empirical results indicate that the proposed forecasting system demonstrates a superior performance in terms of forecasting accuracy. A comparative study has been introduced showing that the ESDLS-SVR model performance is better than autoregressive integrated moving average (ARIMA), seasonal autoregressive integrated moving average (SARIMA), generalized regression neural network (GRNN) and LS-SVR models.

Prediction of Pollutant Emissions of Biomass Flames Through Digital Imaging, Contourlet Transform, and Support Vector Regression Modeling

This paper presents a method for the prediction of NO x emissions in a biomass combustion process through the combination of flame radical imaging, contourlet transform and Zernike moment (CTZM), and least squares support vector regression (LS-SVR) modeling. A novel feature extraction technique based on the CTZM algorithm is developed. The contourlet transform provides the multiscale decomposition for flame radical images and the selected operator based on Zernike moments is designed to provide the well-defined structure for the images. The resulted image features are a variable structure, which is originated from the CTZM. Finally, the variable features of the images of four flame radicals (OH*, CN*, CH*, and $\text{C}^{\ast }_{2}$ ) are defined. The relationship between the variable features of radical images and NO x emissions is established through radial basis function network modeling, SVR modeling, and the LS-SVR modeling. A comparison between the three modeling approaches shows that the LS-SVR model outperforms the other two methods in terms of root-mean-square error and mean relative error criteria. In addition, the structure of the image features has a significant impact on the performance of the prediction models. The test results obtained on a biomass-gas fired test rig show the effectiveness of the proposed technical approach for the prediction of NO x emissions.

Streamflow Forecasting and Estimation Using Least Square Support Vector Regression and Adaptive Neuro-Fuzzy Embedded Fuzzy c-means Clustering

This paper investigates the ability of least square support vector regression (LSSVR) and adaptive neuro-fuzzy embedded fuzzy c-means clustering (ANFIS-FCM) in forecasting and estimation of monthly streamflows. In the first part of the study, the LSSVR and ANFIS-FCM models were tested in 1-month ahead streamflow forecasting by using cross-validation method. Monthly streamflow data belonging to two stations, Besiri Station on Garzan Stream and Baykan Station on Bitlis Stream, in Dicle Basin of Turkey were used. The LSSVR and ANFIS-FCM results were compared with autoregressive moving average (ARMA) models. It was found that the LSSVR models performed better than the ANFIS-FCM and ARMA models in 1-month ahead streamflow forecasting. The ANFIS-FCM models are also found to be better than the ARMA models. The effect of periodicity on forecasting performance of the LSSVR models was also investigated. Adding periodicity component as input to the LSSVR models significantly improved the models’ accuracy in forecasting. In the second part of the study, the accuracy of the LSSVR and ANFIS-FCM models was tested in streamflow estimation using data from nearby stream. Based on the results, the LSSVR was found to be better than the ANFIS-FCM and successfully used in estimating monthly streamflows by using nearby station data.

A hybrid grid-GA-based LSSVR learning paradigm for crude oil price forecasting

In order to effectively model crude oil spot price with inherently high complexity, a hybrid learning paradigm integrating least squares support vector regression (LSSVR) with a hybrid optimization searching approach for the parameters selection in the LSSVR [consisting of grid method and genetic algorithm (GA)], i.e., a hybrid grid-GA-based LSSVR model, is proposed in this study. In the proposed hybrid learning paradigm, the grid method, a simple but efficient searching method, is first applied to roughly but rapidly determine the proper boundaries of the parameters in the LSSVR; then, the GA, an effective and powerful intelligent searching algorithm, is further implemented to select the most suitable parameters. For illustration and verification, the proposed learning paradigm is used to predict the crude oil spot prices of the West Texas Intermediate and the Brent markets. The empirical results demonstrate that the proposed hybrid grid-GA-based LSSVR learning paradigm can outperform its benchmarking models (including some popular forecasting techniques and similar LSSVRs with other parameter searching algorithms) in terms of both prediction accuracy and time-savings, indicating that it can be utilized as one effective forecasting tool for crude oil price with high volatility and irregularity.

A novel hybrid FA-Based LSSVR learning paradigm for hydropower consumption forecasting

Due to the nonlinearity and nonstationary of hydropower market data, a novel hybrid learning paradigm is proposed to predict hydropower consumption, by incorporating firefly algorithm (FA) into least square support vector regression (LSSVR), i.e., FA-based LSSVR model. In the novel model, the powerful and effective artificial intelligence (AI) technique, i.e., LSSVR, is employed to forecast hydropower consumption. Furthermore, a promising AI optimization tool, i.e., FA, is especially introduced to address the crucial but difficult task of parameters determination in LSSVR (e.g., hyper and kernel function parameters). With the Chinese hydropower consumption as sample data, the empirical study has statistically confirmed the superiority of the novel FA-based LSSVR model to other benchmark models (including existing popular traditional econometric models, AI models and similar hybrid LSSVRs with other popular parameter searching tools), in terms of level and directional accuracy. The empirical results also imply that the hybrid FA-based LSSVR learning paradigm with powerful forecasting tool and parameters optimization method can be employed as an effective forecasting tool for not only hydropower consumption but also other complex data.

Evaluation of the value of near infrared (NIR) spectromicroscopy for the analysis of glycyrrizhic acid in licorice

It has been reported that hyperspectral data could be employed to qualitatively elucidate the spatial composition of tablets of Chinese medicinal plants. To gain more insights into this technology, a quantitative profile provided by near infrared (NIR) spectromicroscopy was further studied by determining the glycyrrhizic acid content in licorice, Glycyrrhiza uralensis. Thirty-nine samples from twenty-four different origins were analyzed using NIR spectromicroscopy. Partial least squares, interval partial least square (iPLS), and least squares support vector regression (LS-SVR) methods were used to develop linear and non-linear calibration models, with optimal calibration parameters (number of interval numbers, kernel parameter, etc.) being explored. The root mean square error of prediction (RMSEP) and the coefficient of determination (R2) of the iPLS model were 0.717 7% and 0.936 1 in the prediction set, respectively. The RMSEP and R2 of LS-SVR model were 0.515 5% and 0.951 4 in the prediction set, respectively. These results demonstrated that the glycyrrhizic acid content in licorice could barely be analyzed by NIR spectromicroscopy, suggesting that good quality quantitative data are difficult to obtain from microscopic NIR spectra for complicated Chinese medicinal plant materials.

Shear Strength Prediction in Reinforced Concrete Deep Beams Using Nature-Inspired Metaheuristic Support Vector Regression

AbstractThe shear strength of reinforced concrete (RC) deep beams is a dynamic phenomenon that varies with many mechanical and geometrical factors. Accurately estimating shear strength in RC deep beams is a vital issue in engineering design and management. However, prediction accuracy is still poor. This study presents a nature-inspired metaheuristic regression method for accurately predicting shear strength in RC deep beams that combines a novel smart artificial firefly colony algorithm (SFA) and least squares support vector regression (LS-SVR). The SFA integrates the firefly algorithm (FA), chaotic map (CM), adaptive inertia weight (AIW), and Levy flight (LF). First, an adaptive approach and randomization methods (i.e., CM, AIW, and LF) were incorporated in FA to construct an effective metaheuristic algorithm for global optimization. The SFA was then used to optimize the hyperparameters of the LS-SVR model. The proposed model was constructed using a data set for RC deep beams which was derived from the ...

ESTIMATION OF COPPER CONCENTRATE GRADE BASED ON COLOR FEATURES AND LEAST-SQUARES SUPPORT VECTOR REGRESSION

In this paper, a new method based on color features of microscopic image and least-squares support vector regression model (LS-SVR) is proposed for indirect measurement of copper concentrate grade. Red, green and blue (RGB), hue and color vector angle were extracted from color microscopic images of a copper concentrate sample and selected for the comparison. Three different estimation models based on LS-SVR were developed using RGB, hue, and color vector angle, respectively. A comparison of three models was carried out through a validation test. The best model was obtained for the hue giving a running time of 30.243 ms, root mean square error of 0.8644 and correlation coefficient value of 0.9997. The results indicated that the copper concentrate grade could be estimated by the LS- SVR model using the hue as input parameter with a satisfactory accuracy.

Online monitoring and control of particle size in the grinding process using least square support vector regression and resilient back propagation neural network

Particle size soft sensing in cement mills will be largely helpful in maintaining desired cement fineness or Blaine. Despite the growing use of vertical roller mills (VRM) for clinker grinding, very few research work is available on VRM modeling. This article reports the design of three types of feed forward neural network models and least square support vector regression (LS-SVR) model of a VRM for online monitoring of cement fineness based on mill data collected from a cement plant. In the data pre-processing step, a comparative study of the various outlier detection algorithms has been performed. Subsequently, for model development, the advantage of algorithm based data splitting over random selection is presented. The training data set obtained by use of Kennard–Stone maximal intra distance criterion (CADEX algorithm) was used for development of LS-SVR, back propagation neural network, radial basis function neural network and generalized regression neural network models. Simulation results show that resilient back propagation model performs better than RBF network, regression network and LS-SVR model. Model implementation has been done in SIMULINK platform showing the online detection of abnormal data and real time estimation of cement Blaine from the knowledge of the input variables. Finally, closed loop study shows how the model can be effectively utilized for maintaining cement fineness at desired value.

Rapid vision-based system for secondary copper content estimation

A vision-based color analysis system was developed for rapid estimation of copper content in the secondary copper smelting process. Firstly, cross section images of secondary copper samples were captured by the designed vision system. After the preprocessing and segmenting procedures, the images were selected according to their grayscale standard deviations of pixels and percentages of edge pixels in the luminance component. The selected images were then used to extract the information of the improved color vector angles, from which the copper content estimation model was developed based on the least squares support vector regression (LSSVR) method. For comparison, three additional LSSVR models, namely, only with sample selection, only with improved color vector angle, without sample selection or improved color vector angle, were developed. In addition, two exponential models, namely, with sample selection, without sample selection, were developed. Experimental results indicate that the proposed method is more effective for improving the copper content estimation accuracy, particularly when the sample size is small.