Extreme Support Vector Regression Research Articles

Investigating the potential of EMA-embedded feature selection method for ESVR and LSTM to enhance the robustness of monthly streamflow forecasting from local meteorological information

Accurate forecast of monthly streamflow is helpful to improve the social capability in risk management. Diverse input features are crucial to the accuracy of machine learning-based monthly streamflow forecast models. The embedded feature selection (EFS) method based on binary-coded metaheuristic algorithm is adept at parallel optimization of feature selection and model hyperparameter. It has been rarely studied coupled with deep learning models in monthly streamflow forecasting. This study further investigated the potential of EFS method using the well-known Long short-term memory network (LSTM) and a state-of-the-art model named Extreme support vector regression (ESVR), with the Evolutionary mating algorithm (EMA) as the search algorithm. The benchmark models were established using EMA for hyperparameter optimization without feature selection. Ten basins with distinct hydroclimate characteristics minimally impacted by human activities from the Catchment Attributes and Meteorology for Large-sample Studies (CAMELS) data set were used to comprehensively examine the proposed models. Nonparametric approaches including the Friedman test and Wilcoxon signed ranks test were utilized to rank the models and detect the significance of the performance variation brought by EFS. The kernel ESVR model coupled with EFS-EMA (EFS-KESVREMA) ranked the first in the 1-month-ahead experiment with NSE = 0.734–0.944. EFS-ESVREMA and EFS-LSTMEMA ranked second and third, with NSE = 0.746–0.94 and NSE = 0.701–0.94 respectively. The results also demonstrated that EFS-EMA helps the benchmark models improve prediction accuracy at a confidence level of 95 % but it has no obvious effect on the standard SVR. As the forecast horizon increased to 3- and 5-month-ahead, EFS-ESVREMA maintained a reliable performance. Whereas EFS-LSTMEMA experienced a significant accuracy decrease in the 5-month-ahead experiment. The proposed models and relevant findings in this study have certain reference value for researchers in monthly streamflow forecasting.

Nonlinear causal network learning via Granger causality based on extreme support vector regression.

For complex networked systems, based on the consideration of nonlinearity and causality, a novel general method of nonlinear causal network learning, termed extreme support vector regression Granger causality (ESVRGC), is proposed. The nonuniform time-delayed influence of the driving nodes on the target node is particularly considered. Then, the restricted model and the unrestricted model of Granger causality are, respectively, formulated based on extreme support vector regression, which uses the selected time-delayed components of system variables as the inputs of kernel functions. The nonlinear conditional Granger causality index is finally calculated to confirm the strength of a causal interaction. Generally, based on the simulation of a nonlinear vector autoregressive model and nonlinear discrete time-delayed dynamic systems, ESVRGC demonstrates better performance than other popular methods. Also, the validity and robustness of ESVRGC are also verified by the different cases of network types, sample sizes, noise intensities, and coupling strengths. Finally, the superiority of ESVRGC is successful verified by the experimental study on real benchmark datasets.

Robust regression with extreme support vectors

Extreme Support Vector Machine (ESVM) is a nonlinear robust SVM algorithm based on regularized least squares optimization for binary-class classification. In this paper, a novel algorithm for regression tasks, Extreme Support Vector Regression (ESVR), is proposed based on ESVM. Moreover, kernel ESVR is suggested as well. Experiments show that, ESVR has a better generalization than some other traditional single hidden layer feedforward neural networks, such as Extreme Learning Machine (ELM), Support Vector Regression (SVR) and Least Squares-Support Vector Regression (LS-SVR). Furthermore, ESVR has much faster learning speed than SVR and LS-SVR. Stabilities and robustnesses of these algorithms are also studied in the paper, which shows that the ESVR is more robust and stable.

Spectral Data Modeling Based on Feature Extraction and Extreme Support Vector Regression

Spectral data such as near-infrared spectrum and frequency spectrum can simply the modeling of the difficulty-to-measured parameters. A novel modeling approach combined the feature extraction with extreme support vector regression (ESVR) is proposed. The latent variables space based feature extraction method can successfully complete the dimension reduction and independent variable extraction. The novel proposed ESVR leaning algorithm is realized by using extreme learning machine (ELM) kernel as SVR kernel, which is used to construct final models with better generalization. The experimental results based on the orange juice near-infrared spectra demonstrate that the proposed approach has better generalization performance and prediction accuracy.