Least Squares Support Vector Regression Research Articles

Online Prediction Method of Transmission Line Icing Based on Robust Seasonal Decomposition of Time Series and Bilinear Temporal–Spectral Fusion and Improved Beluga Whale Optimization Algorithm–Least Squares Support Vector Regression

Due to the prevalent challenges of inadequate accuracy, unstandardized parameters, and suboptimal efficiency with regard to icing prediction, this study introduces an innovative online method for icing prediction based on Robust STL–BTSF and IBWO–LSSVR. Firstly, this study adopts the Robust Seasonal Decomposition of Time Series and Bilinear Temporal–Spectral Fusion (Robust STL–BTSF) approach, which is demonstrably effective for short-term and limited sample data preprocessing. Subsequently, injecting a multi-faceted enhancement approach to the Beluga Whale Optimization algorithm (BWO), which integrates a nonlinear balancing factor, a population optimization strategy, a whale fall mechanism, and an ascendant elite learning scheme. Then, using the Improved BWO (IBWO) above to optimize the key hyperparameters of Least Squares Support Vector Regression (LSSVR), a superior offline predictive part is constructed based on this approach. In addition, an Incremental Online Learning algorithm (IOL) is imported. Integrating the two parts, the advanced online icing prediction model for transmission lines is built. Finally, simulations based on actual icing data unequivocally demonstrate that the proposed method markedly enhances both the accuracy and speed of predictions, thereby presenting a sophisticated solution for the icing prediction on the transmission lines.

Performance prediction and optimization strategy for LNG multistage centrifugal pump based on PSO-LSSVR surrogate model

The Multistage LNG cryogenic submersible pump is one of the crucial power equipment in LNG transport process. However, the high hydraulic losses and low efficiency of this pump make it necessary to optimize the structure of its impeller in order to reduce energy losses. In this study, an optimization strategy based on particle swarm optimization (PSO) and least squares support vector regression (LSSVR) machine is proposed to optimize the structure of impeller. The optimization of the impeller structure with the PSO-LSSVR method increased the head and efficiency of the LNG cryogenic submersible pump by 0.57% and 2.72%, respectively. Comparing the calculation results of PSO-LSSVR method with the CFD results, we found that the relative error between them don’t exceed 3%, which verified the calculation accuracy of the method. Then, an enstrophy dissipation theory is introduced to quantitatively analyze the pump energy loss. Comparing the impellers before and after optimization revealed that the maximum reduction in impeller energy loss was 18%.. The distribution and mechanism of vortex generation inside the impeller were analyzed with the Q-criterion and the relative vortex transport equation. The relative vortex transport equation revealed that the horseshoe vortex structure at the leading edge of the optimized impeller blade was suppressed, and the relative vortex stretching term and the Coriolis force term, which dominate the evolution of the vortex structure, were reduced. This study will provide related reference for the optimal design of Multistage LNG cryogenic submersible pumps.

Least Squares Generalization-Memorization Regression

Generalization-Memorization learning endeavors to minimize empirical risk while simultaneously reducing expected risk. Although we often do not pay attention to whether the training samples are accurately memorized during regression, improving generalization performance with better memory is always a goal pursued by regression. To tackle this issue, we introduce two new regression models, Least Squares Generalization-Memorization Regression (LSGMR) and Soft Least Squares Generalization-Memorization Regression (SLSGMR), by introducing the memory kernel learning on Least Squares Support Vector Regression (LSSVR). We conduct tests on these models using synthetic dataset and showcase that the LSSVR model can be viewed as a special case of our proposed model. Our experiments highlight that, for numerous problems, the models incorporating the employed memory mechanisms, LSGMR and SLSGMR, prove highly effective in yielding superior results compared to LSSVR on noise regression.

Uncertain least square support vector regression with imprecise observations

In this study, an innovative approach that combines least square support vector regression (LSSVR) with uncertainty theory to enhance its performance in dealing with low-quality or imprecise data from real-world be proposed. The resulting model, called uncertain least square support vector regression (ULSSVR), incorporates chance constraints and simplified parameter selection, which are critical to handle imprecise observations. A numerical algorithm called the conjugate residual method (CR) is introduced to reduce the computational complexity of the model solution. The experimental results using both small and medium-sized datasets demonstrate the superior performance of ULSSVR in terms of prediction accuracy and generalization ability compared to other models such as uncertain support vector regression (USVR), uncertain linear lodel, uncertain polynomial model, and uncertain growth models. ULSSVR not only improves prediction accuracy by at least 28.49% but also demonstrates faster computational speed. Overall, ULSSVR presents a promising solution for data science and internet applications where dealing with imprecise and low-quality data is a common challenge.

Performance prediction and optimization of hydrogenation feed pump based on particle swarm optimization – least squares support vector regression surrogate model

Due to high power consumption and low energy efficiency of the hydrogenation feed multistage pump, conducting structural optimization design and reducing energy losses for this pump is necessary. In this study, an optimization method based on the particle swarm optimization (PSO) algorithm and least squares support vector regression (LSSVR) surrogate model is proposed. The head and efficiency are taken as the optimization objectives of the multistage pump, and the optimization design of the two impellers is carried out. Through the optimization of the surrogate model, the head and efficiency of the multistage pump are increased by 3.99% and 2.91%, respectively. It is found that the optimized impeller has more stable flow characteristics than original impeller. Further, the entropy production method is introduced to analyze the energy loss of pump. The result shows that the pump optimized by PSO-LSSVR surrogate model can reduce 9% energy consumption under nominal flow rate. It is attributed to the inhibitory effect of the optimization scheme on the horseshoe vortex at leading edge and the wake vortex at trailing edge of impeller. This study provides a new idea for the optimization design of high efficiency and low energy consumption of multistage centrifugal pumps.

The least-square support vector regression model for the dyes and heavy metal ions removal prediction

Wastewater treatment plants are typically complex because they involve physical, chemical, and biological processes. Meanwhile, the efficiency of the removal of pollutants such as auramine (AO), methylene blue (MB), and ion Cadmium (Cd (II)) from the wastewater are difficult to be measured directly in real-time, as this measurement requires laboratory instruments that are time-consuming. The soft sensors could be the solution to perform the real-time prediction of the AO, MB, and Cd (II) removals’ capability. Hence, this study investigates the performances of a soft sensor, namely least-square support vector regression (LSSVR) to estimate the AO, MB, and Cd (II) removals’ ability. In this study, two wastewater-related case studies involving AO, MB, and Cd (II) removals were used to evaluate the predictive performance of the LSSVR. Additionally, its results were compared and analyzed with other soft sensors. For both case studies, notice that LSSVR gives the best results for AO, MB, and Cd (II) removals as compared to other soft sensor models where its root means square errors, mean absolute errors, and the approximate error, are lowered by 83% to 1,756%. Moreover, its coefficients of determination, denoted R 2 are the highest which are all more than or close to 0.9 for all the AO, MB, and Cd (II) removals even for the testing data for the case studies that were not used to develop the LSSVR model. In conclusion, LSSVR is more suitable for evaluating the effectiveness of the AO, MB, and Cd (II) removals at present.

Uncertainty analysis of discharge coefficient predicted for rectangular side weir using machine learning methods

Abstract The present study used three machine learning models, including Least Square Support Vector Regression (LSSVR) and two non-parametric models, namely, Quantile Regression Forest (QRF) and Gaussian Process Regression (GPR), to quantify uncertainty and precisely predict the side weir discharge coefficient (Cd) in rectangular channels. So, 15 input structures were examined to develop the models. The results revealed that the machine learning models used in the study offered better accuracy compared to the classical equations. While the LSSVR and QRF models provided a good prediction performance, the GPR slightly outperformed them. The best input structure that was developed included all four dimensionless parameters. Sensitivity analysis was conducted to identify the effective parameters. To evaluate the uncertainty in the predictions, the LSSVR, QRF, and GPR were used to generate prediction intervals (PI), which quantify the uncertainty coupled with point prediction. Among the implemented models, the GPR and LSSVR models provided more reliable results based on PI width and the percentage of observed data covered by PI. According to point prediction and uncertainty analysis, it was concluded that the GPR model had a lower uncertainty and could be successfully used to predict Cd.

Prediction of elevation points using three different heuristic regression techniques

The aim of this study is to estimate the digital elevation model, which is the most important data of the projects and needed in the engineering project, using latitude and longitude information of the elevation points and three different heuristic regression techniques. As the study area, an area with mid-level elevations, located in the Marmara region, and covering a part of the intersection of Edirne, Kırklareli and Tekirdağ provinces was chosen. In the study, the estimations were investigated for three different sized areas, and these areas are square areas with the dimensions of 1x1 km, 10x10 km and 100x100 km, respectively. A total of 3500 elevation points were used in the study, and this number is constant in all areas, and 60% of these points were used in the testing phase and 40% in the training phase. The models used in the study are M5 model tree (M5-tree), multivariate adaptive regression curves (MARS) and Least Square Support Vector Regression (LSSVR). The results of the models were evaluated according to three different comparison criteria. These, coefficient of determination (R2), Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) were used. When the modeling results are examined; M5-Tree regression method gave the best results (1), LSSVR method was better than MARS methods (2), The most successful input data was found in datasets using X and Y coordinates information, and the worst results were found in datasets using X coordinates (3). As the study area increased, the model performance did not improve (4). The least error was obtained in the modeling of 1x1 km area, and the highest R² was obtained from the modeling of 10x10 km area (5). It was concluded that the M5-tree method is a very successful method in elevation modeling.

Quantitative prediction of AFB1 in various types of edible oil based on absorption, scattering and fluorescence signals at dual wavelengths

This study aims to address the challenge of matrix interference of various types of edible oils on intrinsic fluorescence of aflatoxin B1 (AFB1) by developing a novel solution. Considering the fluorescence internal filtering effect, the absorption (μa) and reduced scattering (μ’s) coefficients at dual wavelengths (excitation: 375 nm, emission: 450 nm) were obtained by using integrating sphere technique, and were used to improve the quantitative prediction results for AFB1 contents in six different kinds of edible oils. A research process of “Monte Carlo (MC) simulation - phantom verification - actual sample validation” was conducted. The MC simulation was used to determine interference rule and correction parameters for fluorescence, the results indicated that the escaped fluorescence flux nonlinearly decreased with the μa, μ’s at emission wavelength (μa,em, μ’s,em) and μa at excitation wavelength (μa,ex), however increased with the μ’s at excitation wavelength (μ’s,ex). And the required optical parameters to eliminate the interference of matrix on fluorescence intensity are: effective attenuation coefficients at excitation and emission wavelengths (μeff,ex, μeff,em) and μ’s,ex. Phantom verification was conducted to explore the feasibility of fluorescence correction based on the identified parameters by MC simulation, and determine the optimal machine learning method. The modelling results showed that least squares support vector regression (LSSVR) model could reach the best performance. Three kinds of edible oil (peanut, rapeseed, corn), each with two brands were used to prepare oil samples with different AFB1 contamination. The LSSVR model for AFB1 based on μeff,ex, μeff,em, μ’s,ex and fluorescence intensity at 450 nm was calibrated, both correlation coefficients for calibration (Rc) and the validation (Rv) sets could reach 1.000, root mean square errors for calibration (RMSEC) and the validation (RMSEV) sets were as low as 0.038 and 0.099 respectively. This study proposed a novel method which is based solely on the absorption, scattering, and fluorescence characteristics at excitation and emission wavelengths to achieve accurate prediction of AFB1 content in different types of vegetable oils.

Deformation health diagnosis of RCC dams considering construction interfaces based on monitoring data and numerical simulation

The health diagnosis procedure applied to determine the deformation of a roller compacted concrete (RCC) dam is different from that of a conventional concrete dam. Hence, in this study, a deformation health diagnosis model was established for an RCC dam considering the construction interfaces by combining the hydrostatic component simulated using ABAQUS (2016) with the temperature and aging components calculated using a statistical method. The combined method can help monitor the structural health of RCC dam and determine the physical meaning and statistical law of the deformation of RCC dams. During the simulation process, the geometric characteristics of the construction interfaces were modeled using the interface equivalent expansion method. The material properties were modeled using multi-output least-squares support vector regression optimized with the Jaya algorithm. A case study demonstrated that the established deformation health diagnosis model has good fitting and prediction ability. The model and methods proposed in this study provide a new idea for the behavior analysis and numerical simulation of mass composite structures similar to RCC dams.

Black-box modeling of ship maneuvering motion using multi-output least-squares support vector regression based on optimal mixed kernel function

Black-box modeling has been widely used to predict ship maneuvering motion. A novel black-box modeling approach using optimal mixed kernel multi-output least-squares support vector regression (MMLSSVR) based on the artificial hummingbird algorithm (AHA) is proposed in this paper. To improve the performance of the MMLSSVR model, a mixed kernel function that combines a radial basis function and a polynomial kernel function to capture both global and local properties is proposed. The optimal hyper-parameters of the mixed kernel function are determined by the AHA, which employs the linear combination of the mean square error and the determinant of error covariance as the objective function to balance between prediction accuracy and stability. The performance of the proposed method is evaluated using SR108 large container ship data. The experimental results demonstrate that the proposed MMLSSVR model achieves desirable prediction accuracy and strong generalizability for ship maneuvering motion modeling.

A novel method for determination of peroxide value and acid value of extra-virgin olive oil based on fluorescence internal filtering effect correction

Peroxide value (PV) and acid value (AV) are widely used indicators for evaluating oxidation degree of olive oils. Fluorescence spectroscopy has been extensively studied on the detection of oil oxidation, however, the detection accuracy is limited due to internal filtering effect (IFE). Due to the primary and secondary IFE, at least two wavelengths of absorption information are required. Least squares support vector regression (LSSVR) models for PV and AV were established based on two absorption coefficients (μa) at 375 nm and emission wavelength and one fluorescence intensity at corresponding wavelength. The regression results proved that the model based on 375 and 475 nm could reach the best performance, with the highest correlation coefficient for prediction (rp) of 0.889 and 0.960 for PV and AV respectively. Finally, the explicit formulations for PV and AV were determined by nonlinear least squares fitting, and the rp could reach above 0.94 for two indicators.

Developing seasonal z-number regression for waste-disposal forecasting in a Taiwanese hospital

This study develops a seasonal z-number regression (SZR) to forecast the daily generated amounts of clinical waste for recycling and related waste. The proposed SZR designs new z-number intervals based on least-squares support-vector regression and is combined with the seasonal-decomposition method. Z-numbers characterize the uncertainty of a variable using a fuzzy number and the associated reliability of this fuzzy number. Biogeography-based optimization is employed to select the parameters of the SZR model. In this study, SZR, long short-term memory, support-vector regression with a genetic algorithm, a generalized regression neural network, the autoregressive integrated moving average, and a recurrent neural network are applied to forecast the daily amounts of generated clinical waste for recycling and related waste. The empirical results indicate the following. First, the SZR model demonstrates better performance and robustness than the other approaches for the prediction of the daily amount of clinical waste for recycling and related waste. Second, the SZR model also exhibits superior performance compared to the other approaches with respect to different testing terms. The proposed SZR method can help experts in hospitals develop reasonable waste-disposal projects that can help waste-management systems effectively achieve operational reliability and economy.

A critical review of multi-output support vector regression

Single-output regression is a widely used statistical modeling method to predict an output based on one or more features of a datapoint. If a dataset has multiple outputs, they can be predicted independently from each other although this disregards potential correlations and thus may negatively affect the predictive performance. Therefore, multi-output regression methods predict multiple outputs simultaneously. One way to approach single-output regression is by using methods based on support vectors such as support vector regression (SVR) or least-squares SVR (LS-SVR). Based on these two, previous works have devised multi-output support vector regression methods. In this review, we introduce a unified notation to summarize the single-output support vector regression methods SVR and LS-SVR as well as state-of-the-art multi-output support vector regression methods. Furthermore, we implemented a workflow for subject- and record-wise bootstrapping and nested cross-validation experiments, which we used for an exhaustive evaluation of all single- and multi-output support vector regression methods on synthetic and non-synthetic datasets. Although the reviewed papers claim that their multi-output methods improve regression performance, we find that none of them outperform both single-output methods SVR and LS-SVR for various reasons. Due to these results, we reflected about the general concept of support vector regression and then concluded that support vector regression methods do not appear to be suitable for the task of multi-output regression.

Large discrepancy between future demand and supply of agricultural water in northwestern Iran; evidence from WEAP-MODFLOW-machine learning under the CMIP6 scenario

The agricultural sector in northwestern Iran uses about 95 % of the region’s available water resources and nearly 98 % of are aquifer water. Despite the regiońs previous richness in water resources, the supply is now quickly decreasing. The Ghorveh-Dehgolan sub-basin in northwestern Iran consists of three dams as well as three aquifers that supply water resources for agriculture, domestic uses, and industry. We used the Water Evaluation and Planning System (WEAP) and MODFLOW hydrologic model together with machine learning to investigate the status of future water resources in this region under the influence of climate change. At first, the current status of water resources in the area was simulated using WEAP and MODFLOW for the period 2008–2021. Then, the impact of climate change on surface water and groundwater resources in 2026–2045 was evaluated under the SSP2.6 and SSP8.5 emission scenarios. Multivariate Adaptive Regression Splines (MARS) were used to generate future runoff and least-squares support vector regression (LSSVR) was used to predict the groundwater levels for the period 2026–2045. The results showed that under the SSP8.5 scenario, precipitation may decrease by about 15 % in the future compared to the reference period (1987–2005). The reservoir inflow is thus likely to decrease by 25 %, which may decrease the groundwater level by 1.2 m per year in the coming period. The groundwater level in Ghorveh and Dehgolan aquifers dropped by nearly 20 m during the period 2008–2021 and still about 5 to 25 million cubic meters (MCM) of water demand for the agricultural sector were not supplied. If the current trend continues, up to 65 MCM of water demand in the region will not be met in the near future. These results call for immediate action by government and water managers. We outline different alternatives to improve the pessimistic future outlook that can be used to amend the water management in the basin.

Dynamic nonlinear soft sensor modelling method using linear slow feature analysis and least squares support vector regression for batch processes

AbstractData‐driven soft sensor models have been extensively utilized in industrial processes. Batch processes are usually employed to manufacture low‐volume and high value‐added products in chemical, materials, and pharmaceutical industries. The most distinctive features of batch process lie in nonlinear, repetition, and slow time varying characteristics. In this paper, a data‐driven soft sensor modelling method based on linear slow feature analysis (LSFA) and least squares support vector regression (LSSVR) is proposed. In this method, LSFA was used to effectively capture the driving force behind the data features that change as slowly as possible. Then, a LSSVR model was constructed between the extracted slow feature variables and quality variables. Finally, a numerical example, industrial penicillin fermentation processes, and cobalt oxalate synthesis process were utilized to confirm the prediction accuracy and model reliability of the proposed approach.

State of energy estimation of lithium battery based on least squares and support vector regression

To enhance the estimation precision of lithium battery state of energy (SOE) and avoid the complex modeling and parameter identification process in the estimation process, a joint algorithm using the least square method (LS) and support vector regression (SVR) is proposed in this paper. Data on the voltage, current, temperature, and charge state of the lithium battery are extracted and incorporated into the energy factor. The lithium battery SOE estimation model is designed by the least-squares support vector regression (LS-SVR) methods, while the grid optimization method is deployed to optimize the hyper-parameters. Finally, the SOE prediction of lithium batteries under a dynamic scenario is realized. Comparison results show that the energy factor chosen in our article accurately reflects the SOE of lithium batteries, and the MAE is within 1.5%.

Developing a Novel Long Short-Term Memory Networks with Seasonal Wavelet Transform for Long-Term Wind Power Output Forecasting

Long-term wind power forecasting is a challenging endeavor that requires predictions that span years into the future. Accurate forecasting is crucial for optimizing energy production, grid integration, maintenance scheduling, and financial planning. This study attempts to first develop the long short-term memory networks (LSTM) with a seasonal wavelet transform forecasting model for practical long-term wind power forecasting problems with seasonal and regional influences on wind power and the instability of data signals. This model encapsulates wavelet transformation and seasonal decomposition, and employs LSTM for forecasting. The new prediction model adopted seasonal decompositions and two LSTMs to approach low- and high-frequency series datasets, as well as the wavelet synthesis prediction values. Furthermore, the parameters of the LSTM models are selected using stochastic optimization. For a comprehensive evaluation, the proposed LSTM with seasonal wavelet transform is compared with seven methods, including seasonal LSTM (SLSTM), wavelet LSTM (WLSTM), and the seasonal auto-regressive integrated moving average (SARIMA), back propagation neural network (BPNN), generalized regression neural network (GRNN), least square support vector regression (LSSVR), and support vector regression (SVR) were employed for long-term wind power output forecasting of wind farms. The empirical results underscore that the performance of the proposed forecasting model is better than other methods in terms of forecasting accuracy, which could efficiently provide reliable long-term predictions for long-term wind power output forecasting.

Influence of phase change material and nano silica aerogel aggregates on the characteristics of cementitious composite: An experimental and predictive study

This paper studies the influence of two high-performance thermal insulating materials including aerogel and phase change materials (PCM) aggregates, on the key characteristics of cementitious composites in experimental and predictive aspects. Thermal conductivity (ThC), rapid chloride migration test (RCMT), permeable pore volume, compressive strength, dry density, and microstructural analysis were performed for the mechanical properties. Moreover, Least Squares Support Vector Regression (LSSVR) as a machine learning method was introduced to evaluate and provide a proper method for non-destructive evaluation of parameters. The results of experiments indicated that both aerogel and PCM aggregates were highly effective although the aerogel showed more efficiency in enhancing the thermal insulation property of cementitious mixtures. In addition, the compressive strength of the mixtures with PCM composites was higher than the aerogel aggregates, where the compressive strength of the mixture containing 20 % paraffin was 50 % higher than the aerogel mixture with similar incorporation levels. Moreover, the mixtures with PCM aggregates had significantly lower water and chloride permeability compared to those of plain mixtures, and mixtures consisted of aerogel aggregates. The scanning electron microscope (SEM) analyses, also, revealed that the use of the aerogel aggregates led to an open and porous structure of the mixtures; however, PCM aggregates had a weak interfacial transition zone (ITZ) in the vicinity of cement paste. Furthermore, LSSVR model provided a close prediction of the effective parameters with an acceptable accuracy in capturing the influences of important parameters.

Evaluation of Landweber Coupled Least Square Support Vector Regression Algorithm for Electrical Capacitance Tomography for LN2–VN2 Flow

The electric capacitance tomography (ECT) technique has been widely used in phase distribution reconstruction, while the practical application raised nonideal noise and other errors for cryogenic conditions, requiring a more accurate algorithm. This paper develops a new image reconstruction algorithm for ECT by coupling the traditional Landweber algorithm with the least square support vector regression (LSSVR) for cryogenic fluids. The performance of the algorithm is quantitatively evaluated by comparing the inversion images with the experimental results for both the room temperature working medium with the dielectric constant ratio close to cryogenic fluid and the cryogenic fluid of liquid nitrogen/nitrogen vapor (LN2-VN2). The inversion images based on the conventional LBP and Landweber algorithms are also presented for comparison. The benefits and drawbacks of the developed algorithms are revealed and discussed, according to the results. It is demonstrated that the correlated coefficients of the images based on the developed algorithm reach more than 0.88 and a maximum of 0.975. In addition, the minimum void fraction error of the algorithm is reduced to 0.534%, which indicates the significant optimization of the LSSVR coupled method over the Landweber algorithm.