Square-support Vector Regression Research Articles

Adsorption simulation of 2,4-D pesticide on novel zinc-based 2-amino-4-(1H-1,2,4-triazole-4-yl)benzoic acid coordination complexes using machine learning approach.

The capacity of zinc-based 2-amino-4-(1H-1,2,4-triazole-4-yl)benzoic acid coordination complex (Zn(NH2-TBA)2) and modified Zn(NH-TBA)2COMe complex for removal of 2,4-dichlorophenoxyacetic acid (2,4-D) from aqueous solutions was investigated through adsorption modeling and artificial intelligence tools. Analyzing the adsorption characteristics of pesticides helps in studying the groundwater pollution by pesticides in agriculture area.In this study, Zn(NH2-TBA)2 was synthesized using Schiff base and its surface was modified using acetic anhydride group and their physical characteristics were identified using proton NMR, FTIR, and XRD. NMR results showed maximum modification yield obtained was 65% after 5days. The porous structure and surface area monitored using nitrogen isotherm and BET surface area analysis presented relatively less surface area and porosity after modification. Adsorption modelling indicated that Toth model with a maximum adsorption capacity of 150.8mg/g and 100.7mg/g represents the homogenous adsorption systems which satisfy both low- and high-end boundary of adsorbate concentration in all settings according to the optimum point, while the kinetics and rate of 2,4-D adsorption follow the pseudo-first-order kinetic model in all situations. Artificial neural network (ANN), support vector regression, and particle swarm optimized least squares-support vector regression (PSO-LSSVR) were used for the optimization and modelling of adsorbent mass, adsorbate concentration, contact time, and temperature to develop predictive equations for the simulation of the adsorption efficiency of 2,4-D pesticide. The obtained results exhibited the better performance of ANN and PSO-LSSVR for prediction of adsorption results. The mean square error values of ANN (0.001, 0.012) and PSO-LSSVR (0.121, 0.105) were obtained for Zn(NH2-TBA)2 and Zn(NH-TBA)2COMe, respectively, while their respective coefficient of determination (R2) obtained were 0.999 and 0.988 for ANN and 0.980 and 0.825 for PSO-LSSVR. The study specified that machine learning predictive behavior performed better for Zn(NH2-TBA)2 compared to Zn(NH-TBA)2COMe that is also supported by theoretical kinetics and isotherm models. The research concludes that artificial intelligence models are the most efficient tools for studying the predictive behavior of adsorption data.

Online Prediction Method of Transmission Line Icing Based on Robust Seasonal Decomposition of Time Series and Bilinear Temporal–Spectral Fusion and Improved Beluga Whale Optimization Algorithm–Least Squares Support Vector Regression

Due to the prevalent challenges of inadequate accuracy, unstandardized parameters, and suboptimal efficiency with regard to icing prediction, this study introduces an innovative online method for icing prediction based on Robust STL–BTSF and IBWO–LSSVR. Firstly, this study adopts the Robust Seasonal Decomposition of Time Series and Bilinear Temporal–Spectral Fusion (Robust STL–BTSF) approach, which is demonstrably effective for short-term and limited sample data preprocessing. Subsequently, injecting a multi-faceted enhancement approach to the Beluga Whale Optimization algorithm (BWO), which integrates a nonlinear balancing factor, a population optimization strategy, a whale fall mechanism, and an ascendant elite learning scheme. Then, using the Improved BWO (IBWO) above to optimize the key hyperparameters of Least Squares Support Vector Regression (LSSVR), a superior offline predictive part is constructed based on this approach. In addition, an Incremental Online Learning algorithm (IOL) is imported. Integrating the two parts, the advanced online icing prediction model for transmission lines is built. Finally, simulations based on actual icing data unequivocally demonstrate that the proposed method markedly enhances both the accuracy and speed of predictions, thereby presenting a sophisticated solution for the icing prediction on the transmission lines.

Least Squares Generalization-Memorization Regression

Generalization-Memorization learning endeavors to minimize empirical risk while simultaneously reducing expected risk. Although we often do not pay attention to whether the training samples are accurately memorized during regression, improving generalization performance with better memory is always a goal pursued by regression. To tackle this issue, we introduce two new regression models, Least Squares Generalization-Memorization Regression (LSGMR) and Soft Least Squares Generalization-Memorization Regression (SLSGMR), by introducing the memory kernel learning on Least Squares Support Vector Regression (LSSVR). We conduct tests on these models using synthetic dataset and showcase that the LSSVR model can be viewed as a special case of our proposed model. Our experiments highlight that, for numerous problems, the models incorporating the employed memory mechanisms, LSGMR and SLSGMR, prove highly effective in yielding superior results compared to LSSVR on noise regression.

Tracing the geographic origin of CTC black tea based on colorimetric sensor array response to taste substances combined with chemometrics

The quality of crush–tear–curl black tea (CTC-BT) varies greatly by geographic origin. Origin traceability is crucial for consumer interest protection, market order regulation, and food safety monitoring. This paper proposes a fast and accurate method for qualitative discrimination of CTC-BT origins and quantitative detection of its key taste-presenting substances. The method involves a simple colorimetric sensor array and ultraviolet–visible spectroscopy. The effects of various variable screening methods on modeling results were compared. A particle swarm optimization–based support vector machine achieved the highest performance in qualitative discrimination, with a correct classification rate of 99.48%. Based on their origin-distinguishing contributions and dose-over-thresholds, seven key taste-presenting substances were screened, namely, theaflavin, caffine, vitexin-2-O-rhamnoside, rutin, epigallocatechin gallate, epicatechin gallate, gallic acid. A least squares-support vector regression model achieved accurate quantification of the seven aforementioned compounds (square root of determination coefficient of prediction >0.9698, residual prediction deviation >2).

Influence of phase change material and nano silica aerogel aggregates on the characteristics of cementitious composite: An experimental and predictive study

This paper studies the influence of two high-performance thermal insulating materials including aerogel and phase change materials (PCM) aggregates, on the key characteristics of cementitious composites in experimental and predictive aspects. Thermal conductivity (ThC), rapid chloride migration test (RCMT), permeable pore volume, compressive strength, dry density, and microstructural analysis were performed for the mechanical properties. Moreover, Least Squares Support Vector Regression (LSSVR) as a machine learning method was introduced to evaluate and provide a proper method for non-destructive evaluation of parameters. The results of experiments indicated that both aerogel and PCM aggregates were highly effective although the aerogel showed more efficiency in enhancing the thermal insulation property of cementitious mixtures. In addition, the compressive strength of the mixtures with PCM composites was higher than the aerogel aggregates, where the compressive strength of the mixture containing 20 % paraffin was 50 % higher than the aerogel mixture with similar incorporation levels. Moreover, the mixtures with PCM aggregates had significantly lower water and chloride permeability compared to those of plain mixtures, and mixtures consisted of aerogel aggregates. The scanning electron microscope (SEM) analyses, also, revealed that the use of the aerogel aggregates led to an open and porous structure of the mixtures; however, PCM aggregates had a weak interfacial transition zone (ITZ) in the vicinity of cement paste. Furthermore, LSSVR model provided a close prediction of the effective parameters with an acceptable accuracy in capturing the influences of important parameters.

Reconstruction of spatially continuous time-series land subsidence based on PS-InSAR and improved MLS-SVR in Beijing Plain area

ABSTRACT Beijing has undergone severe settlement in recent years. Persistent Scatterers Interferometric Synthetic Aperture Radar (PS-InSAR) technique has been widely used to derive time-series land deformation. However, existing studies have faced two challenges: (1) the nonlinear characteristics of time-series subsidence has not been fully investigated; (2) since PS points are normally distributed in urban areas with high building density, measurement gaps usually exist in nonurban areas. To address the challenges, we presented a new method to reconstruct spatially continuous time-series deformation. First, PS-InSAR was used to retrieve the deformation based on 135 scenes of Envisat ASAR and Radarsat-2 images from 2003 to 2020. Polynomial Curve Fitting (PCF) was then used to model nonlinear time-series deformation for the PS points. In the PS measurement gaps, Iterative Self-Organizing Data Analysis Technique (ISODATA) and Multi-output Least Squares Support Vector Regression (MLS-SVR) were used to estimate the PCF coefficients and then time-series deformation considering 40 features including thickness of the compressible layers, annual groundwater level, etc. The major results showed that (1) compared to linear, quadratic, and quartic models, cubic polynomial model generated better fit for the time-series deformation (R2 ≈0.99), suggesting obvious nonlinear temporal pattern of deformation; (2) the time-series deformation over measurement gaps reconstructed by ISODATA and MLS-SVR had satisfactory accuracy (R2 = 0.92, MAPE < 15%) and yielded higher accuracy (R2 = 0.947) than IDW (R2 = 0.687) and Ordinary Kriging (R2 = 0.688) interpolation methods. The reconstructed results maintain the nonlinear characteristics and ensure the high spatial resolution (120 m) of time-series deformation. Among the 40 predictor variables, ground water level datasets are the most influential predictors of time-series deformation.

Integration Vis-NIR Spectroscopy and Artificial Intelligence to Predict Some Soil Parameters in Arid Region: A Case Study of Wadi Elkobaneyya, South Egypt

Understanding and determining soil properties is reflected in improving farm management and crop production. Soil salinity, pH and calcium carbonate are among the factors affecting the soil’s physical and chemical properties. Hence, their estimation is very important for agricultural management, especially in arid regions (Wadi Elkobaneyya valley, located in the northwest of Aswan Governorate, Upper Egypt). The study objectives were to characterize and develop prediction models for soil salinity, pH and calcium carbonate (CaCO3) using integration soil analysis and spectral reflectance vis-NIR spectroscopy. To achieve the study objectives, three multivariate regression models: Partial Least Squares Regression (PLSR), Multivariate Adaptive Regression Splines (MARS) and Least Square-Support Vector Regression (LS-SVR)); and two machine learning algorithms, Random Forest (RF) and Artificial Neural Networks (ANN) were used. Ninety-six surface soil samples were collected from the study area at depths 0–5 cm. The data were divided into a calibration dataset (70% of the total) and a validation dataset (30% of the total dataset). The obtained results represent that the PLSR model was the best model for soil pH parameters where R2 of calibration and validation predictability = 0.68 and 0.52, respectively. The LS-SVR model was the best model to predict soil Electrical Conductivity (EC) and soil Calcium Carbonate (CaCO3) content, with R2 0.70 and 0.74 for calibration and R2 0.26 and 0.47 for validation, respectively. On the other hand, the results of the implemented machine learning algorithm model showed that RF was the best model to predict soil pH and CaCO3, as the R2 was 0.82 for calibration and 0.57 for validation, respectively. Nevertheless, the best model for predicting soil EC was ANN, with an R2 of 0.96 for calibration and 64 for validation. The results show the advantages of machine learning models for predicting soil EC, pH and CaCO3 by Vis-NIR spectroscopy. Therefore, Vis-NIR spectroscopy is considered faster and more cost-efficient and can be further used in environmental monitoring and precision farming.

ε-Insensitive Least Squares Support Vector Regression with Sequential Minimal Optimization

The [Formula: see text]-insensitive-Least Squares Support Vector Regression ([Formula: see text]-LSSVR) optimization problem with [Formula: see text]-insensitive quadratic loss function is a non-smooth, piecewise quadratic complex problem. Therefore, the optimal solution of [Formula: see text]-LSSVR takes longer than Classic LSSVR. Additionally, due to its unbounded influence function, [Formula: see text]-LSSVR, like classical LSSVR, shows poor generalization ability in the presence of outliers. To overcome the above-mentioned drawbacks, this paper presents an [Formula: see text]-insensitive Weighted Least Squares Support Vector Regression ([Formula: see text]-WLSSVR) with equality constraints to increase robustness against outliers. To reduce the computational complexity and time, the optimization problem of [Formula: see text]-WLSSVR is solved by the Sequential Minimal Optimization (SMO) algorithm. Extensive simulations have shown that the presented [Formula: see text]-WLSSVR model combined with the SMO algorithm is more effective and efficient than LSSVR and its known sparse variants.

Modeling and Estimation of CO2 Emissions in China Based on Artificial Intelligence.

Since China's reform and opening up, the social economy has achieved rapid development, followed by a sharp increase in carbon dioxide (CO2) emissions. Therefore, at the 75th United Nations General Assembly, China proposed to achieve carbon peaking by 2030 and carbon neutrality by 2060. The research work on advance forecasting of CO2 emissions is essential to achieve the above-mentioned carbon peaking and carbon neutrality goals in China. In order to achieve accurate prediction of CO2 emissions, this study establishes a hybrid intelligent algorithm model suitable for CO2 emissions prediction based on China's CO2 emissions and related socioeconomic indicator data from 1971 to 2017. The hyperparameters of Least Squares Support Vector Regression (LSSVR) are optimized by the Adaptive Artificial Bee Colony (AABC) algorithm to build a high-performance hybrid intelligence model. The research results show that the hybrid intelligent algorithm model designed in this paper has stronger robustness and accuracy with relative error almost within ±5% in the advance prediction of CO2 emissions. The modeling scheme proposed in this study can not only provide strong support for the Chinese government and industry departments to formulate policies related to the carbon peaking and carbon neutrality goals, but also can be extended to the research of other socioeconomic-related issues.

Parametric study and application of a data-mining model in 2D and 3D micro-fin tubes

• The application of least squares-support vector regression method in micro-fin study. • The best 2D and 3D micro-fins had similar design parameters. • The best 2D micro-fin tube enhanced the efficiency index up to %18. • Importance of micro-fin’s axial pitch over height ratio on the efficiency index. Micro-fins (<0.5 mm tall) are an engineered roughness with the advantage of reducing thermal resistance and the disadvantage of increased pressure drop when applied inside a tube in heat-exchange applications. The competing effects highlight the need for careful optimization that identifies micro-fin surfaces with the potential to match heat exchanger design needs. Hence, the objectives of this study are chosen to enable efficient optimization in future studies. The main goals are: (1) study the effects of micro-fin design variables on heat transfer and friction factors; and (2) evaluate the potential of a data-mining model as a surrogate of computational fluid dynamic ( CFD ) models in 2 dimensional ( D ) and 3 D micro-fin tubes. This study applied conductive and convective heat transfer and turbulent fluid flow simulation to evaluate the performance of different 2 D and 3 D micro-fin tubes. Different configurations were generated by varying micro-fin height ( e ), helix angle ( α ), number of starts ( N f ), and discontinuity features. Coupled solid and periodic fluid domains were applied in ANSYS Fluent 19.1 . Performance was mapped for 210 different simulations (including a smooth tube) using a realizable k - ε turbulence model at Reynolds number ( Re ) of 48,928. Two different least squares-support vector regression ( LS - SVR ) models were employed to estimate the Colburn j factor as a function of geometric variables and the Fanning friction factor ( f ) as a function of geometric variable and j factor. Results of the parametric study showed that the best 2 D micro-fin tube can enhance efficiency index ( η ) up to 1.18. Results of the LS - SVR model showed that the percentage of average absolute error ( AAE ) between simulated and estimated j and f factors are 2.05% and 2.93% for 3 D micro-fin tubes, respectively.

Potential impacts of climate change on groundwater levels in Golpayegan Plain, Iran

AbstractGroundwater level forecasting is an essential priority for planning and managing groundwater resources. This study aims to investigate the effect of climate change on the monthly groundwater level in the Golpayegan aquifer in the future (2017–2032). After a spatio-temporal analysis, the Least Squares Support Vector Regression (LSSVR) model was used to simulate the monthly groundwater level in the historical period (2002–2017). The input data included precipitation, temperature, pan evaporation, soil moisture (from the ESA CCA SM product), and groundwater level in observation wells on a monthly time-scale. Future climatic data were downloaded from the CanEsm5 model of CMIP6 for the SSP1-2.6 and SSP5-8.5 climate scenarios and then downscaled using the Change Factor Approach (CFA). The spatial analysis of groundwater levels indicated four different behaviors in the observation wells in the Golpayegan aquifer, resulting in four different clusters using the AGNES clustering method. Historical and future period modeling were performed separately for each of the four observation wells from each cluster. The modeling in the historical period demonstrated an average of NRMSE (0.09), MBE (0.030), and R2 (0.94) for the four clusters. The groundwater level in all clusters showed a decreasing trend in the future period, with SSP5-8.5 (average: 3.9 cm/month) showing a greater decrease than the SSP1-2.6 (average: 0.5 cm/month) scenario. The decline in groundwater level under SSP5-8.5 compared with SSP1-2.6 was more, respectively, 4.8, 5.8, 9.9 and 3.7 metres for clusters 1–4. The results indicate the acceptable efficiency and accuracy of the LSSVR model results in evaluating the effects of climate change on groundwater levels.

Prediction of Pile Bearing Capacity Using Opposition-Based Differential Flower Pollination-Optimized Least Squares Support Vector Regression (ODFP-LSSVR)

Pile foundations are widely used for high-rise structures constructed in soft ground. The bearing capacity of pile is a crucial parameter required during the design and construction phase of pile foundation engineering projects. In practice, accurate predictions of pile bearing capacity are challenging due to a complex interplay of various geotechnical engineering factors including pile characteristics and ground conditions. This study proposes a data-driven model for coping with the problem of interest that hybridizes machine learning and metaheuristic approaches. Least squares support vector regression (LSSVR) is used for analyzing a dataset containing historical records of pile tests. Based on such datasets, LSSVR is capable of generalizing a multivariate function that estimates values of pile bearing capacity based on a set of variables describing pile characteristics and ground conditions. Moreover, opposition-based differential flower pollination (ODFP) metaheuristic is proposed to optimize the LSSVR learning process. Experimental results supported by the statistical test showed that the proposed ODFP-optimized LSSVR can achieve a good predictive performance in terms of root mean square error, mean absolute percentage error mean absolute error, and coefficient of determination. These results confirm that the ODFP-optimized LSSVR can be a potential alternative to assist civil engineers in the task of pile bearing capacity estimation.

Tuning Rules for Fractional Order PID Controller Using Data Analytics

Flexibility and robust performance have made the FOPID (Fractional Order PID) controllers a better choice than PID (Proportional, Integral, Derivative) controllers. But the number of tuning parameters decreases the usage of FOPID controllers. Using synthetic data in available FOPID tuners leads to abnormal controller performances limiting their applicability. Hence, a new tuning methodology involving real-time data and overcomes the drawbacks of mathematical modeling is the need of the hour. This paper proposes a novel FOPID controller tuning methodology using machine learning algorithms. Feed Forward Back Propagation Neural Network (FFBPNN), Multi Least Squares Support Vector Regression (MLSSVR) chosen to design Machine Learning based Optimal Tuner (MLOT) can handle the interdependency between the controller parameters and multiple outputs for multiple inputs.The proposed tuner finds application in the control of power and energy systems. It can accomplish tracking, disturbance-rejection, and robustness controller performances, thus making FOPID controller design easier and accurate. Comparisons with existing FOPID tuning rules show better controller performances and easy tuning. Thus, this paper addresses a unique, real-time, model-free, easily tunable FOPID tuning methodology satisfying plant requirements.

Early Estimation of the Long-Term Deflection of Reinforced Concrete Beams Using Feature Refinement Integrated with Least Squares Support Vector Regression and Forensic-Based Investigation

Early Estimation of the Long-Term Deflection of Reinforced Concrete Beams Using Feature Refinement Integrated with Least Squares Support Vector Regression and Forensic-Based Investigation

Calibration of SWAT and three data-driven models for monthly stream flow simulation in Maharlu Lake Basin

Abstract In this study, the performance of SWAT hydrological model and three computational intelligence methods used to simulate river flow are investigated. After collecting the data required for all models used, the calibration and validation stages were performed. Using the SWAT model and three methods of the Extreme Machine Learning (EML), the Support Vector Regression (SVR), and the Least Squares Support Vector Regression (LSSVR), Maharlu Lake Basin stream flow was simulated and the results obtained at Shiraz station were used for this study. A noise reduction filter was employed to improve the results from the computational intelligence methods, and SUFI-2 algorithm was used to analyze the uncertainty of the SWAT model. Finally, in order to evaluate the models developed and the SWAT model, three statistics (RMSE), (R²), and (NS) coefficient were used. The results indicated that the SWAT model and the machine learning models were generally appropriate tools for daily flow modeling, but the LSSVR model showed less errors in both learning and testing, with the coefficients NS = 0.997 and R² = 0.997 in the calibration stage and NS = 0.994 and R² = 0.994 in the validation stage, which prove their better performance compared to the other methods and the SWAT model.

Discovery of ester lubricants with low coefficient of friction on material surface via machine learning

The design of lubricants with excellent anti-wear performance is the focus of current tribological research. The anti-wear performance of lubricants is fundamentally determined by their chemical structures. Thus, the development of efficient quantitative structure–property relationship (QSPR) to accurately predict lubricant’s anti-wear performance can be a powerful approach toward the design of new lubricants. To this end, this work aims to establish an efficient QSPR model using Least Squares-Support Vector Regression (LS-SVR) method, which employs the screened important molecular descriptors as the input to represent the characteristics of lubricants. Specifically, 1444 molecular descriptors of 54 ester molecules are computed based on the molecular graphics and adjacency matrix. Multiple stepwise regression (MSR) algorithm is used to select important molecular descriptors, and 5 important and unrelated molecular descriptors are used as input in LS-SVR. The mean relative error (MRE) of the training set, test set and external set from the established QSPR model are 2.92%, 6.03% and 9.30% compared with experimental values, respectively, indicating the developed QSPR model exhibits good predictive ability in CoF values of certain molecules and can be used to predict the anti-wear performance of lubricants only through the 5 important molecular descriptors. In addition, it is shown that the anti-wear performance of lubricants can be enhanced by increasing the longest aliphatic chain, the relative molecular mass and van der Waals volume of the studied lubricant molecules via MSR analysis. In the end, we screened the compounds containing ester groups in the database and obtained 8 potential lubricants, which provided guidance for the discovery of new lubricants.

Numerical learning approximation of time-fractional sub diffusion model on a semi-infinite domain

Abstract The propose of this research is to apply a novel numerical learning approximation of time-fractional sub diffusion model on a semi-infinite domain. This model is a nonlinear fractional differential equation in two unbounded dimensions. Combination of Least Squares Support Vector Regression (LSSVR) based on generalized Laguerre Functions kernel and collocation/Galerkin method is applied to obtain the solutions. The marching in time technique is applied for time discretization. Numerical results verify that proposed methods have high convergence and performance.

Twin Least Square Support Vector Regression Model Based on Gauss-Laplace Mixed Noise Feature with Its Application in Wind Speed Prediction.

In this article, it was observed that the noise in some real-world applications, such as wind power forecasting and direction of the arrival estimation problem, does not satisfy the single noise distribution, including Gaussian distribution and Laplace distribution, but the mixed distribution. Therefore, combining the twin hyperplanes with the fast speed of Least Squares Support Vector Regression (LS-SVR), and then introducing the Gauss–Laplace mixed noise feature, a new regressor, called Gauss-Laplace Twin Least Squares Support Vector Regression (GL-TLSSVR), for the complex noise. Subsequently, we apply the augmented Lagrangian multiplier method to solve the proposed model. Finally, we apply the short-term wind speed data-set to the proposed model. The results of this experiment confirm the effectiveness of our proposed model.

Handling of Constraints in Efficient Global Optimization

Although the Efficient Global Optimization (EGO) algorithm has been widely used in multi-disciplinary optimization, it is still difficult to handle multiple constraint problems. In this study, to increase the accuracy of approximation, the Least Squares Support Vector Regression (LSSVR) is suggested to replace the kriging model for approximating both objective and constrained functions while the variances of these surrogate models are still obtained by kriging. To enhance the ability to search the feasible region, two criteria are suggested. First, a Maximize Probability of Feasibility (MPF) strategy to handle the infeasible initial sample points is suggested to generate feasible points. Second, a Multi-Constraint Parallel (MCP) criterion is suggested for multiple constraints handling, parallel computation and validation, respectively. To illustrate the efficiency of the suggested EGO-based method, several deterministic benchmarks are tested and the suggested methods demonstrate a superior performance compared with two other constrained algorithms. Finally, the suggested algorithm is successfully utilized to optimize the fiber path of variable-stiffness beam and lightweight B-pillar to demonstrate the performance for engineering applications.

A new approach to the numerical solution of Fredholm integral equations using least squares-support vector regression

In this paper, we develop a machine learning method with the Least Squares Support Vector Regression (LS-SVR) for the numerical solution of Fredholm integral equations. Two different approaches are proposed for training the network by using the shifted Legendre kernel, the collocation and Galerkin LS-SVR approaches. As with the standard LS-SVR for known dataset regression, the formulation of the method gives rise to an optimization problem. An equivalent system of algebraic equations is then derived and in linear cases discussed in terms of the sparsity of the matrices and computational efficiency. Finally, the method is carried out on some numerical examples, including nonlinear and multidimensional cases to show the accuracy and efficiency of the method.