Least Square Support Vector Machine Approach Research Articles

Machine learning models applied to altimetry era tide gauge and grid altimetry data for comparative long-term trend estimation: A study from Shikoku Island, Japan

Estimation of sea level trends is essential for understanding sea level rise dynamics. In this study, the performance of traditional Ordinary Least Squares (OLS) linear trend forecasting is compared with modern machine learning techniques, specifically Random Forests (RF) and Least Squares Support Vector Machines (LS-SVM).These methods are applied to 50 years of long-term tide gauge (TG) data from six tide gauge stations off the coast of Shikoku Island, Japan, and CMEMS Grid Altimetry data from 1993 to the present. The analysis uses OLS, RF, and LS-SVM to estimate trends from both data sets and compares the results. The objective is to determine the consistency and accuracy of RF and LS-SVM methods compared to the OLS method. The results indicate that machine learning algorithms (LS-SVM) effectively estimate sea level trends, offering potential improvements in precision for both long-term and medium-term analyses. Shikoku Island's coastal sea level trend is determined as 2.91±1.44 mm/yr using TG data and 3.00±1.52 mm/yr using CMEMS Grid Altimeter data with the OLS approach. Using the LS-SVM approach, the trend is found as 2.96±1.58 mm/yr with TG data and 3.02±1.60 mm/yr with CMEMS Grid Altimetry data. The novelty of this study lies in its thorough comparison of traditional and machine learning approaches for sea level trend estimation, providing valuable insights for future predictions of the sea level rise.

A new hybrid method with data-characteristic-driven analysis for artificial intelligence and robotics index return forecasting

Forecasting returns for the Artificial Intelligence and Robotics Index is of great significance for financial market stability, and the development of the artificial intelligence industry. To provide investors with a more reliable reference in terms of artificial intelligence index investment, this paper selects the NASDAQ CTA Artificial Intelligence and Robotics (AIRO) Index as the research target, and proposes innovative hybrid methods to forecast returns by considering its multiple structural characteristics. Specifically, this paper uses the ensemble empirical mode decomposition (EEMD) method and the modified iterative cumulative sum of squares (ICSS) algorithm to decompose the index returns and identify the structural breakpoints. Furthermore, it combines the least-square support vector machine approach with the particle swarm optimization method (PSO-LSSVM) and the generalized autoregressive conditional heteroskedasticity (GARCH) type models to construct innovative hybrid forecasting methods. On the one hand, the empirical results indicate that the AIRO index returns have complex structural characteristics, and present time-varying and nonlinear characteristics with high complexity and mutability; on the other hand, the newly proposed hybrid forecasting method (i.e., the EEMD-PSO-LSSVM-ICSS-GARCH models) which considers these complex structural characteristics, can yield the optimal forecasting performance for the AIRO index returns.

Machine learning approach for estimating the human-related VOC emissions in a university classroom.

Indoor air quality becomes increasingly important, partly because the COVID-19 pandemic increases the time people spend indoors. Research into the prediction of indoor volatile organic compounds (VOCs) is traditionally confined to building materials and furniture. Relatively little research focuses on estimation of human-related VOCs, which have been shown to contribute significantly to indoor air quality, especially in densely-occupied environments. This study applies a machine learning approach to accurately estimate the human-related VOC emissions in a university classroom. The time-resolved concentrations of two typical human-related (ozone-related) VOCs in the classroom over a five-day period were analyzed, i.e., 6-methyl-5-hepten-2-one (6-MHO), 4-oxopentanal (4-OPA). By comparing the results for 6-MHO concentration predicted via five machine learning approaches including the random forest regression (RFR), adaptive boosting (Adaboost), gradient boosting regression tree (GBRT), extreme gradient boosting (XGboost), and least squares support vector machine (LSSVM), we find that the LSSVM approach achieves the best performance, by using multi-feature parameters (number of occupants, ozone concentration, temperature, relative humidity) as the input. The LSSVM approach is then used to predict the 4-OPA concentration, with mean absolute percentage error (MAPE) less than 5%, indicating high accuracy. By combining the LSSVM with a kernel density estimation (KDE) method, we further establish an interval prediction model, which can provide uncertainty information and viable option for decision-makers. The machine learning approach in this study can easily incorporate the impact of various factors on VOC emission behaviors, making it especially suitable for concentration prediction and exposure assessment in realistic indoor settings.

Comprehensive Modeling in Predicting Liquid Density of the Refrigerant Systems Using Least-Squares Support Vector Machine Approach

A robust machine learning algorithm known as the least-squares support vector machine (LSSVM) model was used to predict the liquid densities of 48 different refrigerant systems. Hence, a massive dataset was gathered using the reports published previously. The proposed model was evaluated via various analyses. Based on the statistical analysis results, the actual values predicted by this model have high accuracy, and the calculated values of RMSE, MRE, STD, and R2 were 0.0116, 0.158, 0.1070, and 0.999, respectively. Moreover, sensitivity analysis was done on the efficient input parameters, and it was found that CF2H2 has the most positive effect on the output parameter (with a relevancy factor of +50.19). Furthermore, for checking the real data accuracy, the technique of leverage was considered, the results of which revealed that most of the considered data are reliable. The power and accuracy of this simple model in predicting liquid densities of different refrigerant systems are high; therefore, it is an appropriate alternative for laboratory data.

Estimation of catalytic hydrogen production through water-gas shift reaction using a machine learning technique

ABSTRACT Hydrogen (H2) is an important and environmentally friendly energy source and the need for its usage is growing across the world. Water-Gas Shift (WGS) reaction is the key approach for hydrogen production which uses different catalysts with respect to conditions at which the reaction occurs. This research examines the applicability of a machine learning technique called Least Square Support Vector Machine (LSSVM) to predict the conversion of carbon monoxide (CO) in WGS reactions according to various compositions for active phase and different kinds of support compounds for catalysts. The implemented method considers the intrinsic catalyst variables to predict the performance of the reaction by using variables such as surface area, calcination time and temperature. The outcomes indicate that the LSSVM approach can precisely estimate the real CO conversion information with overall R2, AARD%, and RMSE values of 0.9996, 2.55, and 0.0069, respectively. This research shows the reliable performance of machine learning approaches such as the LSSVM technique for predicting better catalysts and process conditions for this important reaction which is of great importance in researches related to the environmental field.

Robust intelligent topology for estimation of heat capacity of biochar pyrolysis residues

Biochar is recognized as a promising spectrum for thermal science applications considering environmental protection. In this study, different artificial intelligence (AI) techniques are employed to determine the heat capacity of biochar derived from pyrolysis of different biomass sources as a function of pyrolysis temperature, post-treatments process, and temperature. Relatively substantial experimental measurements are employed for adjusting the hyper-parameters of the considered AI methodologies. Accurate measurement using seven statistical criteria confirmed that the least-squares support vector machine (LS-SVM) with the Gaussian-type kernel is the best paradigm for the considered task. Statistical analysis shows an excellent capacity of the developed LS-SVM approach for estimation of heat capacity of biochar, i.e., mean square error (MSE) of 6.82, absolute average relative deviation (AARD) of 0.15%, and regression coefficient (R2) of one. Furthermore, the LS-SVM method accuracy is more than 700 times better than the developed empirical correlation in literature.

Determination of biodiesel purity through feature mapping to the multi-dimensional space by the LS-SVM approach

Purity is one of the essential properties of biodiesel. Since the purity parameter depends on different operating conditions, its direct measurement is too hard and can only be obtained for specific ranges of conditions. Therefore, this work considers the least-squares support vector machines (LS-SVMs) that transform operating conditions to a multi-dimensional space to simulate biodiesel purity in wide ranges of operating conditions. Indeed, we develop a reliable LS-SVM approach for modeling the biodiesel purity as a function of catalyst type and its concentration, reaction time, temperature, methanol-to-oil volume ratio, frequency, and amplitude of ultrasonic waves. The designed LS-SVM’s predictive performance is compared with four available artificial intelligence (AI) techniques in reliable literature. The obtained results confirm that the LS-SVM paradigm outperforms other considered AI-based techniques regarding five different statistical criteria. Our LS-SVM model provides AARD = 2.2%, RMSE = 3.46, and R2 = 0.9868 for the prediction of 267 experimental data points, which includes 267 data points. This model is finally employed for investigating the effect of different influential variables on biodiesel purity.

Modeling stability conditions of methane Clathrate hydrate in ionic liquid aqueous solutions

Mixing ionic liquid (IL) and water can influence the thermodynamic characteristics such as water activity. Hence, some ILs can be utilized as inhibitors for hydrate systems. This work was intended to determine the three-phase equilibria of methane hydrate in aqueous solutions of 32 ILs using the least squares support vector machine (LSSVM), adaptive neuro-fuzzy inference system (ANFIS), and classification and regression tree (CART). The modeling studies on clathrate hydrates in aqueous solutions of ILs are also reviewed. The used databank contains anion groups such as sulphate, dicyanamide, tetrafluoroborate, and halides. The dissociation temperature of methane hydrate in aqueous solutions of ILs is modelled considering the ILs' critical pressure and critical temperature, the pressure of hydrate system, and the concentration of IL in aqueous phase as the independent parameters. All the developed models are found to well represent/predict the methane hydrate dissociation temperatures in aqueous solutions of ILs. The calculated values of average absolute relative deviation for the presented LSSVM, ANFIS, and CART models are equal to 0.08, 0.31, and 0.10, respectively. Hence, the results reveal that the introduced CART and ANFIS models cannot compete with the developed LSSVM approach in representing the dissociation temperatures of the methane + hydrate + IL + water systems studied in this research work. Using the proposed LSSVM model, all the investigated database equilibrium temperatures are within the absolute deviation percent of 0.0-1.0% except the data that show an absolute deviation percent of 1.35%. The findings of this study can help researchers and engineers to better understand the effects of ILs on methane hydrate stability conditions toward effective hydrate management.

Machine learning-based models for predicting permeability impairment due to scale deposition

Water injection is one of the robust techniques to maintain the reservoir pressure and produce trapped oil from oil reservoirs and improve an oil recovery factor. However, incompatibility between injected water and reservoir water causes an unflavored issue named “scale deposition.” Owing to the deposited scales, effective permeability of a reservoir reduced, and pore throats might be plugged. To determine formation damage owing to scale deposition during a water injection process, two well-known machine learning methods, least squares support vector machine (LSSVM) and artificial neural network (ANN), are employed in the present paper. To improve the performance of the LSSVM method, a metaheuristic optimization algorithm, genetic algorithm (GA), is used. The constructed LSSVM model is examined using real formation damage data samples experimentally measured, which was reported in the literature. According to the obtained outputs of the above models, LSSVM has a high performance based on the correlation coefficient, and infinitesimal uncertainty based on a relative error between the model predictions and the corresponding actual data samples was less than 15%. Outcomes from this study indicate the useful application of the LSSVM approach in the prediction of permeability reduction due to scale deposition, and it can lead to a better and more reliable understanding of formation damage effects through water flooding without expensive laboratory measurements.

State-Space Kernelized Closed-Loop Identification of Nonlinear Systems

In this paper, we propose a non-parametric state-space identification approach for open-loop and closed-loop discrete-time nonlinear systems with multiple inputs and multiple outputs. Employing a least squares support vector machine (LS-SVM) approach in a reproducing kernel Hilbert space framework, a nonlinear auto-regressive model with exogenous terms is identified to provide a non-parametric estimate of the innovation noise sequence. Subsequently, this estimate is used to obtain a compatible non-parametric estimate of the state sequence in an unknown basis using kernel canonical correlation analysis. Finally, the estimate of the state sequence is used together with the estimated innovation noise sequence to find a non-parametric state-space model, again using a LS-SVM approach. The performance of the approach is analyzed in a simulation study with a nonlinear system operating both in open loop and closed loop. The identification approach can be viewed as a nonlinear counterpart of consistent subspace identification techniques for linear time-invariant systems operating in closed loop.

A Kernel Principal Component Regressor for LPV System Identification

This article describes a Kernel Principal Component Regressor (KPCR) to identify Auto Regressive eXogenous (ARX) Linear Parmeter Varying (LPV) models. The new method differs from the Least Squares Support Vector Machines (LS-SVM) algorithm in the regularisa-tion of the Least Squares (LS) problem, since the KPCR only keeps the principal components of the Gram matrix while LS-SVM performs the inversion of the same matrix after adding a regularisation factor. Also, in this new approach, the LS problem is formulated in the primal space but it ends up being solved in the dual space overcoming the fact that the regressors are unknown.The method is assessed and compared to the LS-SVM approach through 2 Monte Carlo (MC) experiments. Every experiment consists of 100 runs of a simulated example, and a different noise level is used in each experiment, with Signal to Noise Ratios of 20db and 10db, respectively. The obtained results are twofold, first the performance of the new method is comparable to the LS-SVM, for both noise levels, although the required calculations are much faster for the KPCR. Second, this new method reduces the dimension of the primal space and may convey a way of knowing the number of basis functions required in the Kernel. Furthermore, having a structure very similar to LS-SVM makes it possible to use this method in other types of models, e.g. the LPV state-space model identification.

Sensitivity analysis and application of machine learning methods to predict the heat transfer performance of CNT/water nanofluid flows through coils

Nowadays, nanofluids are broadly utilized for various engineering and industrial systems including heat exchangers, power plants, air-conditioning, etc. The helically coiled tube heat exchangers are of the most interesting and efficient kinds of heat exchangers. The current study has focused on proposing model to predict Nusselt number by considering Prandtl number, volumetric concentration, and helical number of helically coiled heat exchanger as input variables. The investigated heat exchanger utilizes water carbon nanofluid. To propose an accurate model, a multilayer perceptron artificial neural network (MLP-ANN), adaptive neuro-fuzzy inference system (ANFIS), and least squares support vector machine (LSSVM) models are used. 72 experimental data are utilized as input data. Results indicate that LSSVM approach has the best performance and the proposed model by this approach has R-squared value equals to 1.

Applying of LSSVM approach as a novel tool for accurate prediction of asphaltene inhibition efficiency

ABSTRACTAsphaltene precipitation is one of critical problems for petroleum industries. There are different methods for inhibition of asphaltene precipitation. One of the common and effective methods for inhibition of asphaltene precipitation is utilizing asphaltene inhibitors. In this work, Least squares support vector machine (LSSVM) algorithm was coupled with simplex optimizer to create a novel and accurate tool for estimation of effect of inhibitors on asphaltene precipitation as function of concentration and structure of inhibitors and crude oil properties. To this end a total number of 75 measured data was extracted from the literature for training and testing of predicting model. The average absolute relative deviation (AARD), the coefficient of determination (R2) and root mean square error (RMSE) of total data for prediction algorithm were determined as 1.1479, 0.99406 and 0.61039. According to these parameters and graphical comparisons the LSSVM algorithm has potential to predict asphaltene precipitation in high degree of accuracy.

Forecasting Air Passengers of Changi Airport Based on Seasonal Decomposition and an LSSVM Model

This work aimed to determine a suitable method to provide air traffic passenger forecasts of Changi airport. A linear forecasting technique in the form of a seasonal autoregressive integrated moving average (SARIMA) model and a nonlinear technique known as the least squares support vector machine (LSSVM) were compared. A hybrid X-13 LSSVM approach was also compared. A fourth approach was proposed to leverage the outputs of the hybrid X-13 LSSVM method to conduct forecasts for longer forecasting horizons. Results showed that SARIMA, direct LSSVM and X-13 LSSVM methods were able to provide accurate 1-month-ahead forecasts. However, SARIMA and direct LSSVM methods both suffered from forecasting inaccuracy, as the forecasting horizon increased. The X-13 LSSVM outperformed both SARIMA and direct LSSVM methods, in terms of small magnitude errors and forecasting directional changes across the forecasting horizons. The proposed fourth approach was able to provide 24-months-ahead forecasts and was easy to implement.

Estimation of the higher heating value of biomass using proximate analysis

ABSTRACTThe higher heating value (HHV) parameter of biomass is well known for its wide application in bioenergy industry and the economical study of energy resources. In the present study, the least squares support vector machine (LSSVM) strategy is used as a novel approach to estimate HHV of biomass as a function of volatile matters (VM), fixed carbon (FC), and ash content (ASH). A total number of 350 experimental data points have been extracted from previous works to train and test the proposed algorithm. In order to judge the proposed model, the statistical parameters such as R2, RMSE, and AARD are calculated as 0.92936, 4.2731%. Based on the calculated parameters, it can be concluded that the LSSVM approach has great ability to estimate the HHV of biomass in terms of VM, FC, and ASH. Due to the fact that the experimental investigations are time-consuming and costly, the importance of this study has been highlighted.

On the estimation of viscosities and densities of CO2-loaded MDEA, MDEA + AMP, MDEA + DIPA, MDEA + MEA, and MDEA + DEA aqueous solutions

As noteworthy properties of amine aqueous solutions, the densities and viscosities of aqueous N-Methyldiethanolamine (MDEA) solutions and mixtures of MDEA with 2-Amino-2-methyl-1-propanol (AMP), Diisopropanolamine (DIPA), Monoethanolamine (MEA), and Diethanolamine (DEA) were estimated under CO2 gas loading using Adaptive Neuro-Fuzzy Inference System (ANFIS), Multi-Layer Perceptron Artificial Neural Network (MLPANN), Support Vector Machine (SVM), and Least Square Support Vector Machine (LSSVM). The density and viscosity were estimated as a function of temperature, CO2 loading, pressure, and molecular weight of mixtures. In this regard, the actual data points were collected from the literature. Genetic Algorithm (GA) was employed to determine hyper variables of the LSSVM approach and Levenberg–Marquardt algorithm was employed to optimize bias and weight values of the ANN model. In addition, Particle Swarm Optimization algorithm (PSO) was used to determine membership parameters of the ANFIS approach and related parameters of the SVM were optimized using GA. The developed tools can be of massive value for chemical engineers and chemists to have a quick check of the densities and viscosities of the aforementioned amine solutions. Results obtained from the proposed models are in satisfactory agreement with actual data. According to statistical analyses, while the obtained values of Mean Squared Error (MSE) and R-squared (R2) for estimating densities of amine solutions are 0.000011 and 0.9938, 0.000013 and 0.9937, 0.000009 and 0.9953, 0.000001 and 0.9993 for the ANN, ANFIS, SVM, and LSSVM models respectively, these values obtained 0.079 and 0.994, 0.113 and 0.9911, 0.0634 and 0.9971, 0.0079 and 0.9996 for estimating the viscosities of amine-based solutions.

Online Spatiotemporal Least-Squares Support Vector Machine Modeling Approach for Time-Varying Distributed Parameter Processes

Nonlinear and time-varying distributed parameter systems (DPSs) are challenging to accurately model due to potential spatiotemporal coupling, infinite-dimensional property, and time-varying dynamics. Although the least-squares support vector machine (LS-SVM) can effectively model lumped parameter systems, it is less effective to model the time-varying dynamics of a DPS, as it lacks the ability to incorporate time-varying spatiotemporal dynamics. Here, an online spatiotemporal LS-SVM approach is proposed to model a nonlinear and time-varying DPS. An adaptive spatial kernel function is first developed for online capture of the time-varying relationship between spatial locations. An online time coefficient model is then constructed to account for the time-varying temporal dynamics of the DPS. Combination of the adaptive spatial kernel function with the online time coefficient model allows for reconstruction of the complex DPS and ensures the model can reflect real-time spatiotemporal dynamics well. Experimen...

Rigorous modeling of CO2 equilibrium absorption in ionic liquids

Over the past few decades, solution of high amount of carbon dioxide in ionic liquids (ILs) has been the object of extensive studies. It is believed that ILs can be applied to capture CO2 and avoiding greenhouse gas emissions into the atmosphere. In this communication, the predictive capability of the Least Square Support Vector Machine (LSSVM), Adaptive Neuro-Fuzzy Inference System (ANFIS), Multi-Layer Perceptron Artificial Neural Network (MLP-ANN), and Radial Basis Function Artificial Neural Network (RBF-ANN) has been evaluated for estimating carbon dioxide solubility in 67 different ILs as a function of the operational temperature (T), pressure (P) accompanied with the properties of ILs including the critical temperature (Tc), critical pressure (Pc) and, acentric factor (ω). In this regard, an extensive data bank containing 5368 data gathered from the literature was employed. Results obtained from the LSSVM approach indicate its satisfactory predictions than other strategies. Moreover, an outlier analysis was utilized to detect suspected data points. Obtained values of R-squared (R2), Mean Squared Error (MSE) were 0.9942 & 0.00035, 0.9135 & 0.004883, 0.9135 & 0.004883, and 0.9135 & 0.004883 for the LSSVM, ANFIS, MLP-ANN, and RBF-ANN respectively. Accordingly, the LSSVM strategy was introduced as a great tool for estimating CO2 solubility in such ionic liquids, which is easy to apply and can avoid time-consuming experimental measurement and expensive experimental apparatuses as well as complicated interpretation procedures. In addition, it can help chemists and chemical engineers to have a low parameter model with satisfactory results for the estimation of CO2 solubility in ILs.

A KLD-LSSVM based computational method applied for feature ranking and classification of primary biliary cirrhosis stages

Medical datasets having variety of features can increase the complexity of decision process. Prioritising these features with respect to medical sickness is a prime task for effective assessment of patients' health. In this study, a two-step computational method based on Kullback-Leibler divergence (KLD) method and least squares support vector machine (LSSVM) is presented as an integrated model and a LSSVM-based approach is projected as an individual model. KLD was employed to rank the features and radial basis kernel function-based LSSVM approach was deployed to classify primary biliary cirrhosis (PBC) stages. Performance of several machine-learning algorithms, individually as well as in integration with KLD, was evaluated on a real-life biomedical PBC dataset. Simulation results indicated that proposed LSSVM and KLD-LSSVM-based frameworks had shown robustness to the noisy data and had outperformed other individual and integrated methods, respectively. It is concluded that the proposed methodologies can be productively applied to real-life health examination datasets containing a variety of features and multiple decision classes.

Baker’s yeast invertase purification using Aqueous Two Phase System—Modeling and optimization with PCA/LS-SVM

Least Squares-Support Vector Machine (LS-SVM) was used to predict data of Baker’s yeast invertase purification using PEG/MgSO4 Aqueous Two Phase-System (ATPS). Experiments were carried out changing the average molecular mass and percentage of PEG, pH, percentage of MgSO4 as well as of raw extract in order to observe the percentage of yield (% Yield) and Purification Factor (PF) at the bottom phase. The Principal Component Analysis (PCA) was used to eliminate the less significant input variables on the % Yield as well as on the PF. The generalization capacity evaluation for these two parameters has shown that the model generated by the LS-SVM (R2=0.974; 0.932) approach has given the best performance than partial least squares (R2=0.960; 0.926), base radial neural network (R2=0.874; 0.687) and multi-layer perceptron (R2=0.911; 0.652). Also, a bi-objective optimization has been carried out using the previously adjusted models in order to obtain a set of input data producing higher % Yield for the enzymatic activity (448.34%) as well as for the PF (8.45).