Least Squares Support Vector Regression Model Research Articles

Real-time product quality control for batch processes based on stacked least-squares support vector regression models

Abstract A novel real-time final product quality control method for batch operations based on stacked least-squares support vector regression models (stacked LSSVR) is proposed. It combines midcourse correction (MCC) and batch-to-batch control. To enhance the model prediction accuracy and generalization capability, a stacked LSSVR approach is presented. Quality control is achieved by predicting the final product quality using stacked LSSVR models and adjusting process variables at some pre-specified decision points. Then a decision is made on whether or not control action is taken at every decision point. Once the control action is expected, the manipulated variable values are calculated and the control action is taken to bring the off-spec product quality back to the target. Then a batch-to-batch control is used to overcome the model plant mismatches and unmeasured disturbances. At last, the proposed modeling and quality control strategy is illustrated on a simulated batch reactor.

Product Form Feature Selection for Mobile Phone Design Using LS-SVR and ARD

In the product design field, it is important to pin point critical product form features (PFFs) that influence consumers' affective responses (CARs) of a product design. In this paper, an approach based on least squares support vector regression (LS-SVR) and automatic relevance determination (ARD) is proposed to streamline the task of product form feature selection (PFFS) according to the CAR data. The representation of PFFs is determined by morphological analysis and pairwise adjectives are used to express CARs. In order to gather the CAR data, an experiment of semantic differential (SD) evaluation on collected product samples was conducted. The LS-SVR prediction model can be constructed using the PFFs as input data and the evaluated SD scores as output value. The optimal parameters of the LS-SVR model are tuned by using Bayesian inference. Finally, an ARD selection process is used to analyze the relative relevance of PFFs to obtain feature ranking.

Runway Incursion Event Forecast Model Based on LS-SVR with Multi-kernel

Forecasting of runway incursion event is very significant to guide the job of civil aviation safety management. It is an important part of the runway incursion early warning management. However, prediction of runway incursion event is a complicated problem due to its non-linearity and the small quantity of training data. As a novel type of learning machine, support vector machine (SVM) has been gaining popularity due to their promising performance, such as dealing with the data of small sample, the high dimension and the excellent generalization ability. However, the generalization ability of SVM often relies on whether the selected kernel function is suitable for real data. To lessen the sensitivity of different kernels and improve generalization ability, least square support vector regression (LS-SVR) with multi-kernel is proposed to forecast the runway incursion event in this paper. The two experimental results indicate that LS-SVR with multi-kernel model is better than LS-SVR with individual kernel model and generalized regression neural network (GRNN) model. Consequently, multi-kernel LS-SVR model is a proper alternative for forecasting of the runway incursion event.

Time-series gas prediction model using LS-SVR within a Bayesian framework

The traditional least squares support vector regression (LS-SVR) model, using cross validation to determine the regularization parameter and kernel parameter, is time-consuming. We propose a Bayesian evidence framework to infer the LS-SVR model parameters. Three levels Bayesian inferences are used to determine the model parameters, regularization hyper-parameters and tune the nuclear parameters by model comparison. On this basis, we established Bayesian LS-SVR time-series gas forecasting models and provide steps for the algorithm. The gas outburst data of a Hebi 10th mine working face is used to validate the model. The optimal embedding dimension and delay time of the time series were obtained by the smallest differential entropy method. Finally, within a MATLAB7.1 environment, we used actual coal gas data to compare the traditional LS-SVR and the Bayesian LS-SVR with LS-SVMlab1.5 Toolbox simulation. The results show that the Bayesian framework of an LS-SVR significantly improves the speed and accuracy of the forecast.

Noise reduction and drift removal using least-squares support vector regression with the implicit bias term

Random noise and drifts often degrade the quality of the seismic data and harm further processing. The support vector machine (SVM) is a universal machine learning method based on the statistical learning theory, which has been widely used in classification and regression. On the basis of the least squares support vector regression (LS-SVR), we propose the least squares support vector regression with implicit bias term (ILS-SVR) to reduce the random noise and remove the drifts in seismic data. The proposed method stacks the weight vector and the bias term into a new vector, and incorporates the bias term into the objective function of the optimization problem in the LS-SVR model. As a result, for the translation invariant kernel we obtain a simpler solution that can be computed more efficiently than the LS-SVR. More importantly, the ILS-SVR with a Ricker wavelet kernel can not only effectively suppress the random noise, but also remove drifts in seismic data. Behind the proposed method, the implicit bias term and the special choice of the kernel (such as the Ricker wavelet kernel) are two key ingredients to remove drifts. We test the proposed method on both synthetic and real seismic data, and compare it with the ILS-SVR using the popular radial basis function (RBF) kernel, the [Formula: see text] prediction filtering, and the wavelet transform-based method. Our experimental results illustrate that the proposed algorithm achieves better performance over the other three algorithms.

Improvement of Near-Infrared Spectral Calibration Models for Brix Prediction in ‘Gannan’ Navel Oranges by a Portable Near-Infrared Device

A portable near-infrared (NIR) device was developed to nondestructively predict Brix value in intact ‘Gannan’ navel oranges. This research focused on developing calibration models which were less disturbed by the challenges of portable applications. The spectra of 150 samples were collected in the wavelength range of 820–950 nm. Wavelet transformed (WT) was applied to compress the raw data for improving the optimization efficiency. Classical linear partial least squares regression and nonlinear least squares support vector regression (LSSVR) were applied to building calibration models. By comparison, both prediction precision and optimization efficiency of the compressed regression models were improved. The LSSVR models outperformed the PLS models with higher accuracy and lower error. LSSVR combined with WT compression (WT–LSSVR) produced the best correlation coefficient value (r) and the root mean squared error of prediction of 0.918 and 0.321 oBrix. Based on these results, WT–LSSVR is to be a promising method to improve precision and optimization efficiency of NIR spectral calibration models for Brix prediction in ‘Gannan’ navel oranges by the portable near-infrared device.

Simultaneous chemiluminescence determination of amoxicillin and clavulanic acid using least squares support vector regression

In this work, both the batch and flow injection chemiluminescence (CL) methods have been proposed for the simultaneous determination of two structurally similar β-lactams including amoxicillin and clavulanic acid (CLAV). Chemiluminescence spectral overlap of these two structurally similar β-lactams is the main limitation for the simultaneous analysis of the two compounds. Least squares support vector regression (LS-SVR) was applied to relate concentration of both the compounds to their CL profiles. The parameters of the model, consisting of kernel parameter, σ 2, and the regularization parameter, γ, were optimized by constructing different LS-SVR models and the model with the minimum root mean squared error of cross-validation (RMSECV) for the calibration set was selected as the best model and its parameters were chosen as the optimized values. The performance of LS-SVR model was compared with Partial Least Squares (PLS) and the results revealed the superiority of the LS-SVR over PLS model. Under the optimized experimental conditions for both the compounds, when LS-SVR was applied, the detection limits obtained were 0.2 and 0.60 μmol L −1 in the batch mode and 0.3 and 0.5 μmol L −1 in the flow injection mode for CLAV and amoxicillin, respectively. The proposed method was utilized for the simultaneous determination of the compounds in pharmaceutical formulations and spiked plasma samples.

A hybrid slantlet denoising least squares support vector regression model for exchange rate prediction

Abstract Despite the active exploration of linear and nonlinear modeling of exchange rates, there is no consensus on the optimal forecasting model other than the traditional random walk and ARMA benchmark models in the literature. Given the increasing recognition of heterogeneous market structure, this paper proposes an alternative Slantlet denoising based hybrid methodology that attempts to incorporate the linear and nonlinear data features. The recently emerging Slantlet analysis is introduced to separate the linear data features as it constructs filters with varying lengths at different scales and has more appealing time localization features than the normal wavelet analysis. Meanwhile, the Least Squares Support Vector Regression (LSSVR) is used to model and correct for the empirical errors nonlinear in nature. As empirical studies were conducted in the Euro exchange rate market, the performance of the proposed algorithm was compared with those of benchmark models including random walk, ARMA and LSSVR models. The proposed algorithm outperforms the benchmark models. More importantly the proposed methodology explores and offers deeper insights as to the underlying data generating process.

Application of QSPR for prediction of percent conversion of esterification reactions in supercritical carbon dioxide using least squares support vector regression

A quantitative structure property relationship (QSPR) model was conducted for the first time prediction of the percent conversion of esterification reactions in supercritical carbon dioxide (scCO 2). The data set consisting of 42 esterification reactions of different acids and alcohols in scCO 2 using different temperatures and pressures. The percent conversion was related to the most feasible descriptors such as temperature, pressure, dipole moment and the highest occupied molecular orbital (HOMO) of the alcohols, dipole moment, the lowest unoccupied molecular orbital (LUMO) and heat of formation of the acids. These descriptors were related to the percent conversion of the esterification reactions using least squares support vector regression (LS-SVR) approach. The parameters of the LS-SVR consisting of the regularization and the kernel, were optimized by constructing models with all the possible combinations of these two parameters. The model with the minimum root mean squared error of cross-validation (RMSECV) for the calibration set was selected as the best model and the parameters of this model were selected as the optimized values. Both internal and external validations were performed to validate the performance of the model. The results revealed that the calculated conversion values are in good agreement with the experimental ones, and the performance of the LS-SVR model was superior to the multiple linear regression (MLR) ones.

Particle Swarm Optimization-based LS-SVM for Building Cooling Load Prediction

Accurate predicting of building cooling load has been one of the most important issues in the energy-saving building, which provides an approach to integrate and optimize the heating, ventilating, and air-conditioning (HVAC) system cooling supply system efficiently. Because of the remarkable nonlinear mapping capabilities of forecasting, artificial neural networks have played a crucial role in forecasting building cooling load, but suffer from the phenomena of local minimum and over-fitting. This paper investigates the feasibility of using Least Squares Support vector regression (LS-SVR) to forecast building cooling load. LS-SVR is a novel type of learning machine, which has been successfully employed to solve nonlinear regression and time series problems. Due to the importance of parameters optimization in LS-SVR model, particle swarm optimization (PSO) was used to optimize the model parameters. The experiment results show that PSO can quickly obtain the optimal parameters satisfying the precision requirement with a simple calculation, which solves the problem of complex calculation and empiricism in conventional methods. The evaluation on the testing cases shows the SVR model with PSO has a good generalization performance and can be a promising alternative for building cooling load prediction.

Support vector regression and least squares support vector regression for hormetic dose–response curves fitting

Accurate description of hormetic dose–response curves (DRC) is a key step for the determination of the efficacy and hazards of the pollutants with the hormetic phenomenon. This study tries to use support vector regression (SVR) and least squares support vector regression (LS-SVR) to address the problem of curve fitting existing in hormesis. The SVR and LS-SVR, which are entirely different from the non-linear fitting methods used to describe hormetic effects based on large sample, are at present only optimum methods based on small sample often encountered in the experimental toxicology. The tuning parameters ( C and p1 for SVR, gam and sig2 for LS-SVR) determining SVR and LS-SVR models were obtained by both the internal and external validation of the models. The internal validation was performed by using leave-one-out (LOO) cross-validation and the external validation was performed by splitting the whole data set (12 data points) into the same size (six data points) of training set and test set. The results show that SVR and LS-SVR can accurately describe not only for the hermetic J-shaped DRC of seven water-soluble organic solvents consisting of acetonitrile, methanol, ethanol, acetone, ether, tetrahydrofuran, and isopropanol, but also for the classical sigmoid DRC of six pesticides including simetryn, prometon, bromacil, velpar, diquat-dibromide monohydrate, and dichlorvos.

Predicting the Parts Weight in Plastic Injection Molding Using Least Squares Support Vector Regression

To achieve the desired quality in plastic injection molding, advanced monitoring techniques are often recommended in the workshop. Unfortunately, the signal in plastic injection modeling process such as nozzle pressure that is relevant to part quality is not easy to obtain because of the cost of sensors. The sensor-based modeling idea is therefore adopted. In this paper, a new method for predicting the parts weight in plastic injection molding using least squares support vector regression (LS-SVR) is proposed, which is composed of two steps. The first step is to estimate the nozzle pressure with the hydraulic system pressure using an LS-SVR model. The second step is to predict product weight using the estimated nozzle pressure, which is done using another LS-SVR model. The experimental results show that the new method is very effective.

A consensus least squares support vector regression (LS-SVR) for analysis of near-infrared spectra of plant samples

Consensus modeling of combining the results of multiple independent models to produce a single prediction avoids the instability of single model. Based on the principle of consensus modeling, a consensus least squares support vector regression (LS-SVR) method for calibrating the near-infrared (NIR) spectra was proposed. In the proposed approach, NIR spectra of plant samples were firstly preprocessed using discrete wavelet transform (DWT) for filtering the spectral background and noise, then, consensus LS-SVR technique was used for building the calibration model. With an optimization of the parameters involved in the modeling, a satisfied model was achieved for predicting the content of reducing sugar in plant samples. The predicted results show that consensus LS-SVR model is more robust and reliable than the conventional partial least squares (PLS) and LS-SVR methods.