Nonlinear Least-squares Support Vector Machine Research Articles

Evaluation of least square support vector machine, generalized regression neural network and response surface methodology in modeling the removal of Levofloxacin and Ciprofloxacin from aqueous solutions using ionic liquid @Graphene oxide@ ionic liquid NC

This study focused on the preparation, characterization of a supported dicationic ionic liquid nanocomposite (IL@GO@IL NC) and employing of it in the adsorption of Levofloxacin (LVX) and Ciprofloxacin (CPF) antibiotic. Also, the nonlinear least square support vector machine (N-LSSVM), generalized regression neural network (GRNN), along with response surface methodology used for modeling the removal of both antibiotics (i.e. LVX and CPF) from aqueous solution. In this work, the experiments designed based on critical parameters such as adsorbent dose, antibiotic concentration, sonication time and temperature. The nonlinear adsorption isotherms and kinetics models as well as thermodynamic used for fitting the experimental data. The statistical results demonstrated that the LSSVM and GRNN model efficiently predicted the antibiotics removal percentage with very high accuracy, respectively. Furthermore, for an optimal adsorption of LVX and CPF by IL@GO@IL NC, the adsorbent dose, antibiotic concentration, temperature and sonication time should be set to 0.23 and 0.35 g, 95.20 and 100.00 mg L-1, 2.10 and 4.60 min, and 45.00 ℃, for LVX and CPF, respectively. Under these optimal conditions, the removal percentages and adsorption capacity of LVX and CPF will be 98.20 and 97.50%, as well as 730 and 630 mg g−1, respectively. Besides, the IL@GO@IL NC efficiency under optimal conditions drops up by 15% after seven cycles for both antibiotics. Therefore, the IL@GO@IL NC may be considered as a promising option in removing fluoroquinolones antibiotics from water.

Nonlinear joint PP-PS AVO inversion based on improved Bayesian inference and LSSVM

Multiwave seismic technology promotes the application of joint PP-PS amplitude versus offset (AVO) inversion; however conventional joint PP-PS AVO inversioan is linear based on approximations of the Zoeppritz equations for multiple iterations. Therefore the inversion results of P-wave, S-wave velocity and density exhibit low precision in the faroffset; thus, the joint PP-PS AVO inversion is nonlinear. Herein, we propose a nonlinear joint inversion method based on exact Zoeppritz equations that combines improved Bayesian inference and a least squares support vector machine (LSSVM) to solve the nonlinear inversion problem. The initial parameters of Bayesian inference are optimized via particle swarm optimization (PSO). In improved Bayesian inference, the optimal parameter of the LSSVM is obtained by maximizing the posterior probability of the hyperparameters, thus improving the learning and generalization abilities of LSSVM. Then, an optimal nonlinear LSSVM model that defines the relationship between seismic reflection amplitude and elastic parameters is established to improve the precision of the joint PP-PS AVO inversion. Further, the nonlinear problem of joint inversion can be solved through a single training of the nonlinear inversion model. The results of the synthetic data suggest that the precision of the estimated parameters is higher than that obtained via Bayesian linear inversion with PP-wave data and via approximations of the Zoeppritz equations. In addition, results using synthetic data with added noise show that the proposed method has superior anti-noising properties. Real-world application shows the feasibility and superiority of the proposed method, as compared with Bayesian linear inversion.

Identification of oil, sugar and crude fiber during tobacco (Nicotiana tabacum L.) seed development based on near infrared spectroscopy

Tobacco seeds are a potential feedstock for biofuels. To insure make full use of tobacco seed biomass, the present study was carried out to estimate seed oil, sugar and crude fiber during seed development through near infrared spectroscopy (NIRS) nondestructive determination. Four pre-processing methods, Savitzky-Goly after standard normal variate (SNV-SG), first Savitzky-Goly derivative after standard normal variate (SNV-SG-1stD), Savitzky-Goly after multiplicative scatter correction (MSC-SG) and first Savitzky-Goly derivative after multiplicative scatter correction (MSC-SG-1stD), were respectively performed to optimize the original spectra before establishment of the calibration models. Then linear partial least squares (PLS) and nonlinear least-squares support vector machine (LS-SVM) methods were utilized to develop the calibration models, in which the LS-SVM models were found to have better performance than PLS models. The best LS-SVM models of oil, sugar and crude fiber were established after pre-processed by MSC-SG-1st D. These results indicated that NIRS was suitable to rapidly and accurately analyze tobacco seed composition.

A novel hybrid algorithm for electricity price and load forecasting in smart grids with demand-side management

Smart grid is a platform that enables the participants of electricity market to adjust their bidding strategies based on Demand-Side Management (DSM) models. Responsiveness of the market participants can improve reliability of system operation as well as capital cost investments. In this regard, the accurate forecast of electricity price and demand in smart grids is an important challenge as their strong correlation makes a separate forecasting to be ineffective. Therefore, this paper proposes a novel hybrid algorithm for simultaneous forecast of price and demand that uses a set of effective tools in preprocessing part, forecast engine and tuned algorithm. To highlight our contributions, the proposed forecast algorithm classified into three main parts. The first part employs a new Flexible Wavelet Packet Transform (FWPT) to decompose a signal into multiple terms at different frequencies, and a new feature selection method that employs Conditional Mutual Information (CMI) and adjacent features in order to select valuable input data. The second part consists of a novel Multi-Input Multi-Output (MIMO) model based on Nonlinear Least Square Support Vector Machine (NLSSVM) and Autoregressive Integrated Moving Average (ARIMA) in order to model the linear and nonlinear correlation between price and load in two stages. The final part employs a modified version of Artificial Bee Colony (ABC) algorithm based on time-varying coefficients and stumble generation operator, called TV-SABC, in order to optimize NLSSVM parameters in a learning process. The proposed hybrid forecasting algorithm is evaluated on several real and well-known markets illustrating its high accuracy in simultaneous forecast of electricity price and demand. Moreover, the interactive effects of demand-side management programs on load factor (load curve) and price signal are investigated by numerical indices.

In situ measurement of some soil properties in paddy soil using visible and near-infrared spectroscopy.

In situ measurements with visible and near-infrared spectroscopy (vis-NIR) provide an efficient way for acquiring soil information of paddy soils in the short time gap between the harvest and following rotation. The aim of this study was to evaluate its feasibility to predict a series of soil properties including organic matter (OM), organic carbon (OC), total nitrogen (TN), available nitrogen (AN), available phosphorus (AP), available potassium (AK) and pH of paddy soils in Zhejiang province, China. Firstly, the linear partial least squares regression (PLSR) was performed on the in situ spectra and the predictions were compared to those with laboratory-based recorded spectra. Then, the non-linear least-square support vector machine (LS-SVM) algorithm was carried out aiming to extract more useful information from the in situ spectra and improve predictions. Results show that in terms of OC, OM, TN, AN and pH, (i) the predictions were worse using in situ spectra compared to laboratory-based spectra with PLSR algorithm (ii) the prediction accuracy using LS-SVM (R2>0.75, RPD>1.90) was obviously improved with in situ vis-NIR spectra compared to PLSR algorithm, and comparable or even better than results generated using laboratory-based spectra with PLSR; (iii) in terms of AP and AK, poor predictions were obtained with in situ spectra (R2<0.5, RPD<1.50) either using PLSR or LS-SVM. The results highlight the use of LS-SVM for in situ vis-NIR spectroscopic estimation of soil properties of paddy soils.

Rapid and Nondestructive Determination of Fatty acid Contents in Shell-Intact Cottonseed by Near Infrared Spectroscopy

Near infrared (NIR) spectroscopy was applied for the rapid and nondestructive determination of fatty acid contents in shell-intact crop seed. Total 385 samples of cottonseed were used in this experiment. Linear partial least squares (PLS) and nonlinear least-squares support vector machine (LS-SVM) methods with the variables selected by Monte Cario uninformative variables elimination (MC-UVE) method were used to develop the calibration models to predict the content of fatty acid contents. A best prediction performance was obtained by using the MC-UVE-LS-SVM models. The correlation coefficients (R2) of calibration models for palmitic acid, stearic acid, oleic acid, linoleic acid, saturate fatty acids, unsaturated fatty acids were 0.863, 0.881, 0.843, 0.806, 0.894 and 0.917, respectively; the residual predictive deviations (RPD) were 2.669, 2.880, 2.508, 2.202, 3.023 and 3.473, respectively. The results indicated that NIR technology could be used for the rapid quality analysis of shell-intact cottonseed avoiding the need of grinding. Furthermore, the variable selection of MC-UVE could provide more robust and accurate calibration models.

Feasibility study of near infrared spectroscopy with variable selection for non-destructive determination of quality parameters in shell-intact cottonseed

The potential of near infrared (NIR) spectroscopy for non-destructive determination of quality parameters including oil and protein contents in shell-intact cottonseed was investigated. Linear partial least squares (PLS) and nonlinear least-squares support vector machine (LS-SVM) methods were used to develop the calibration models to determinate the protein and oil contents. Moreover, as variable selection techniques, the Monte Carlo uninformative variable elimination (MC-UVE) and the successive projections algorithm (SPA) were applied to improve the predictive ability of the model. Finally, the MC-UVE–LS-SVM models show the best prediction performance. The coefficient of determination (R2), residual predictive deviation (RPD) and root mean squares error of prediction (RMSEP) were 0.959, 4.871 and 0.977 for protein, and 0.950, 4.429 and 0.834 for oil, respectively. The results indicate that NIR technology could be very useful for the rapid quality analysis of shell-intact cottonseed avoiding the need of grinding. Furthermore, the variable selection of MC-UVE can provide more robust and accurate calibration models than SPA.

Prediction method for surface finishing of spiral bevel gear tooth based on least square support vector machine

The predictive model of surface roughness of the spiral bevel gear (SBG) tooth based on the least square support vector machine (LSSVM) was proposed. A nonlinear LSSVM model with radial basis function (RBF) kernel was presented and then the experimental setup of PECF system was established. The Taguchi method was introduced to assess the effect of finishing parameters on the gear tooth surface roughness, and the training data was also obtained through experiments. The comparison between the predicted values and the experimental values under the same conditions was carried out. The results show that the predicted values are found to be approximately consistent with the experimental values. The mean absolute percent error (MAPE) is 2.43% for the surface roughness and 2.61% for the applied voltage.

Analysis of altered gait cycle duration in amyotrophic lateral sclerosis based on nonparametric probability density function estimation

Human locomotion is regulated by the central nervous system (CNS). The neurophysiological changes in the CNS due to amyotrophic lateral sclerosis (ALS) may cause altered gait cycle duration (stride interval) or other gait rhythm. This article used a statistical method to analyze the altered stride interval in patients with ALS. We first estimated the probability density functions (PDFs) of stride interval from the outlier-processed gait rhythm time series, by using the nonparametric Parzen-window approach. Based on the PDFs estimated, the mean of the left-foot stride interval and the modified Kullback–Leibler divergence (MKLD) can be computed to serve as dominant features. In the classification experiments, the least squares support vector machine (LS-SVM) with Gaussian kernels was applied to distinguish the stride patterns in ALS patients. According to the results obtained with the stride interval time series recorded from 16 healthy control subjects and 13 patients with ALS, the key findings of the present study are summarized as follows. (1) It is observed that the mean of stride interval computed based on the PDF for the left foot is correlated with that for the right foot in patients with ALS. (2) The MKLD parameter of the gait in ALS is significantly different from that in healthy controls. (3) The diagnostic performance of the nonlinear LS-SVM, evaluated by the leave-one-out cross-validation method, is superior to that obtained by the linear discriminant analysis. The LS-SVM can effectively separate the stride patterns between the groups of healthy controls and ALS patients with an overall accurate rate of 82.8% and an area of 0.869 under the receiver operating characteristic curve.

Applying Chemometrics Approaches to Model and Predict the Binding Affinities Between the Human Amphiphysin SH3 Domain and Its Peptide Ligands (Suplementary Material)

In this study, we used two categories of molecular descriptors as CODESSA and DPPS (divided physicochemical property scores of amino acids) to parameterize structural characteristics of 2015 human amphiphysin SH3 domain-binding decapeptides at atom and residue levels. Based upon that, several robust quantitative structure-affinity relationship (QSAR) models were then constructed using partial least squares regression (PLS) and least squares-support vector machine (LSSVM) coupled with genetic algorithm (GA)-variable selection. Results show that (1) GA is a powerful tool for variable selection by which the most informative variable combinations can be efficiently determined for PLS and LSSVM modeling, (2) regression models constructed using nonlinear LSSVM approach are more robust and predictable than those by linear PLS method, (3) the residue level descriptor (DPPS) performs better in capturing peptide structural characteristics, more amenable than those from the atom level descriptor (CODESSA). By investigating the optimal DPPS-based GA-LSSVM model, it is indicated that the core motif of SH3 domain-binding peptides contributes significantly to the binding affinity, whereas the two end residues, especially the N-terminal residue, have a little effect on the binding process.

Prediction of Ion Drift Times for a Proteome‐Wide Peptide Set Using Partial Least Squares Regression, Least‐Squares Support Vector Machine and Gaussian Process

AbstractQuantitative structure‐property relationships (QSPRs) have been developed to predict the ion mobility spectrometry (IMS) drift time tD for a set of 1481 peptides generated by protease digestion of the Drosophila melanogaster proteome using information directly derived from molecular structures. The relationship between peptide structure and the drift time tD was constructed by using partial least squares regression (PLS), least‐squares support vector machine (LSSVM) and Gaussian process (GP) coupled with genetic algorithm‐variable selection. Among these models, the linear PLS was incapable of capturing all dependences in this peptide system; nonlinear LSSVM and GP methods presented a good statistical performance on reproducing peptide mobility behavior. Moreover, since GP was able to handling both linear and nonlinear‐hybrid relationship, it gave a stronger fitting ability and a better predictive power than the LSSVM. Systematic analysis of the GP model showed that diversified properties contribute remarkable effect to the relationship between the drift time and the peptide structure. Particularly, the structural topological information and charge distribution contribute significantly to the drift time of peptides.

Predicting liquid chromatographic retention times of peptides from the Drosophila melanogaster proteome by machine learning approaches

Three machine learning algorithms as least-squares support vector machine (LSSVM), random forest (RF) and Gaussian process (GP) were used to model the quantitative structure–retention relationship (QSRR) for predicting and explaining the retention behavior of proteome-wide peptides in the reverse-phase liquid chromatography. Peptides were parameterized using CODESSA approach and 145 descriptors were obtained for each peptide, including diverse structural information such as constitutional, topological, geometrical and physicochemical property. Based upon that, the nonlinear LSSVM, RF and GP as well as another sophisticated linear method (partial least-squares regression (PLS)) were employed in the QSRR model development. By a series of systematic validations as internal cross-validation, external test and Monte Carlo cross-validation, the stability and predictive power of the constructed models were confirmed. Results show that regression models developed using nonlinear approaches such as LSSVM, RF and GP predict better than linear PLS models. Considering the retention times used in this work were measured in different columns and thus have a relatively large uncertainty (reproducibility within 7%), the optimal statistics obtained from GP modeling are satisfactory, with the coefficients of determination ( R 2) for training set and test set of 0.894 and 0.866, respectively.

Identification of Wiener-Hammerstein Systems using LS-SVMs

This paper extends the overparametrization technique as used for Hammerstein systems employing nonlinear Least-Squares Support Vector Machines (LS-SVMs) towards the identification of Wiener-Hammerstein systems. We present some practical guidelines as well as empirical results on the performance of the method with respect to various deficiencies of the excitation signal. Finally we apply our method to the SYSID2009 Wiener-Hammerstein benchmark data set.

Classification of Activated Sludge Settleability Using Linear and Nonlinear Classification Functions

In this paper, two classifiers are proposed to distinguish between bulking and non-bulking situations in an activated sludge wastewater treatment plant, based on available image analysis information. The first classifier consists of a simple linear classification function, while the second classifier uses a highly nonlinear least squares support vector machine (LS-SVM) to distinguish between both situations. It is shown that the nonlinear LS-SVM classification function outperforms the linear classifier. Both exhibit identical misclassification rates, but fewer samples are located in the uncertainty area when using the nonlinear classifier. However, this better classification performance requires the identification of a substantial amount of model parameters, while the linear classifier is, except for the threshold values, parameterless.

Generalized Predictive Control with Online Least Squares Support Vector Machines

This paper proposes a practical generalized predictive control (GPC) algorithm based on online least squares support vector machines (LS-SVM) which can deal with nonlinear systems effectively. At each sampling period the algorithm recursively modifies the model by adding a new data pair and deleting the least important one out of the consideration on realtime property. The data pair deleted is determined by the absolute value of lagrange multiplier from last sampling period. The paper gives the recursive algorithm of model parameters when adding a new data pair and deleting an existent one, respectively, and thus the inversion of a large matrix is avoided and the memory can be controlled by the algorithm entirely. The nonlinear LS-SVM model is applied in GPC algorithm at each sampling period. The experiments of generalized predictive control on pH neutralizing process show the effectiveness and practicality of the proposed algorithm.

Structure–activity relationship study of oxindole-based inhibitors of cyclin-dependent kinases based on least-squares support vector machines

The least-squares support vector machines (LS-SVMs), as an effective modified algorithm of support vector machine, was used to build structure–activity relationship (SAR) models to classify the oxindole-based inhibitors of cyclin-dependent kinases (CDKs) based on their activity. Each compound was depicted by the structural descriptors that encode constitutional, topological, geometrical, electrostatic and quantum-chemical features. The forward-step-wise linear discriminate analysis method was used to search the descriptor space and select the structural descriptors responsible for activity. The linear discriminant analysis (LDA) and nonlinear LS-SVMs method were employed to build classification models, and the best results were obtained by the LS-SVMs method with prediction accuracy of 100% on the test set and 90.91% for CDK1 and CDK2, respectively, as well as that of LDA models 95.45% and 86.36%. This paper provides an effective method to screen CDKs inhibitors.