Nonlinear Support Vector Regression Research Articles

Remaining useful life estimation based on nonlinear feature reduction and support vector regression

Prognostics and health management (PHM) of rotating machines is gaining importance in industry and allows increasing reliability and decreasing machines’ breakdowns. Bearings are one of the most components present in mechanical equipments and one of their most common failures. So, to assess machines’ degradations, fault prognostic of bearings is developed in this paper. The proposed method relies on two steps (an offline step and an online step) to track the health state and predict the remaining useful life (RUL) of the bearings. The offline step is used to learn the degradation models of the bearings whereas the online step uses these models to assess the current health state of the bearings and predict their RUL. During the offline step, vibration signals acquired on the bearings are processed to extract features, which are then exploited to learn models that represent the evolution of the degradations. For this purpose, the isometric feature mapping reduction technique (ISOMAP) and support vector regression (SVR) are used.The method is applied on a laboratory experimental degradations related to bearings. The obtained results show that the method can effectively model the evolution of the degradations and predict the RUL of the bearings.

Combined shifted-excitation Raman difference spectroscopy and support vector regression for monitoring the algal production of complex polysaccharides

The applicability of shifted-excitation Raman difference spectroscopy (SERDS) in combination with signal regression analysis as an alternative and non-invasive approach for monitoring the cultivation of phototrophic microorganisms producing complex molecules of pharmaceutical relevance in a bioreactor is demonstrated. As a model system, the cultivation of the red unicellular algae Porphyridium purpureum is used for focusing on the segregation of sulfated exopolysaccharides (EPS) which exhibit antiviral activity. The spectroscopic results obtained by partial linear least squares regression (PLSR) and by nonlinear support vector regression (SVR) are discussed against the corresponding results from the reference analytics based on the phenol-sulfuric acid assay. The SERDS-approach turns out to have strong potential as a non-invasive tool for online-monitoring of biotechnological processes.

Hybridizing nonlinear independent component analysis and support vector regression with particle swarm optimization for stock index forecasting

One of the major activities of financial firms and private investors is to predict future prices of stocks. However, stock index prediction is regarded as a challenging task of the prediction problem since the stock market is a complex, chaotic and nonlinear dynamic system. As stock markets are highly dynamic and exhibit wide variation, it may be more realistic and practical that assumed the stock index data are a nonlinear mixture data. In this study, a hybrid stock index prediction model by utilizing nonlinear independent component analysis (NLICA), support vector regression (SVR) and particle swarm optimization (PSO) is proposed. In the proposed model, first, the NLICA is used to deal with the nonlinearity property of the stock index data. The proposed model utilizes NLICA to extract features from the observed stock index data. The features which can be used to represent underlying/hidden information of the data are then served as the inputs of SVR to build the stock index prediction model. Finally, PSO is applied to optimize the parameters of the SVR prediction model since the parameters of SVR must be carefully selected in establishing an effective and efficient SVR model. In order to evaluate the performance of the proposed approach, the closing indexes of the Taiwan stock exchange capitalization weighted stock index, Shanghai stock exchange composite index and Bombay stock exchange index are used as illustrative examples. Experimental results showed that the proposed hybrid stock index prediction method significantly outperforms the other six comparison models. It is an efficient and effective alternative for stock index forecasting.

A Nonlinear Adaptive Beamforming Algorithm Based on Least Squares Support Vector Regression

To overcome the performance degradation in the presence of steering vector mismatches, strict restrictions on the number of available snapshots, and numerous interferences, a novel beamforming approach based on nonlinear least-square support vector regression machine (LS-SVR) is derived in this paper. In this approach, the conventional linearly constrained minimum variance cost function used by minimum variance distortionless response (MVDR) beamformer is replaced by a squared-loss function to increase robustness in complex scenarios and provide additional control over the sidelobe level. Gaussian kernels are also used to obtain better generalization capacity. This novel approach has two highlights, one is a recursive regression procedure to estimate the weight vectors on real-time, the other is a sparse model with novelty criterion to reduce the final size of the beamformer. The analysis and simulation tests show that the proposed approach offers better noise suppression capability and achieve near optimal signal-to-interference-and-noise ratio (SINR) with a low computational burden, as compared to other recently proposed robust beamforming techniques.

Integration of nonlinear independent component analysis and support vector regression for stock price forecasting

Forecasting stock prices is a major activity of financial firms and private investors. In developing a stock price forecasting model, the first step is usually feature extraction. Nonlinear independent component analysis (NLICA), a novel feature extraction technique that assumes the observed mixtures are non-linear combinations of latent source signals, is used to find independent sources when observed data are mixtures of unknown sources, and prior knowledge of the mixing mechanisms is not available. In this paper, a stock price forecasting model which first uses NLICA as preprocessing to extract features from forecasting variables is developed. Then the features, called independent components (ICs), serve as the inputs of support vector regression (SVR) to build the forecasting model. The advantage of the proposed methodology is that the information hidden in the original data can be discovered by feature extraction. Therefore, NLICA can provide more valuable information for financial forecasting. Two datasets of major Asian stock markets—China and Japan, Shanghai Stock Exchange Composite (SSEC) and Nikkei 225 stock indexes, are used as illustrative examples. For comparison, the integration of traditional principal component analysis (PCA) with SVR (called PCA–SVR), linear ICA with SVR (called LICA–SVR) and single SVR approaches were applied to evaluate the prediction accuracy of the proposed approach. Empirical results show that the proposed method (NLICA–SVR) not only improves the prediction accuracy of the SVR approach but also outperforms the PCA–SVR, LICA–SVR and single SVR methods.

Non-Linear Post-Processing of Numerical Seasonal Climate Forecasts

Although numerical models are increasingly being used to generate operational seasonal forecasts, the reliability of these products remains relatively low. Regression-based post-processing methods have proven useful in increasing forecast skill, but efforts have focused on linear regression. Given the non-linear nature of the climate system and sources of model error, non-linear analogues of these post-processing methods may offer considerable improvements. The current study tests this hypothesis, applying both linear and non-linear regression to the correction of climate hindcasts produced with general circulation models. Results indicate that non-linear support vector regression is better able to extract indices of the Pacific/North American teleconnection pattern and the North Atlantic Oscillation from coupled model output, while linear approaches are better suited to atmosphere-only model output. Statistically significant predictions are produced at lead times of up to nine months and can be obtained from model output with no forecast skill prior to processing. RÉSUMÉ [Traduit par la rédaction] Même si les modèles numériques sont de plus en plus utilisés pour produire des prévisions saisonnières opérationnelles, la fiabilité de ces produits reste plutôt faible. Des méthodes de post-traitement utilisant la régression se sont avérées utiles pour améliorer l'habileté des prévisions, mais les efforts ont surtout porté sur la régression linéaire. Étant donné la nature non linéaire du système climatique et des sources d'erreur des modèles, des analogues non linéaires de ces méthodes de post-traitement pourraient permettre des améliorations importantes. La présente étude cherche à vérifier cette hypothèse en appliquant une régression linéaire et une régression non linéaire à la correction de prévisions climatiques a posteriori produites à l'aide de modèles de circulation générale. Les résultats indiquent que la régression non linéaire des vecteurs de support arrive à mieux extraire les indices de configuration de la téléconnexion Pacifique/Amérique du Nord et de l'oscillation nord-atlantique de la sortie de modèles couplés alors que les approches linéaires font mieux avec la sortie de modèles atmosphériques seulement. Des prévisions statistiquement significatives sont produites pour jusqu’à neuf mois dans le futur et peuvent être obtenues de la sortie d'un modèle sans habileté de prévision avant le traitement.

Quantitative Structure–Activity Relationships and Docking Studies of Calcitonin Gene‐Related Peptide Antagonists

Defining the role of calcitonin gene-related peptide in migraine pathogenesis could lead to the application of calcitonin gene-related peptide antagonists as novel migraine therapeutics. In this work, quantitative structure-activity relationship modeling of biological activities of a large range of calcitonin gene-related peptide antagonists was performed using a panel of physicochemical descriptors. The computational studies evaluated different variable selection techniques and demonstrated shuffling stepwise multiple linear regression to be superior over genetic algorithm-multiple linear regression. The linear quantitative structure-activity relationship model revealed better statistical parameters of cross-validation in comparison with the non-linear support vector regression technique. Implementing only five peptide descriptors into this linear quantitative structure-activity relationship model resulted in an extremely robust and highly predictive model with calibration, leave-one-out and leave-20-out validation R(2) of 0.9194, 0.9103, and 0.9214, respectively. We performed docking of the most potent calcitonin gene-related peptide antagonists with the calcitonin gene-related peptide receptor and demonstrated that peptide antagonists act by blocking access to the peptide-binding cleft. We also demonstrated the direct contact of residues 28-37 of the calcitonin gene-related peptide antagonists with the receptor. These results are in agreement with the conclusions drawn from the quantitative structure-activity relationship model, indicating that both electrostatic and steric factors should be taken into account when designing novel calcitonin gene-related peptide antagonists.

Parameter-insensitive kernel in extreme learning for non-linear support vector regression

Support vector regression (SVR) is a state-of-the-art method for regression which uses the ε ‐ sensitive loss and produces sparse models. However, non-linear SVRs are difficult to tune because of the additional kernel parameter. In this paper, a new parameter-insensitive kernel inspired from extreme learning is used for non-linear SVR. Hence, the practitioner has only two meta-parameters to optimise. The proposed approach reduces significantly the computational complexity yet experiments show that it yields performances that are very close from the state-of-the-art. Unlike previous works which rely on Monte-Carlo approximation to estimate the kernel, this work also shows that the proposed kernel has an analytic form which is computationally easier to evaluate.

Seasonal prediction of winter extreme precipitation over Canada by support vector regression

Abstract. For forecasting the maximum 5-day accumulated precipitation over the winter season at lead times of 3, 6, 9 and 12 months over Canada from 1950 to 2007, two nonlinear and two linear regression models were used, where the models were support vector regression (SVR) (nonlinear and linear versions), nonlinear Bayesian neural network (BNN) and multiple linear regression (MLR). The 118 stations were grouped into six geographic regions by K-means clustering. For each region, the leading principal components of the winter maximum 5-d accumulated precipitation anomalies were the predictands. Potential predictors included quasi-global sea surface temperature anomalies and 500 hPa geopotential height anomalies over the Northern Hemisphere, as well as six climate indices (the Niño-3.4 region sea surface temperature, the North Atlantic Oscillation, the Pacific-North American teleconnection, the Pacific Decadal Oscillation, the Scandinavia pattern, and the East Atlantic pattern). The results showed that in general the two robust SVR models tended to have better forecast skills than the two non-robust models (MLR and BNN), and the nonlinear SVR model tended to forecast slightly better than the linear SVR model. Among the six regions, the Prairies region displayed the highest forecast skills, and the Arctic region the second highest. The strongest nonlinearity was manifested over the Prairies and the weakest nonlinearity over the Arctic.

Wavelength selection-based nonlinear calibration for transcutaneous blood glucose sensing using Raman spectroscopy.

While Raman spectroscopy provides a powerful tool for noninvasive and real time diagnostics of biological samples, its translation to the clinical setting has been impeded by the lack of robustness of spectroscopic calibration models and the size and cumbersome nature of conventional laboratory Raman systems. Linear multivariate calibration models employing full spectrum analysis are often misled by spurious correlations, such as system drift and covariations among constituents. In addition, such calibration schemes are prone to overfitting, especially in the presence of external interferences that may create nonlinearities in the spectra-concentration relationship. To address both of these issues we incorporate residue error plot-based wavelength selection and nonlinear support vector regression (SVR). Wavelength selection is used to eliminate uninformative regions of the spectrum, while SVR is used to model the curved effects such as those created by tissue turbidity and temperature fluctuations. Using glucose detection in tissue phantoms as a representative example, we show that even a substantial reduction in the number of wavelengths analyzed using SVR lead to calibration models of equivalent prediction accuracy as linear full spectrum analysis. Further, with clinical datasets obtained from human subject studies, we also demonstrate the prospective applicability of the selected wavelength subsets without sacrificing prediction accuracy, which has extensive implications for calibration maintenance and transfer. Additionally, such wavelength selection could substantially reduce the collection time of serial Raman acquisition systems. Given the reduced footprint of serial Raman systems in relation to conventional dispersive Raman spectrometers, we anticipate that the incorporation of wavelength selection in such hardware designs will enhance the possibility of miniaturized clinical systems for disease diagnosis in the near future.

Quantitative Structure-Fluorescence Property Relationship Analysis of a Large BODIPY Library.

A quantitative structure-fluorescence property relationship (QSPR) analysis of a large 288-membered library based on a single fluorescent BODIPY scaffold is presented for the first time. BODIPY is a versatile fluorescent scaffold with outstanding photophysical properties. Absorption (λabs ) and fluorescence emission (λem ) wavelength maxima were modeled with help of stepwise multiple linear regression (MLR) and support vector regression (SVR). The models were rigorously validated by 10-times 10-fold cross-validation (CV), y-scrambling CV and with an external validation set. Non-linear SVR models (R(2) =0.92 and Q(2) =0.71 for λabs ; R(2) =0.89 and Q(2) =0.69 for λem ) performed significantly better than linear models. A small root mean squared error (RMSE) of 5.62 nm and 11.07 nm was achieved for λabs and λem , respectively, and confirmed by external validation. A novel intramolecular charge transfer descriptor was developed based on the QSPR analysis and its inclusion in the modeling significantly improved models of λem . We conclude that QSPR is a useful tool for modeling λabs and λem of BODIPY fluorophores and suggest QSPR as an ideal partner for the design of compounds with tailored fluorescence properties in a diversity-oriented fluorescence library approach (DOFLA).

Financial risk forecasting with nonlinear dynamics and support vector regression

We propose a dynamical description of financial time series capable of making short-term prediction utilizing support vector regression on neighbourhood points. We include in our analysis estimation on the uncertainty by capturing the exogenous from historical prediction errors and adopting a probabilistic description of the prediction. Evidences from a series of backtesting using financial time series indicate that our model provides accurate description of real market data comparable with GARCH(1,1).

Parallel randomized sampling for support vector machine (SVM) and support vector regression (SVR)

A parallel randomized support vector machine (PRSVM) and a parallel randomized support vector regression (PRSVR) algorithm based on a randomized sampling technique are proposed in this paper. The proposed PRSVM and PRSVR have four major advantages over previous methods. (1) We prove that the proposed algorithms achieve an average convergence rate that is so far the fastest bounded convergence rate, among all SVM decomposition training algorithms to the best of our knowledge. The fast average convergence bound is achieved by a unique priority based sampling mechanism. (2) Unlike previous work (Provably fast training algorithm for support vector machines, 2001) the proposed algorithms work for general linear-nonseparable SVM and general non-linear SVR problems. This improvement is achieved by modeling new LP-type problems based on Karush–Kuhn–Tucker optimality conditions. (3) The proposed algorithms are the first parallel version of randomized sampling algorithms for SVM and SVR. Both the analytical convergence bound and the numerical results in a real application show that the proposed algorithm has good scalability. (4) We present demonstrations of the algorithms based on both synthetic data and data obtained from a real word application. Performance comparisons with SVMlight show that the proposed algorithms may be efficiently implemented.

Recursive Finite Newton Algorithm for Support Vector Regression in the Primal

Some algorithms in the primal have been recently proposed for training support vector machines. This letter follows those studies and develops a recursive finite Newton algorithm (IHLF-SVR-RFN) for training nonlinear support vector regression. The insensitive Huber loss function and the computation of the Newton step are discussed in detail. Comparisons with LIBSVM 2.82 show that the proposed algorithm gives promising results.

A novel approach for prediction of tacrolimus blood concentration in liver transplantation patients in the intensive care unit through support vector regression

IntroductionTacrolimus is an important immunosuppressive drug for organ transplantation patients. It has a narrow therapeutic range, toxic side effects, and a blood concentration with wide intra- and interindividual variability. Hence, it is of the utmost importance to monitor tacrolimus blood concentration, thereby ensuring clinical effect and avoiding toxic side effects. Prediction models for tacrolimus blood concentration can improve clinical care by optimizing monitoring of these concentrations, especially in the initial phase after transplantation during intensive care unit (ICU) stay. This is the first study in the ICU in which support vector machines, as a new data modeling technique, are investigated and tested in their prediction capabilities of tacrolimus blood concentration. Linear support vector regression (SVR) and nonlinear radial basis function (RBF) SVR are compared with multiple linear regression (MLR).MethodsTacrolimus blood concentrations, together with 35 other relevant variables from 50 liver transplantation patients, were extracted from our ICU database. This resulted in a dataset of 457 blood samples, on average between 9 and 10 samples per patient, finally resulting in a database of more than 16,000 data values. Nonlinear RBF SVR, linear SVR, and MLR were performed after selection of clinically relevant input variables and model parameters. Differences between observed and predicted tacrolimus blood concentrations were calculated. Prediction accuracy of the three methods was compared after fivefold cross-validation (Friedman test and Wilcoxon signed rank analysis).ResultsLinear SVR and nonlinear RBF SVR had mean absolute differences between observed and predicted tacrolimus blood concentrations of 2.31 ng/ml (standard deviation [SD] 2.47) and 2.38 ng/ml (SD 2.49), respectively. MLR had a mean absolute difference of 2.73 ng/ml (SD 3.79). The difference between linear SVR and MLR was statistically significant (p < 0.001). RBF SVR had the advantage of requiring only 2 input variables to perform this prediction in comparison to 15 and 16 variables needed by linear SVR and MLR, respectively. This is an indication of the superior prediction capability of nonlinear SVR.ConclusionPrediction of tacrolimus blood concentration with linear and nonlinear SVR was excellent, and accuracy was superior in comparison with an MLR model.