Nonlinear Support Vector Regression Research Articles

Long-term monthly average temperature forecasting in some climate types of Iran, using the models SARIMA, SVR, and SVR-FA

Temporal changes of the global surface temperature have been used as a prominent indicator of global climate change; therefore, making dependable forecasts underlies the foundation of sound environmental policies. In this research, the accuracy of the Seasonal Autoregressive Integrated Moving Average (SARIMA) Stochastic model has been compared with the Support Vector Regression (SVR) and its merged type with Firefly optimization algorithm (SVR-FA) as a meta-innovative model, in long-term forecasting of average monthly temperature. For this, 5 stations from different climates of Iran (according to the Extended De Martonne method) were selected, including Abadan, Anzali, Isfahan, Mashhad, and Tabriz. The data were collected during 1951–2011, for training (75%) and testing (25%). After selecting the best models, the average monthly temperature has been forecasted for the period 2012–2017. The results showed that the models had better performances in Extra-Arid and Warm (Abadan) and after that Extra-Arid and Cold (Isfahan) climate, in long-term forecasting. The weakest performances of the models were reported in Semi-Arid and Cold climate, including Mashhad and Tabriz. Also, despite the use of the non-linear SVR model and its meta-innovative type, SVR-FA, the results showed that, in the climates of Iran, the linear and classical SARIMA model still offers a more appropriate performance in temperature long-term forecasting. So that it could forecast the average monthly temperature of Abadan with root mean square error (RMSE) = 1.027 °C, and Isfahan with RMSE = 1.197 °C for the 6 years ahead. The SVR and SVR-FA models also had good performances. The results of this checking also report the effectiveness of the merging SVR model with the Firefly optimization algorithm in temperature forecasting in Iran’s climates, so, compared with the SVR model, it is suggested to use SVR-FA for temperature forecasting.

Nonlinear Complex Support Vector Regression for Fading Channel Estimation in FBMC-OQAM System

Most channel estimation techniques for orthogonal frequency division multiplexing system are not directly applicable to filter bank multi-carrier with offset quadrature amplitude modulation (FBMC-OQAM) system because of the imaginary interference of FBMC-OQAM system. Thus, a novel support vector regression (SVR) algorithm for fading channel estimation in FBMC-OQAM system is proposed. We propose a nonlinear complex SVR algorithm (NCSVR) adapted to two auxiliary pilots (AP)-based FBMC-OQAM signal, namely as AP-NCSVR. The practicality of our method is proved by simulation results obtained for Jakes fading model and vehicular A model. Simulation results reveal the accuracy of the proposed AP-NCSVR algorithm to estimate channel frequency response coefficients and show that the bit error rate performance of the proposed AP-NCSVR algorithm is better than the conventional interpolation method for Jakes fading model and vehicular A model.

A hybrid electricity price forecasting model with Bayesian optimization for German energy exchange

Electricity price forecasting affects the operation of the entire electricity market and it is extremely important to every market participant. In this paper, a novel hybrid method, with using empirical wavelet transform (EWT), support vector regression (SVR), Bi-directional long short-term memory (BiLSTM) and Bayesian optimization (BO), is proposed to increase the accuracy of electricity price forecasting. First, EWT is used as a processing tool to decompose the original signal into specific modal components according to the characteristics of the signal itself. Then, considering the complexity of forecasting nonlinear subseries, SVR and BiLSTM are used as basic framework to forecast the nonlinear subseries. At the same time, BO is introduced to adjust parameters and optimize model performance. In last, the final prediction results are combined by the prediction results of different models. The proposed hybrid model is employed on the data gathered from the European Power Exchange Spot (EPEXSPOT). Five different case studies are adopted to verify the effectiveness of BO, EWT and hybrid model respectively. Statistical tests of experimental results compared with other situations demonstrated the proposed hybrid model can achieve a better forecasting performance.

Yield Estimation of Paddy Rice Based on Satellite Imagery: Comparison of Global and Local Regression Models

Precisely estimating the yield of paddy rice is crucial for national food security and development evaluation. Rice yield estimation based on satellite imagery is usually performed with global regression models; however, estimation errors may occur because the spatial variation is not considered. Therefore, this study proposed an approach estimating paddy rice yield based on global and local regression models. In our study area, the overall per-field data might not available because it took lots of time and manpower as well as resources. Therefore, we gathered and accumulated 26 to 63 ground survey sample fields, accounting for about 0.05% of the total cultivated areas, as the training samples for our regression models. To demonstrate whether the spatial autocorrelation or spatial heterogeneity exists and dominates the estimation, global models including the ordinary least squares (OLS), support vector regression (SVR), and the local model geographically weighted regression (GWR) were used to build the yield estimation models. We obtained the representative independent variables, including 4 original bands, 11 vegetation indices, and 32 texture indices, from SPOT-7 multispectral satellite imagery. To determine the optimal variable combination, feature selection based on the Pearson correlation was used for all of the regression models. The case study in Central Taiwan rendered that the error rate was between 0.06% and 13.22%. Through feature selection, the GWR model’s performance was more relatively stable than the OLS model and nonlinear SVR model for yield estimation. Where the GWR model considers the spatial autocorrelation and spatial heterogeneity of the relationships between the yield and the independent variables, the OLS and nonlinear SVR models lack this feature; this led to the rice yield estimation of GWR in this study be more stable than those of the other two models.

Shear strength of steel fiber-unconfined reinforced concrete beam simulation: Application of novel intelligent model

The research promotes a new nonlinear model-based hybridized response surface method (RSM) and support vector regression (RSM-SVR) to predict shear capacity of steel fiber-reinforced concrete beams (SFRCB). Two approaches are integrated using RSM which is calibrated based on two input datasets; whereas, the SVR is calibrated based on all the predicted datasets generated by RSM. The high-cross correlation of the input dataset is provided using two nonlinear steps for modeling the SFRCB shear strength. The capacity of hybrid RSM-SVR model is validated with stand-alone intelligent models RSM, SVR and neural network (NN) in addition to eight empirical formulations. The dataset of 139 laboratory experimental tests of shear failure capacity belongs to SFRCB without stirrups, are obtained from the literature. The effects of fiber volume and the longitudinal steel ratio on the shear predictions of the normal and high-strange concrete reinforced by steel fiber are compared for the intelligent and empirical based approaches. The achieved results indicated the RSM-SVR model performed superior prediction over the comparable models. The improved agreement indexes using the RSM-SVR were improved with (0.35 and 1.9) over the empirical formulations and with (0.8, 1.2 and 3.5) over the three intelligent models of RSM, SVR and NN, respectively.

Permeability prediction of isolated channel sands using machine learning

Permeability prediction has long been one of the most important and difficult tasks of reservoir characterization, especially for tight sands with strong heterogeneity. A unified model is important for the un-cored wells and can reduce a lot of work in practice. In this case, we are going to establish a field-scale unified permeability model for the tight gas sands of the Middle Jurassic Shaximiao Formation in the Western Sichuan Basin, China. The tight gas sands are isolated channel sands with strong heterogeneity. In addition to the porosity, well logs are also selected as the input feature to help to derive the non-linear relations. We choose simple linear regression (SLR) using porosity, multiple linear regression (MLR), multiple-layer perceptron (MLR) and support vector regression (SVR) using multiple input features as our machine learning methods for this task. For the non-linear methods, MLP and SVR, we present how to decide their key parameters based on the theoretical analysis and experiments to keep their generalization capability. Finally, the results show that the non-linear MLP and SVR outperform the linear SLR and MLR for the unified permeability model. The blind well average performance of MLP and SVR improves about 50% on R2 and decrease about 30% on MARE than MLR.

Early Findings on Functional Connectivity Correlates of Behavioral Outcomes of Brain-Computer Interface Stroke Rehabilitation Using Machine Learning.

The primary goal of this work was to apply data-driven machine learning regression to assess if resting state functional connectivity (rs-FC) could estimate measures of behavioral domains in stroke subjects who completed brain-computer interface (BCI) intervention for motor rehabilitation. The study cohort consisted of 20 chronic-stage stroke subjects exhibiting persistent upper-extremity motor deficits who received the intervention using a closed-loop neurofeedback BCI device. Over the course of this intervention, resting state functional MRI scans were collected at four distinct time points: namely, pre-intervention, mid-intervention, post-intervention and 1-month after completion of intervention. Behavioral assessments were administered outside the scanner at each time-point to collect objective measures such as the Action Research Arm Test, Nine-Hole Peg Test, and Barthel Index as well as subjective measures including the Stroke Impact Scale. The present analysis focused on neuroplasticity and behavioral outcomes measured across pre-intervention, post-intervention and 1-month post-intervention to study immediate and carry-over effects. Rs-FC, changes in rs-FC within the motor network and the behavioral measures at preceding stages were used as input features and behavioral measures and associated changes at succeeding stages were used as outcomes for machine-learning-based support vector regression (SVR) models. Potential clinical confounding factors such as age, gender, lesion hemisphere, and stroke severity were included as additional features in each of the regression models. Sequential forward feature selection procedure narrowed the search for important correlates. Behavioral outcomes at preceding time-points outperformed rs-FC-based correlates. Rs-FC and changes associated with bilateral primary motor areas were found to be important correlates of across several behavioral outcomes and were stable upon inclusion of clinical variables as well. NIH Stroke Scale and motor impairment severity were the most influential clinical variables. Comparatively, linear SVR models aided in evaluation of contribution of individual correlates and seed regions while non-linear SVR models achieved higher performance in prediction of behavioral outcomes.

Financial Time Series Forecasting Using Empirical Mode Decomposition and FNN: A Study on Selected Foreign Exchange Rates

The exchange rate, an economic indicator of the country is the relative price of one country’s currency in terms of another country’s currency. The stability of the exchange rate is important for a stable economic growth. Exchange rate series are non-linear and non-stationary. The fluctuations in the forecasting exchange rate are very important to the economy of the country. Researchers have proposed many hybrid machine learning models to get a more accurate forecast. This study proposes a hybrid forecasting model using Empirical Mode Decomposition (EMD) and Feedforward Neural Network (FNN) for foreign exchange rates forecasting and comparing its performance with widely used Non-linear Autoregressive (NAR) and Support Vector Regression (SVR) models. EMD is used to decompose the original non-linear and non-stationary series into several Intrinsic Mode Functions (IMFs) and one residual. The hybrid model is then used to forecast the exchange rate with IMFs and residual obtained as inputs. Empirical results obtained from forecasting daily exchange rates of Sri Lankan Rupees to Euro and Yen showed that the proposed EMD-FNN model outperforms NAR and SVR models without time series decomposition.

Large-scale support vector regression with budgeted stochastic gradient descent

Support vector regression (SVR) is a widely used regression technique for its competent performance. However, non-linear SVR is time consuming for large-scale tasks due to the dimension curse of kernelization. Recently, a budgeted stochastic gradient descent (BSGD) method has been developed to train large-scale kernelized SVC. In this paper, we extend the BSGD method to non-linear regression tasks. According to the performance of different budget maintenance strategies, we combine the stochastic gradient descent (SGD) method with the merging strategy. Experimental results on real-world datasets show that the proposed kernelized SVR with BSGD can achieve competent accuracy, with good computational efficiency compared to some state-of-the-art algorithms.

Modeling and forecasting of oilseed production of India through artificial intelligence techniques

Indian agriculture has made considerable progress in respect of principle food crops, but the performance in case of oilseed crops is so far not good as compared to food grains. Production of oilseeds and oils are not meeting the increasing demand for edible oils and this widening demand-supply gap has necessitated imports of edible oils. India is world's largest importer of oil seeds as it imports more than 50 percent of its total production. Forecasting is used to analyze the past and current behavior to forecasts the future oilseeds production which intern provide an aid to decision-making and in planning for the future effectively and efficiently. Autoregressive integrated moving average (ARIMA) model is the most widely used model for forecasting time series. One of the main drawback of this model is the presumption of linearity. To model the series which contains nonlinear patterns, the artificial intelligence techniques like time delay neural network (TDNN) and non-linear support vector regression (NLSVR) model are commonly employed. In this paper an attempt has been made to forecast the oilseed production of India using ARIMA, TDNN and NLSVR models. Empirical results clearly reveal that the artificial intelligence techniques outperformed the ARIMA model.

Estimations of Nitrogen Concentration in Sugarcane Using Hyperspectral Imagery

This study aims to estimate the spatial variation of sugarcane Canopy Nitrogen Concentration (CNC) using spectral data, which were measured from a spaceborne hyperspectral image. Stepwise Multiple Linear Regression (SMLR) and Support Vector Regression (SVR) were applied to calibrate and validate the CNC estimation models. The raw spectral reflectance was transformed into a First-Derivative Spectrum (FDS) and absorption features to remove the spectral noise and finally used as input variables. The results indicate that the estimation models developed by non-linear SVR based Radial Basis Function (RBF) kernel yield the higher correlation coefficient with CNC compared with the models computed by SMLR. The best model shows the coefficient of determination value of 0.78 and Root Mean Square Error (RMSE) value of 0.035% nitrogen. The narrow sensitive spectral wavelengths for quantifying nitrogen content in the combined cultivar environments existed mainly in the electromagnetic spectrum of the visible-red, longer portion of red edge, shortwave infrared regions and far-near infrared. The most important conclusion from this experiment is that spectral signals from the space hyperspectral data contain the meaningful information for quantifying sugarcane CNC across larger geographic areas. The nutrient deficient areas could be corrected by applying suitable farm management.

Life assessment and health monitoring of rolling element bearings: an experimental study

Rolling element bearings are the key elements of almost all rotating machineries and play a major role in efficient performance of such machines. Industries are facing difficulties to develop a reliable methodology that can help to predict the remaining useful life of such machine elements, so that these can be replaced before catastrophic failure. This study proposes a data-driven framework for predicting the remnant life of bearings, based on nonlinear dimensionality reduction and least-square support vector regression. Experiments are conducted to assess the life of bearing. Vibration signals are extracted from the test bearing and various time and frequency domain features are used to form a health index. This health index is then used for learning and training the regression model which helps in assessment of remaining useful life. Vibration parameters monitoring and wear mechanisms observations have been carried out to identify the various degradation stages of the bearing. Results show the potential of proposed methodology for predicting the remaining useful life of the bearing.

Applied Machine Learning for the Prediction of Growth of Abdominal Aortic Aneurysm in Humans

ObjectiveAccurate prediction of abdominal aortic aneurysm (AAA) growth in an individual can allow personalised stratification of surveillance intervals and better inform the timing for surgery. The authors recently described the novel significant association between flow mediated dilatation (FMD) and future AAA growth. The feasibility of predicting future AAA growth was explored in individual patients using a set of benchmark machine learning techniques.MethodsThe Oxford Abdominal Aortic Aneurysm Study (OxAAA) prospectively recruited AAA patients undergoing the routine NHS management pathway. In addition to the AAA diameter, FMD was systemically measured in these patients. A benchmark machine learning technique (non-linear Kernel support vector regression) was applied to predict future AAA growth in individual patients, using their baseline FMD and AAA diameter as input variables.ResultsProspective growth data were recorded at 12 months (360 ± 49 days) in 94 patients. Of these, growth data were further recorded at 24 months (718 ± 81 days) in 79 patients. The average growth in AAA diameter was 3.4% at 12 months, and 2.8% per year at 24 months. The algorithm predicted the individual's AAA diameter to within 2 mm error in 85% and 71% of patients at 12 and 24 months.ConclusionsThe data highlight the utility of FMD as a biomarker for AAA and the value of machine learning techniques for AAA research in the new era of precision medicine.

Predicting Top-of-Atmosphere Thermal Radiance Using MERRA-2 Atmospheric Data with Deep Learning

Image data from space-borne thermal infrared (IR) sensors are used for a variety of applications, however they are often limited by their temporal resolution (i.e., repeat coverage). To potentially increase the temporal availability of thermal image data, a study was performed to determine the extent to which thermal image data can be simulated from available atmospheric and surface data. The work conducted here explored the use of Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) developed by The National Aeronautics and Space Administration (NASA) to predict top-of-atmosphere (TOA) thermal IR radiance globally at time scales finer than available satellite data. For this case study, TOA radiance data was derived for band 31 (10.97 μ m) of the Moderate-Resolution Imaging Spectroradiometer (MODIS) sensor. Two approaches have been followed, namely an atmospheric radiative transfer forward modeling approach and a supervised learning approach. The first approach uses forward modeling to predict TOA radiance from the available surface and atmospheric data. The second approach applied four different supervised learning algorithms to the atmospheric data. The algorithms included a linear least squares regression model, a non-linear support vector regression (SVR) model, a multi-layer perceptron (MLP), and a convolutional neural network (CNN). This research found that the multi-layer perceptron model produced the lowest overall error rates with an root mean square error (RMSE) of 1.36 W/m 2 ·sr· μ m when compared to actual Terra/MODIS band 31 image data. These studies found that for radiances above 6 W/m 2 ·sr· μ m, the forward modeling approach could predict TOA radiance to within 12 percent, and the best supervised learning approach can predict TOA to within 11 percent.

Advanced monitoring and control of pharmaceutical production processes with Pichia pastoris by using Raman spectroscopy and multivariate calibration methods.

This contribution includes an investigation of the applicability of Raman spectroscopy as a PAT analyzer in cyclic production processes of a potential Malaria vaccine with Pichia pastoris. In a feasibility study, Partial Least Squares Regression (PLSR) models were created off-line for cell density and concentrations of glycerol, methanol, ammonia and total secreted protein. Relative cross validation errors RMSEcvrel range from 2.87% (glycerol) to 11.0% (ammonia). In the following, on-line bioprocess monitoring was tested for cell density and glycerol concentration. By using the nonlinear Support Vector Regression (SVR) method instead of PLSR, the error RMSEPrel for cell density was reduced from 5.01 to 2.94%. The high potential of Raman spectroscopy in combination with multivariate calibration methods was demonstrated by the implementation of a closed loop control for glycerol concentration using PLSR. The strong nonlinear behavior of exponentially increasing control disturbances was met with a feed-forward control and adaptive correction of control parameters. In general the control procedure works very well for low cell densities. Unfortunately, PLSR models for glycerol concentration are strongly influenced by a correlation with the cell density. This leads to a failure in substrate prediction, which in turn prevents substrate control at cell densities above 16 g/L.

Improved predictions of nonlinear support vector regression and artificial neural network models via preprocessing of data with orthogonal projection to latent structures: A case study

In the presented study, orthogonal projection to latent structures (OPLS) is introduced asa data preprocessing method that handles nonlinear data prior to modelling with two well established nonlinear multivariate models; namely support vector regression (SVR) and artificial neural networks (ANN). The proposed preprocessing proved to significantly improve prediction abilities through removal of uncorrelated data.The study was established based on a case study nonlinear spectrofluorimetric data of agomelatine (AGM) and its hydrolysis degradation products (Deg I and Deg II), where a 3 factor 4 level experimental design was used to provide a training set of 16 mixtures with different proportions of studied components. An independent test set which consisted of 9 mixtures was established to confirm the prediction ability of the introduced models. Excitation wavelength was 227nm, and working range for emission spectra was 320–440nm.The couplings of OPLS-SVR and OPLS-ANN provided better accuracy for prediction of independent nonlinear test set. The root mean square error of prediction RMSEP for the test set mixtures was used asa major comparison parameter, where RMSEP results for OPLS-SVR and OPLS-ANN are 2.19 and 1.50 respectively.

Support-vector-regression-based prediction of water holdup in horizontal oil-water flow by using a bicircular conductance probe array

This paper presents a water holdup prediction method based on support vector regression (SVR) for horizontal oil-water two-phase flow when using a bicircular conductance probe array that consists of 24 conductance probes. The support vector machine (SVM) was employed to establish a nonlinear SVR model mapping the probe array responses into water holdup directly. Experiments were carried out in the 16m long and 125mm inner diameter horizontal pipe of an industrial scale experimental setup. The experimental data obtained under 220 flow conditions were first divided into modeling data set and comparing data set. The modeling data set is used for establishing a nonlinear SVR and a linear least squares regression (LSR) models, while the comparing data set is used for comparing both models with the equi-weight and optimal weight estimate methods. Comparison results obtained by using the comparing data set show that when the binary data of the probes’ responses are used only, the measurement accuracy of the optimal weight estimate method is the best. If the analog data can be obtained, the measurement accuracy of both regression methods are better than those of both weighting estimate methods, especially, the nonlinear SVR method provide the best measurement accuracy.

An improved variable selection method for support vector regression in NIR spectral modeling

Support vector regression (SVR) has become increasingly popular in analyzing near-infrared spectroscopic data. As an alternative to the conventional partial least squares (PLS) methods, the advantages of SVR include better generalization performance and its robustness against nonlinear input-output relationships. Since most variable selection methods in literature focus on identification of linear input-output relationships, these methods are suboptimal when nonlinear SVR models are constructed. In this paper, we propose a variable selection method using recursive feature elimination with mutual information-based bias correction (RFE-MICBC). The proposed method is able to handle nonlinearity and correlations within the feature space. Through two case studies (one with simulated data and one with diesel NIR spectra), the performance of SVR-RFE-MICBC is compared against variable importance in projection (VIP) based feature selection method for PLS and a probabilistic prediction method for SVR [1]. The results demonstrate that the new method is superior in performance especially when the data set contains a high level of nonlinearities. Lastly, we also show that variable selection is beneficial in improving model efficiency and interpretability. The resulting wavelength bands considered to be important provide valuable guidance for future spectral analysis of diesel fuels.

Quantitative prediction of ionization effect on human skin permeability

Although skin permeability of an active ingredient can be severely affected by its ionization in a dose solution, most of the existing prediction models cannot predict such impacts. To provide reliable predictors, we curated a novel large dataset of in vitro human skin permeability coefficients for 322 entries comprising chemically diverse permeants whose ionization fractions can be calculated. Subsequently, we generated thousands of computational descriptors, including LogD (octanol–water distribution coefficient at a specific pH), and analyzed the dataset using nonlinear support vector regression (SVR) and Gaussian process regression (GPR) combined with greedy descriptor selection. The SVR model was slightly superior to the GPR model, with externally validated squared correlation coefficient, root mean square error, and mean absolute error values of 0.94, 0.29, and 0.21, respectively. These models indicate that Log D is effective for a comprehensive prediction of ionization effects on skin permeability. In addition, the proposed models satisfied the statistical criteria endorsed in recent model validation studies. These models can evaluate virtually generated compounds at any pH; therefore, they can be used for high-throughput evaluations of numerous active ingredients and optimization of their skin permeability with respect to permeant ionization.

Single Image Super-Resolution by Non-Linear Sparse Representation and Support Vector Regression

Sparse representations are widely used tools in image super-resolution (SR) tasks. In the sparsity-based SR methods, linear sparse representations are often used for image description. However, the non-linear data distributions in images might not be well represented by linear sparse models. Moreover, many sparsity-based SR methods require the image patch self-similarity assumption; however, the assumption may not always hold. In this paper, we propose a novel method for single image super-resolution (SISR). Unlike most prior sparsity-based SR methods, the proposed method uses non-linear sparse representation to enhance the description of the non-linear information in images, and the proposed framework does not need to assume the self-similarity of image patches. Based on the minimum reconstruction errors, support vector regression (SVR) is applied for predicting the SR image. The proposed method was evaluated on various benchmark images, and promising results were obtained.