Kernel Support Vector Regression Research Articles

Sensor Data Fusion by Support Vector Regression Methodology—A Comparative Study

Multisensor data fusion can be considered as a strong nonlinear system. A precise analytical solution is challenging to obtain, thus making it hard to dissect with routine diagnostic systems. Since tried-and-true logical systems are extremely difficult to undertake, soft computing methodologies are deemed having potential for such applications. This paper presents the support vector regression (SVR) methodology for sensor fusion to improve tracking ability. Radial basis function (RBF) and polynomial function are used as SVR kernel functions. The system combines Kalman filtering and soft computing principle, i.e., SVR, to structure an effective information combination method for the target framework. A radar-infrared system is proposed to adapt contextual changes and lessen the dubious unsettling influence of an information estimation from multisensory data. The experimental results show that an improvement in predictive accuracy and generalization capability can be achieved using the SVR with RBF kernel compared with the SVR with polynomial kernel approach.

Prediction of the solar radiation on the Earth using support vector regression technique

The solar rays on the surface of Earth is one of the major factor in water resources, environmental and agricultural modeling. The main environmental factors influencing plants growth are temperature, moisture, and solar radiation. Solar radiation is rarely obtained in weather stations; as a result, many empirical approaches have been applied to estimate it by using other parameters. In this study, a soft computing technique, named support vector regression (SVR) has been used to estimate the solar radiation. The data was collected from two synoptic stations with different climate conditions (Zahedan and Bojnurd) during the period of 5 and 7years, respectively. These data contain sunshine hours, maximum temperature, minimum temperature, average relative humidity and daily solar radiation. In this study, the polynomial and radial basis functions (RBF) are applied as the SVR kernel function to estimate solar radiation. The performance of the proposed estimators is confirmed with the simulation results.

A Fast Luminance Inspector for Backlight Modules Based on Multiple Kernel Support Vector Regression

The liquid crystal display (LCD) is widely used in various devices nowadays, for examples, mobile phones, digital cameras, and machine controllers. Among the components composing an LCD, the backlight module is one of the most important components because it provides a uniform light source for the display. In the current practice of backlight module inspection, a luminance colorimeter is used to examine the luminance emitted from selected areas called check points within the backlight module. Since a backlight module consists of many check points, it takes too much time to inspect one backlight module in the current practice. In this paper, we propose a fast luminance inspector that uses a CCD camera to inspect multiple backlight modules, simultaneously. The inspection speed is improved considerably, and thus, makes the proposed system suitable for use in the production line. The proposed system translates image intensities of all check points into luminance values based on a multiple kernel support vector regression model (MKSVR). The parameters of multiple kernel functions are automatically generated, according to the data distribution characteristics of the training samples. Compared with other studies using a grid search or a heuristic search algorithm to determine optimal kernel parameters, the proposed approach is more flexible and faster. Besides that, as the kernel parameters are adaptive to the training data, the proposed method could make its learning more specific to certain golden training samples. Other methods using learning algorithms such as the neural network and the SVR do not have such flexibility, because in these learning algorithms all training samples are treated equally. The flexible learning property makes the proposed system more appealing for use in the real practical world. Compared with other methods using the neural network model and the SVR model in backlight module inspection, the proposed method is superior in both the accuracy and the training time required.

Support Vector Regression Model for the prediction of Loadability Margin of a Power System

Loadability limits are critical points of particular interest in voltage stability assessment, indicating how much a system can be stressed from a given state before reaching instability. Thus estimating the loadability margin of a power system is essential in the real time voltage stability assessment. A new methodology is developed based on Support Vector Regression (SVR) which is the most common application form of Support Vector Machines (SVM). The proposed SVR methodology can successfully estimate the loadability margin under normal operating conditions and different loading directions. SVR has the feature of minimizing the generalization error in achieving the generalized network over the other mapping methods. In this paper, the SVR input vector is in the form of real and reactive power load, while the target vector is lambda (loading margin). To reduce both mean square error and prediction time in SVR, the kernel type and SVR parameters are chosen determined by using grid search based on 10-fold cross-validation method for the best SVR network. The results of SVRs (nu-SVR and epsilon-SVR) are compared with RBF neural networks and validated in the IEEE 30 bus system and IEEE 118 bus system at different operating scenarios. The results demonstrate the effectiveness of the proposed method for on-line prediction of loadability margins of a power system.

Disease Diagnosis with a hybrid method SVR using NSGA-II

Early diagnosis of any disease at a lower cost is preferable. Automatic medical diagnosis classification tools reduce financial burden on health care systems. In medical diagnosis, patterns consist of observable symptoms and the results of diagnostic tests, which have various associated costs and risks. In this paper, we have experimented and suggested an automated pattern classification method for classifying four diseases into two classes.In the literature on machine learning or data mining, regression and classification problems are typically viewed as two distinct problems differentiated by continuous or categorical dependent variables. There are endeavors to use regression methods to solve classification problems and vice versa. To regard a classification problem as a regression one, we propose a method based on the Support Vector Regression (SVR) classification model as one of the powerful methods in intelligent field management. We apply the Non-dominated Sorting Genetic Algorithm-II (NSGA-II), a kind of multi-objective evolutionary algorithm, to find mapping points (MPs) for rounding a real-value to an integer one. Also, we employ the NSGA-II to find out and tune the SVR kernel parameters optimally so as to enhance the performance of our model and achieve better results. The results of the study are compared with the results of some previous studies focusing on the diagnoses of four diseases using the same UCI machine learning database. The experimental results show that the proposed method yields a superior and competitive performance in these four real-world datasets.

Dynamic customer lifetime value prediction using longitudinal data: An improved multiple kernel SVR approach

Customer lifetime value (CLV), as an important metric in customer relationship management (CRM), has attracted widespread attention over the last decade. Most CLV prediction models do not take into consideration the dynamics of the customer purchase behavior and changes of the marketing environment such as the adoption of different promotion policies. In this study, a framework for the dynamic CLV prediction using longitudinal data is presented. In the framework, both the dynamic customer purchase behavior and customized promotions are considered. An improved multiple kernel support vector regression (MK-SVR) approach is developed to predict the future CLV and select the best promotion using both the customer behavioral variables and controlled variable about multiple promotions. Computational experiments using two databases show that the MK-SVR exhibits good prediction performance and the usage of longitudinal data in the MK-SVR facilitate the dynamic prediction and promotion optimization.

Evolutionary optimization of multi-parametric kernel $$\epsilon$$ -SVMr for forecasting problems

In this paper, we propose a novel multi-parametric kernel Support Vector Regression algorithm (SVMr) optimized with an evolutionary technique, specially well suited for forecasting problems. The multi-parametric SVMr model and the evolutionary algorithm proposed are both described in detail in the paper. In addition, several new bounds for the multi-parametric kernel considered are obtained, in such a way that the SVMr hyper-parameters' search space is reduced. We present experimental evidences of the good performance of the evolutionary algorithm for optimizing the multi-parametric kernel, when compared to a standard SVMr with a Grid Search approach. Specifically, results in different real regression problems from public repositories are obtained, and also a real application focused on the short-term temperature prediction at Barcelona's airport. The results obtained have shown the good performance of the multi-parametric kernel approach both in accuracy and computation time.

Protein secondary structure prediction using DWKF based on SVR-NSGAII

Prediction of protein secondary structure is an important step towards elucidating its three dimensional structure and its function. This is a challenging problem in bioinformatics. By introduction of machine learning for protein structure prediction, a solution has brought to this challenge to some extent. In the literature of Machine learning or data mining, regression and classification problems are typically viewed as two distinct problems differentiated by continuous or categorical dependent variable. There are endeavors to use regression methods to solve the classification problem and vice versa. To regard a classification problem as a regression one, we proposed a method which is based on Support Vector Regression (SVR) classification model as one of the powerful methods in the field of machine intelligence. We applied non-dominated Sorting Genetic Algorithm II (NSGAII) to find mapping points (MPs) for rounding a real-value to an integer one. Also NSGAII is used for finding out and tuning SVR kernel parameters optimally to enhance the performance of our model and achieve better results. At the other hand, using a suitable SVR kernel function for a particular problem can improve the prediction results remarkably but there is not a kernel which can predict all protein secondary structure classes with acceptable accuracy. Therefore we use a Dynamic Weighted Kernel Fusion (DWKF) method for fusing of three SVR kernels to achieve a supreme performance. Also to improve our method, Position Scoring Matrix (PSSM) profiles are used as the input information to it. The goals of this research are to regulate SVR parameters and fuse different SVR kernel outputs in order to determine protein secondary structure classes accurately. The obtained classification accuracies of our method are 85.79% and 84.94% on RS126 and CB513 datasets respectively and they are promising with regard to other classification methods in the literature. Moreover, for gauging our method behavior in comparison to other state of arts methods, an independent dataset is used and achieves 81.4% accuracy. Our method cannot achieve the best value for any considered performance metrics on an independent dataset but its values for whole metrics are quite acceptable.

A comparison of statistical methods for genomic selection in a mice population

BackgroundThe availability of high-density panels of SNP markers has opened new perspectives for marker-assisted selection strategies, such that genotypes for these markers are used to predict the genetic merit of selection candidates. Because the number of markers is often much larger than the number of phenotypes, marker effect estimation is not a trivial task. The objective of this research was to compare the predictive performance of ten different statistical methods employed in genomic selection, by analyzing data from a heterogeneous stock mice population.ResultsFor the five traits analyzed (W6W: weight at six weeks, WGS: growth slope, BL: body length, %CD8+: percentage of CD8+ cells, CD4+/ CD8+: ratio between CD4+ and CD8+ cells), within-family predictions were more accurate than across-family predictions, although this superiority in accuracy varied markedly across traits. For within-family prediction, two kernel methods, Reproducing Kernel Hilbert Spaces Regression (RKHS) and Support Vector Regression (SVR), were the most accurate for W6W, while a polygenic model also had comparable performance. A form of ridge regression assuming that all markers contribute to the additive variance (RR_GBLUP) figured among the most accurate for WGS and BL, while two variable selection methods ( LASSO and Random Forest, RF) had the greatest predictive abilities for %CD8+ and CD4+/ CD8+. RF, RKHS, SVR and RR_GBLUP outperformed the remainder methods in terms of bias and inflation of predictions.ConclusionsMethods with large conceptual differences reached very similar predictive abilities and a clear re-ranking of methods was observed in function of the trait analyzed. Variable selection methods were more accurate than the remainder in the case of %CD8+ and CD4+/CD8+ and these traits are likely to be influenced by a smaller number of QTL than the remainder. Judged by their overall performance across traits and computational requirements, RR_GBLUP, RKHS and SVR are particularly appealing for application in genomic selection.

ℓp-Norm Multikernel Learning Approach for Stock Market Price Forecasting

Linear multiple kernel learning model has been used for predicting financial time series. However, ℓ 1-norm multiple support vector regression is rarely observed to outperform trivial baselines in practical applications. To allow for robust kernel mixtures that generalize well, we adopt ℓ p-norm multiple kernel support vector regression (1 ≤ p < ∞) as a stock price prediction model. The optimization problem is decomposed into smaller subproblems, and the interleaved optimization strategy is employed to solve the regression model. The model is evaluated on forecasting the daily stock closing prices of Shanghai Stock Index in China. Experimental results show that our proposed model performs better than ℓ 1-norm multiple support vector regression model.

The comparative performance of four respiratory motion predictors for real-time tumour tracking

Prediction of respiratory motion is essential for real-time tracking of lung or liver tumours in radiotherapy to compensate for system latencies. This study compares the performance of respiratory motion prediction based on linear regression (LR), neural networks (NN), kernel density estimation (KDE) and support vector regression (SVR) for various sampling rates and system latencies ranging from 0.2 to 0.6 s. Root-mean-squared prediction errors are evaluated on 12 3D lung tumour motion traces acquired at 30 Hz during radiotherapy treatments. The effect of stationary predictor training versus continuous predictor retraining as well as full 3D motion processing versus independent coordinate-wise motion processing is investigated. Model parameter optimization is performed through a grid search in the model parameter space for each predictor and all considered latencies, sampling rates, training schemes and 3D data-processing modes. Comparison of the predictors is performed in the clinically applicable setting of patient-independent model parameters. The considered predictors roughly halve the prediction errors compared to using no prediction. When averaging over all sampling rates and latencies, prediction errors normalized to errors of using no prediction of 0.44, 0.46, 0.49 and 0.55 for NN, SVR, LR and KDE are observed. The small differences between the predictors emphasize the relative importance of adequate model parameter optimization compared to the actual prediction model selection. Thorough model parameter tuning is therefore essential for fair predictor comparisons.

Prediction of TVB-N content in eggs based on electronic nose

Total volatile basic nitrogen (TVB-N) content is an important freshness index of egg. An electronic nose was used to distinguish room-temperature storage periods of eggs by means of principal component analysis (PCA). The loadings plot analysis was used to identify the sensor responses as input parameters of support vector regression (SVR) model. Responses of sensor array in electronic nose were employed to establish TVB-N content model able to describe egg storage periods. Results showed that the E-nose could distinguish eggs of different storage periods by PCA. The optimum SVR kernel function was selected as Gaussian kernel by simulation. The optimum SVR inner parameters of the penalty parameter C, the radius ε and the width parameter δ2 were studied and set at 25, 2−3 and 22 by the grid searching method. The simulation results demonstrated that the SVR model could achieve better accuracy and generalization than the back-propagation neural network (BPNN). The experimental results showed that the average prediction accuracy, RMSE and MRE of the TVB-N content prediction SVR model were achieved as 94.62%, 0.09% and 0.05% respectively, which implied that the E-nose was effective for TVB-N content prediction in eggs by the SVR model.

Opening the kernel of kernel partial least squares and support vector machines

Kernel partial least squares (KPLS) and support vector regression (SVR) have become popular techniques for regression of complex non-linear data sets. The modeling is performed by mapping the data in a higher dimensional feature space through the kernel transformation. The disadvantage of such a transformation is, however, that information about the contribution of the original variables in the regression is lost. In this paper we introduce a method which can retrieve and visualize the contribution of the variables to the regression model and the way the variables contribute to the regression of complex data sets. The method is based on the visualization of trajectories using so-called pseudo samples representing the original variables in the data. We test and illustrate the proposed method to several synthetic and real benchmark data sets. The results show that for linear and non-linear regression models the important variables were identified with corresponding linear or non-linear trajectories. The results were verified by comparing with ordinary PLS regression and by selecting those variables which were indicated as important and rebuilding a model with only those variables.

Extremal optimization for optimizing kernel function and its parameters in support vector regression

The performance of the support vector regression (SVR) model is sensitive to the kernel type and its parameters. The determination of an appropriate kernel type and the associated parameters for SVR is a challenging research topic in the field of support vector learning. In this study, we present a novel method for simultaneous optimization of the SVR kernel function and its parameters, formulated as a mixed integer optimization problem and solved using the recently proposed heuristic ‘extremal optimization (EO)’. We present the problem formulation for the optimization of the SVR kernel and parameters, the EO-SVR algorithm, and experimental tests with five benchmark regression problems. The results of comparison with other traditional approaches show that the proposed EO-SVR method provides better generalization performance by successfully identifying the optimal SVR kernel function and its parameters.

Exploring Stock Market Dynamism in Multi-nations with Genetic Algorithm, Support Vector Regression, and Optimal Technical Analysis

in this research, an approach in combination with support vector regression (SVR), genetic algorithm (GA), and optimal technical analysis is proposed to explore stock dynamism of multi-nations under different economical environments. First, we apply full search algorithm to select the optimal number of trading days used to calculate the technical indicator values. Genetic algorithm is then used to search the best combination of parameters for SVR kernel function and technical indicators used as SVR input variables. Finally, support vector regression is then used to classify stock data based on the characteristics of non-linear classification. Also, we apply sliding windows to training data to build a steady stock exploratory approach. The data sources are stock data from four countries with different economic development degree. They include United States of America, Singapore, Taiwan, and Indonesia. In empirical results, the input variables of middle-long-term technical indicators can bring stable profits and developed country shows better efficient market.

Application of kernel principal component analysis and support vector regression for reconstruction of cardiac transmembrane potentials

Non-invasively reconstructing the transmembrane potentials (TMPs) from body surface potentials (BSPs) constitutes one form of the inverse ECG problem that can be treated as a regression problem with multi-inputs and multi-outputs, and which can be solved using the support vector regression (SVR) method. In developing an effective SVR model, feature extraction is an important task for pre-processing the original input data. This paper proposes the application of principal component analysis (PCA) and kernel principal component analysis (KPCA) to the SVR method for feature extraction. Also, the genetic algorithm and simplex optimization method is invoked to determine the hyper-parameters of the SVR. Based on the realistic heart–torso model, the equivalent double-layer source method is applied to generate the data set for training and testing the SVR model. The experimental results show that the SVR method with feature extraction (PCA-SVR and KPCA-SVR) can perform better than that without the extract feature extraction (single SVR) in terms of the reconstruction of the TMPs on epi- and endocardial surfaces. Moreover, compared with the PCA-SVR, the KPCA-SVR features good approximation and generalization ability when reconstructing the TMPs.

Soft Sensor Modeling via Support Vector Regression with KICA-Based Feature Extraction

A novel two-level integrated soft sensor modeling method using kernel independent component analysis (KICA) and support vector regression (SVR) is proposed for chemical processes. In the first level, the KICA approach is adopted to extract information of input variables in the high dimensional feature space. Based on this strategy, the correlation of input variables can be eliminated and thus the complexity is reduced. Then, the model is established using SVR in the second level. The KICA-SVR soft sensor modeling method is applied to estimate product compositions in the Tennessee Eastman process. The obtained results show that it can exhibit better performance, compared to the traditional ICA, principal component analysis (PCA) and kernel PCA based information extraction methods, under different operating conditions.

Efficient orthogonalization in gas sensor arrays using reciprocal kernel support vector regression

In this paper support vector regression is presented, and it is used to model the responses of metal oxide gas sensors to combustion byproducts. A new version of the reciprocal kernel is presented for use in the regression, and it is tested in multiple dimensions. The orthogonality of the sensors is also calculated to determine if the sensors are suitable for use in arrays. A fast numerical approximation of the sensor orthogonality, which takes advantage of the reciprocal kernel, is presented as a way of quickly optimizing the effective response of large arrays. Comparison reveals advantages over standard approaches like principal component analysis.

On finite Newton method for support vector regression

In this paper, we propose a Newton iterative method of solution for solving an e-insensitive support vector regression formulated as an unconstrained optimization problem. The proposed method has the advantage that the solution is obtained by solving a system of linear equations at a finite number of times rather than solving a quadratic optimization problem. For the case of linear or kernel support vector regression, the finite termination of the Newton method has been proved. Experiments were performed on IBM, Google, Citigroup and Sunspot time series. The proposed method converges in at most six iterations. The results are compared with that of the standard, least squares and smooth support vector regression methods and of the exact solutions clearly demonstrate the effectiveness of the proposed method.

Reducing samples for accelerating multikernel semiparametric support vector regression

In this paper, the reducing samples strategy instead of classical ν -support vector regression ( ν -SVR), viz. single kernel ν -SVR, is utilized to select training samples for admissible functions so as to curtail the computational complexity. The proposed multikernel learning algorithm, namely reducing samples based multikernel semiparametric support vector regression (RS-MSSVR), has an advantage over the single kernel support vector regression (classical ε -SVR) in regression accuracy. Meantime, in comparison with multikernel semiparametric support vector regression (MSSVR), the algorithm is also favorable for computational complexity with the comparable generalization performance. Finally, the efficacy and feasibility of RS-MSSVR are corroborated by experiments on the synthetic and real-world benchmark data sets.