Robust LS-SVR based on variational Bayesian and its applications

Abstract

Outliers often exist in the data for modeling in actual industrial processes. If these outliers are used as support vectors, the obtained Support Vector Regression function maybe unreliable. In this paper, we propose a new Robust Least Squares Support Vector Regression based on variational Bayesian (RB-LSSVR). The main idea of RB-LSSVR is to learn the parameters of LSSVR in Bayesian framework, but replace the Gaussian distribution with Student's t-distribution as the probability density function of residuals of the model output and real output, which makes the model more robust to outliers. In order to solve RB-LSSVR, the Student's t-distribution is written as a scale-mixture form and variational approximation is used to iteratively learn the parameters of RB-LSSVR. The hyperparameters of the Gamma distribution that can't be solved explicitly are optimized by using Newton method. And, by using variational Bayesian, the user-specified parameters selection is simplified in RB-LSSVR. The numerical results based on several benchmark regression problems and one actual industrial modeling problem show the proposed RB-LSSVR can handle outliers very well.