Bayesian Support Vector Regression Model Research Articles

Bayesian Support Vector Regression for Reliability-Based Design Optimization

Solving reliability-based design optimization (RBDO) by combining surrogate models is a powerful tool to deal with the output variation induced by uncertainties during actual engineering design. This paper aims to develop a strategy for solving RBDO problems by support vector regression (SVR) under the Bayesian inference, referred to as Bayesian SVR (BSVR). The BSVR model possesses the features of original SVR as well as providing the prediction variance to direct the sequential sampling process for probabilistic analysis in RBDO. In the meanwhile, a learning function combined with the sample pool truncation strategy is proposed to select the new training samples to adaptively update the BSVR model, which can accurately and efficiently approximate the important probabilistic constraint boundaries in the desired regions, and the newly selected training samples tend to be far away from the existing training points under the current design to avoid clustering. The computational capability and the good engineering applicability of the proposed method are verified by three numerical examples and two engineering applications about a stiffened rib of the wing edge and a latch-lock mechanism of the cabin hatch.

Adaptive Bayesian support vector regression model for structural reliability analysis

In this paper, Bayesian support vector regression (SVR) model is developed for structural reliability analysis adaptively. Two SVR models, namely, least-square SVR and ε-SVR, are constructed under the Bayesian inference framework with a square loss function and a ε-insensitive square one respectively. In this framework, a Gaussian process prior is assigned to the regression function, and maximum posterior estimate results in a SVR problem. The proposed Bayesian SVR models provide point-wise probabilistic prediction while keeps the structural risk minimization principle, and it allows us to determine the optimal hyper-parameters by maximizing Bayesian model evidence. Two active learning algorithms are presented based on the Bayesian SVR models to estimate large and small failure probability of complex structure with limited model evaluations respectively. Four benchmark examples are employed to validate the performance of the presented method.

Active learning Bayesian support vector regression model for global approximation

Machine learning techniques have received much attention in many areas for regression and classification tasks. In this paper, two new support vector regression (SVR) models, namely, least-square SVR and ε-SVR, are developed under the Bayesian inference framework with a square loss function and a ε-insensitive squared one respectively. In this framework, a Gaussian process prior is assigned to the regression function, and maximum posterior estimate of this function results in a support vector regression problem. The proposed method provides point-wise probabilistic prediction while keeps the structural risk minimization principle, and it allows us to determine the optimal hyper-parameters by maximizing Bayesian model evidence. Based on the Bayesian SVR model, an active learning algorithm is developed, and new training points are selected adaptively based on a learning function to update the SVR model progressively. Numerical results reveal that the developed two Bayesian SVR models are very promising for constructing accurate regression model for problems with diverse characteristics, especially for medium and high dimensional problems.

Modeling the viscosity of nanofluids using artificial neural network and Bayesian support vector regression

This study demonstrates the application of artificial neural networks (ANNs) and Bayesian support vector regression (BSVR) models for predicting the relative viscosity of nanofluids. The study examined 19 nanofluids comprising 1425 experimental datasets that were randomly split in a ratio of 70:30 as a training dataset and a testing dataset, respectively. To establish the inputs that will yield the best model prediction, we conducted a systematic analysis of the influence of volume fraction of nanoparticles, the density of nanoparticles, fluid temperature, size of nanoparticles, and viscosity of base fluids on the relative viscosity of the nanofluids. Also, we analyzed the results of all possible input combinations by developing 31 support vector regression models based on all possible input combinations. The results revealed that the exclusion of the viscosity of the base fluids (as a model input) leads to a significant improvement in the model result. To further validate our findings, we used the four inputs—volume fraction of nanoparticles, the density of nanoparticles, fluid temperature, and size of nanoparticles to build an ANN model. Based on the 428 testing datasets, the BSVR and ANN predicted the relative viscosity of nanofluids with an average absolute relative deviation of 3.22 and 6.64, respectively. This indicates that the BSVR model exhibits superior prediction results compared to the ANN model and existing empirical models. This study shows that the BSVR model is a reliable approach for the estimation of the viscosity of nanofluids. It also offers a generalization ability that is much better than ANN for predicting the relative viscosity of nanofluids.

Predicting the specific heat capacity of alumina/ethylene glycol nanofluids using support vector regression model optimized with Bayesian algorithm

Nanofluids are now considered the most essential constituent of solar thermal collector due to their superior thermal performance over conventional fluids. An accurate determination of the thermal efficiency of the solar collector depends on the value of the specific heat capacity of the nanofluid. So far, limited attention has been devoted towards accurate modelling of specific heat capacity of nanofluids CPnf despite their relevance in many solar energy-related applications. Surprisingly, there are only two main analytic models for estimating the CPnf in the literature. In most of the reports, these models have shown considerable inconsistencies for predicting the values of CPnf. Moreover, the modelling performance of these models necessitates the need to develop accurate models for the prediction of CPnf. Herein, a Bayesian support vector regression (BSVR) model is proposed to estimate the specific heat capacity of Al2O3/ethylene glycol nanofluid. The model proposed was trained on eighty-four (84) experimental datasets and its predictive accuracy was validated on seventeen (17) new test set. The BSVR model exhibits high accuracy as measured by the values of Pearson's correlation coefficient and the absolute average relative deviation (AARD) of 99.95% and 0.1888, respectively. Remarkably, the accuracy obtained from the proposed BSVR model is an order of magnitude better than existing theoretical models. The proposed technique and model will be useful towards a more reliable and accurate computation of the efficiency of solar collectors.

Computationally Efficient Multi-Fidelity Bayesian Support Vector Regression Modeling of Planar Antenna Input Characteristics

Bayesian support vector regression (BSVR) modeling of planar antennas with reduced training sets for computational efficiency is presented. Coarse-discretization electromagnetic (EM) simulations are exploited in order to find a reduced number of fine-discretization training points for establishing a high-fidelity BSVR model of the antenna. As demonstrated using three planar antennas with different response types, the proposed technique allows substantial reduction (up to 48%) of the computational effort necessary to set up the fine-discretization training data sets for the high-fidelity models with negligible loss in predictive power. The accuracy of the reduced-data BSVR models is confirmed by their successful use within a space mapping optimization/design algorithm.

Cost-effective global surrogate modeling of planar microwave filters using multi-fidelity bayesian support vector regression

A computationally efficient method is presented for setting up accurate Bayesian support vector regression (BSVR) models of the highly nonlinear |S21| responses of planar microstrip filters using substantially reduced finely discretized training data (compared to traditional design of experiments techniques). Inexpensive coarse-discretization full-wave simulations are exploited in conjunction with the sparseness property of BSVR to identify the regions of the input space requiring denser sampling. The proposed technique allows for substantial reduction (by up to 51%) of the computational expense necessary to collect the finely discretized training data, with negligible loss in predictive accuracy. The accuracy of the reduced-data BSVR models is confirmed by their use within a space mapping optimization algorithm. © 2013 Wiley Periodicals, Inc. Int J RF and Microwave CAE 24:11–17, 2014.

Bayesian multiple response kernel regression model for high dimensional data and its practical applications in near infrared spectroscopy

Non-linear regression based on reproducing kernel Hilbert space (RKHS) has recently become very popular in fitting high-dimensional data. The RKHS formulation provides an automatic dimension reduction of the covariates. This is particularly helpful when the number of covariates (p) far exceed the number of data points. In this paper, we introduce a Bayesian nonlinear multivariate regression model for high-dimensional problems. Our model is suitable when we have multiple correlated observed response corresponding to same set of covariates. We introduce a robust Bayesian support vector regression model based on a multivariate version of Vapnik’s ϵ-insensitive loss function. The likelihood corresponding to the multivariate Vapnik’s ϵ-insensitive loss function is constructed as a scale mixture of truncated normal and gamma distribution. The regression function is constructed using the finite representation of a function in the reproducing kernel Hilbert space (RKHS). The kernel parameter is estimated adaptively by assigning a prior on it and using the Markov chain Monte Carlo (MCMC) techniques for computation.Practical applications of our model are demonstrated via applications in near-infrared (NIR) spectroscopy and simulation studies. Our Bayesian kernel models are highly accurate in predicting composition of materials based on its near infrared (NIR) spectroscopy signature. We have compared our method with popularly used methodologies in NIR spectroscopy, like partial least square (PLS), principal component regression (PCA), support vector machine (SVM), Gaussian process regression (GPR), and random forest (RF). In all the simulation and real case studies, our multivariate Bayesian RKHS regression model outperforms the standard methods by a substantially large margin. The implementation of our models based on MCMC is fairly fast and straight forward.

Bayesian nonlinear regression for large [formula omitted] small [formula omitted] problems

Statistical modeling and inference problems with sample sizes substantially smaller than the number of available covariates are challenging. This is known as large p small n problem. Furthermore, the problem is more complicated when we have multiple correlated responses. We develop multivariate nonlinear regression models in this setup for accurate prediction. In this paper, we introduce a full Bayesian support vector regression model with Vapnik’s ϵ-insensitive loss function, based on reproducing kernel Hilbert spaces (RKHS) under the multivariate correlated response setup. This provides a full probabilistic description of support vector machine (SVM) rather than an algorithm for fitting purposes. We have also introduced a multivariate version of the relevance vector machine (RVM). Instead of the original treatment of the RVM relying on the use of type II maximum likelihood estimates of the hyper-parameters, we put a prior on the hyper-parameters and use Markov chain Monte Carlo technique for computation. We have also proposed an empirical Bayes method for our RVM and SVM. Our methods are illustrated with a prediction problem in the near-infrared (NIR) spectroscopy. A simulation study is also undertaken to check the prediction accuracy of our models.