Bayesian Additive Regression Tree for Seemingly Unrelated Regression with Automatic Tree Selection

Abstract

Abstract In this chapter, we propose a flexible Bayesian regression tree model when the response variable is a vector and the components of the vector are highly correlated. Our proposed Bayesian additive tree model can jointly model the correlation structure among the related response variables and provide a highly flexible and nonlinear regression structure for each of the individual regression functions. The number of trees in our multivariate Bayesian additive regression tree (seemingly unrelated regression) model (BART-SUR) is selected adaptively by treating it as a model parameter and assigning a prior distribution on it. We have designed an efficient Bayesian backfitting algorithm with reversible jump MCMC for our BART-SUR model. Our BART-SUR can jointly model the correlated response vector and at the same time it can adaptively select the number of trees required to fit the data set. The adaptive tree selection makes our model extremely fast and efficient. We demonstrate the superiority of our BART-SUR model over several out of the shelve popular methods like random forest, neural network, wavelet regression, and support vector machine through two simulation studies and three real data applications.