Bayesian Additive Regression Trees Research Articles

MBART: Multidimensional Monotone BART

For the discovery of regression relationships between Y and a large set of p potential predictors x1,…,xp, the flexible nonparametric nature of BART (Bayesian Additive Regression Trees) allows for a much richer set of possibilities than restrictive parametric approaches. However, subject matter considerations sometimes warrant a minimal assumption of monotonicity in at least some of the predictors. For such contexts, we introduce mBART, a constrained version of BART that can flexibly incorporate monotonicity in any predesignated subset of predictors using a multivariate basis of monotone trees, while avoiding the further confines of a full parametric form. For such monotone relationships, mBART provides (i) function estimates that are smoother and more interpretable, (ii) better out-of-sample predictive performance, and (iii) less post-data uncertainty. While many key aspects of the unconstrained BART model carry over directly to mBART, the introduction of monotonicity constraints necessitates a fundamental rethinking of how the model is implemented. In particular, the original BART Markov Chain Monte Carlo algorithm relied on a conditional conjugacy that is no longer available in a monotonically constrained space. Various simulated and real examples demonstrate the wide ranging potential of mBART.

Novel Pediatric Height Outlier Detection Methodology for Electronic Health Records via Machine Learning With Monotonic Bayesian Additive Regression Trees.

To create a new methodology that has a single simple rule to identify height outliers in the electronic health records (EHR) of children. We constructed 2 independent cohorts of children 2 to 8 years old to train and validate a model predicting heights from age, gender, race and weight with monotonic Bayesian additive regression trees. The training cohort consisted of 1376 children where outliers were unknown. The testing cohort consisted of 318 patients that were manually reviewed retrospectively to identify height outliers. The amount of variation explained in height values by our model, R2 , was 82.2% and 75.3% in the training and testing cohorts, respectively. The discriminatory ability to assess height outliers in the testing cohort as assessed by the area under the receiver operating characteristic curve was excellent, 0.841. Based on a relatively aggressive cutoff of 0.075, the outlier sensitivity is 0.713, the specificity 0.793; the positive predictive value 0.615 and the negative predictive value is 0.856. We have developed a new reliable, largely automated, outlier detection method which is applicable to the identification of height outliers in the pediatric EHR. This methodology can be applied to assess the veracity of height measurements ensuring reliable indices of body proportionality such as body mass index.

A FLEXIBLE SENSITIVITY ANALYSIS APPROACH FOR UNMEASURED CONFOUNDING WITH MULTIPLE TREATMENTS AND A BINARY OUTCOME WITH APPLICATION TO SEER-MEDICARE LUNG CANCER DATA.

In the absence of a randomized experiment, a key assumption for drawing causal inference about treatment effects is the ignorable treatment assignment. Violations of the ignorability assumption may lead to biased treatment effect estimates. Sensitivity analysis helps gauge how causal conclusions will be altered in response to the potential magnitude of departure from the ignorability assumption. However, sensitivity analysis approaches for unmeasured confounding in the context of multiple treatments and binary outcomes are scarce. We propose a flexible Monte Carlo sensitivity analysis approach for causal inference in such settings. We first derive the general form of the bias introduced by unmeasured confounding, with emphasis on theoretical properties uniquely relevant to multiple treatments. We then propose methods to encode the impact of unmeasured confounding on potential outcomes and adjust the estimates of causal effects in which the presumed unmeasured confounding is removed. Our proposed methods embed nested multiple imputation within the Bayesian framework, which allow for seamless integration of the uncertainty about the values of the sensitivity parameters and the sampling variability, as well as use of the Bayesian Additive Regression Trees for modeling flexibility. Expansive simulations validate our methods and gain insight into sensitivity analysis with multiple treatments. We use the SEER-Medicare data to demonstrate sensitivity analysis using three treatments for early stage non-small cell lung cancer. The methods developed in this work are readily available in the R package SAMTx.

Robust Bayesian inference for Big Data: Combining sensor-based records with traditional survey data

Big Data often presents as massive nonprobability samples. Not only is the selection mechanism often unknown but larger data volume amplifies the relative contribution of selection bias to total error. Existing bias adjustment approaches assume that the conditional mean structures have been correctly specified for the selection indicator or key substantive measures. In the presence of a reference probability sample, these methods rely on a pseudolikelihood method to account for the sampling weights of the reference sample, which is parametric in nature. Under a Bayesian framework, handling the sampling weights is an even bigger hurdle. To further protect against model misspecification, we expand the idea of double robustness such that more flexible nonparametric methods as well as Bayesian models can be used for prediction. In particular, we employ Bayesian additive regression trees which not only capture nonlinear associations automatically but permit direct quantification of the uncertainty of point estimates through its posterior predictive draws. We apply our method to sensor-based naturalistic driving data from the second Strategic Highway Research Program using the 2017 National Household Travel Survey as a benchmark.

Correlates of cancer prevalence across census tracts in the United States: A Bayesian machine learning approach

Preventive measures, health behaviors, environmental exposures, and sociodemographic characteristics affect individual-level cancer risks. It is unclear how they influence neighborhood-level cancer risks. We developed a large-scale neighborhood health dataset for 72,337 census tracts in the United States by combining data from three publicly available sources. We used Bayesian additive regression trees to identify the most important predictors of tract-level cancer prevalence among adults (age ≥18 years), and examined their impact on cancer prevalence using partial dependence plots. The five most important census tract-level correlates of cancer prevalence were the proportion of population who were aged 65 years and older, had routine checkup and were non-Hispanic White, the proportion of houses built before 1960, and the proportion of population living below the poverty line. The identified predictors of neighborhood-level cancer prevalence may inform public health practitioners and policymakers to prioritize the improvement of environmental and neighborhood factors in reducing the cancer burden.

Uncertainty in Marine Species Distribution Modelling: Trying to Locate Invasion Hotspots for Pterois miles in the Eastern Mediterranean Sea

Biological invasions are considered among the largest threats to native biodiversity. The Mediterranean Sea, connecting the Indo-Pacific and Atlantic oceans, is characterized as a global marine invasion hotspot, due to a multitude of human pathways and vectors such as shipping, aquaculture, tourism, and the opening of the Suez Canal, which have led to the introduction of nearly 700 alien species into the Mediterranean Sea. Among the species introduced, the lionfish Pterois miles could be considered the fastest spreading invasive fish species of the last decade (2012–2022) and has been recorded in all countries of the eastern Mediterranean Sea, reaching as far north as Croatia. Here, we present a Bayesian additive regression tree modelling framework for an updated species distribution modelling invasion map under current and future climate conditions. All climate uncertainty sources have been used, as these are available from the Bio-Oracle, the unique marine predictors database. Important outputs of the current approach are the model’s inadequacy to accurately predict the most recent expansion of species in the Adriatic Sea, and the uncertainty estimation, that are high in areas with confirmed occurrence of individuals, in simulations that can help the decision makers and policy officers understand model limitations and take more informed actions.

Shrinkage Bayesian Causal Forests for Heterogeneous Treatment Effects Estimation

This article develops a sparsity-inducing version of Bayesian Causal Forests, a recently proposed nonparametric causal regression model that employs Bayesian Additive Regression Trees and is specifically designed to estimate heterogeneous treatment effects using observational data. The sparsity-inducing component we introduce is motivated by empirical studies where not all the available covariates are relevant, leading to different degrees of sparsity underlying the surfaces of interest in the estimation of individual treatment effects. The extended version presented in this work, which we name Shrinkage Bayesian Causal Forest, is equipped with an additional pair of priors allowing the model to adjust the weight of each covariate through the corresponding number of splits in the tree ensemble. These priors improve the model’s adaptability to sparse data generating processes and allow to perform fully Bayesian feature shrinkage in a framework for treatment effects estimation, and thus to uncover the moderating factors driving heterogeneity. In addition, the method allows prior knowledge about the relevant confounding covariates and the relative magnitude of their impact on the outcome to be incorporated in the model. We illustrate the performance of our method in simulated studies, in comparison to Bayesian Causal Forest and other state-of-the-art models, to demonstrate how it scales up with an increasing number of covariates and how it handles strongly confounded scenarios. Finally, we also provide an example of application using real-world data. Supplementary materials for this article are available online.

A flexible approach for variable selection in large-scale healthcare database studies with missing covariate and outcome data

BackgroundPrior work has shown that combining bootstrap imputation with tree-based machine learning variable selection methods can provide good performances achievable on fully observed data when covariate and outcome data are missing at random (MAR). This approach however is computationally expensive, especially on large-scale datasets.MethodsWe propose an inference-based method, called RR-BART, which leverages the likelihood-based Bayesian machine learning technique, Bayesian additive regression trees, and uses Rubin’s rule to combine the estimates and variances of the variable importance measures on multiply imputed datasets for variable selection in the presence of MAR data. We conduct a representative simulation study to investigate the practical operating characteristics of RR-BART, and compare it with the bootstrap imputation based methods. We further demonstrate the methods via a case study of risk factors for 3-year incidence of metabolic syndrome among middle-aged women using data from the Study of Women’s Health Across the Nation (SWAN).ResultsThe simulation study suggests that even in complex conditions of nonlinearity and nonadditivity with a large percentage of missingness, RR-BART can reasonably recover both prediction and variable selection performances, achievable on the fully observed data. RR-BART provides the best performance that the bootstrap imputation based methods can achieve with the optimal selection threshold value. In addition, RR-BART demonstrates a substantially stronger ability of detecting discrete predictors. Furthermore, RR-BART offers substantial computational savings. When implemented on the SWAN data, RR-BART adds to the literature by selecting a set of predictors that had been less commonly identified as risk factors but had substantial biological justifications.ConclusionThe proposed variable selection method for MAR data, RR-BART, offers both computational efficiency and good operating characteristics and is utilitarian in large-scale healthcare database studies.

Abstract 19: Assessing The Impact Of Stroke Systems Of Care Policies On Stroke-related Outcomes

Introduction: Stroke systems of care (SSOC) laws have been adopted across the US as early as 2002 to improve access to time- sensitive, life-saving treatments and care for stroke patients. However, the impact of these laws on stroke care delivery and patient outcomes remains largely unknown. The study goal was to determine whether the policy interventions are associated with improved stroke outcomes by examining SSOC laws implemented by states. Methods: We developed a policy surveillance dataset of the 50 states and D.C. SSOC laws in effect at any time from 2002-2018. A stakeholder group of federal, state, and local SSOC experts informed the study. Based on an evidence-informed conceptual model, the study linked the policy surveillance dataset with three state-level outcomes: 1) age-adjusted overall stroke mortality rates; 2) percentage of hospitals certified as primary stroke centers (PSC); and 3) percentage of patients who received brain scan results within 45 minutes of arrival at a stroke center. To estimate policy effectiveness, we used Bayesian Additive Regression Trees to predict outcomes as if no policies were in effect and compared them to the actual observed outcomes in each state that put at least one policy into effect. For each modeled outcome, we controlled for state-level contextual factors such as stroke prevalence, percent rural population, non-white population, density of hospital closure, Paul Coverdell National Acute Stroke Program participation, and Medicaid expansion status. Results: Results identified 38 states with at least one SSOC law in effect during the study period. Of these states, 11 states had lower and 3 had higher overall stroke mortality rates than predicted; 9 states had a higher and 4 had a lower proportion of PSCs than predicted; and 12 states had higher and 2 had lower brain scan rates within 45 minutes than predicted. Conclusion: Having one or more SSOC laws in effect is associated with improved stroke outcomes. This information can inform national chronic disease organizations and state decisionmaker efforts to improve SSOC at state and regional levels.

A rapid feature selection method for catalyst design: Iterative Bayesian additive regression trees (iBART).

Feature selection (FS) methods often are used to develop data-driven descriptors (i.e., features) for rapidly predicting the functional properties of a physical or chemical system based on its composition and structure. FS algorithms identify descriptors from a candidate pool (i.e., feature space) built by feature engineering (FE) steps that construct complex features from the system's fundamental physical properties. Recursive FE, which involves repeated FE operations on the feature space, is necessary to build features with sufficient complexity to capture the physical behavior of a system. However, this approach creates a highly correlated feature space that contains millions or billions of candidate features. Such feature spaces are computationally demanding to process using traditional FS approaches that often struggle with strong collinearity. Herein, we address this shortcoming by developing a new method that interleaves the FE and FS steps to progressively build and select powerful descriptors with reduced computational demand. We call this method iterative Bayesian additive regression trees (iBART), as it iterates between FE with unary/binary operators and FS with Bayesian additive regression trees (BART). The capabilities of iBART are illustrated by extracting descriptors for predicting metal-support interactions in catalysis, which we compare to those predicted in our previous work using other state-of-the-art FS methods (i.e., least absolute shrinkage and selection operator + l0, sure independence screening and sparsifying operator, and Bayesian FS). iBART matches the performance of these methods yet uses a fraction of the computational resources because it generates a maximum feature space of size O(102), as opposed to O(106) generated by one-shot FE/FS methods.

Association of Cardiovascular Risk Trajectory With Cognitive Decline and Incident Dementia.

Background and ObjectivesCardiovascular risk factors have a recently established association with cognitive decline and dementia, yet most studies examine this association through cross-sectional data, precluding an understanding of the longitudinal dynamics of such risk. The current study aims to explore how the ongoing trajectory of cardiovascular risk affects subsequent dementia and memory decline risk. We hypothesize that an accelerated, long-term accumulation of cardiovascular risk, as determined by the Framingham Risk Score (FRS), will be more detrimental to cognitive and dementia state outcomes than a stable cardiovascular risk.MethodsWe assessed an initially healthy, community-dwelling sample recruited from the prospective cohort Betula study. Cardiovascular disease risk, as assessed by the FRS, episodic memory performance, and dementia status were measured at each 5-year time point (T) across 20 to 25 years. Analysis was performed with bayesian additive regression tree, a semiparametric machine-learning method, applied herein as a multistate survival analysis method.ResultsOf the 1,244 participants, cardiovascular risk increased moderately over time in 60% of sample, with observations of an accelerated increase in 18% of individuals and minimal change in 22% of individuals. An accelerated, as opposed to a stable, cardiovascular risk trajectory predicted an increased risk of developing Alzheimer disease dementia (average risk ratio [RR] 3.3–5.7, 95% CI 2.6–17.5 at T2, 1.9–6.7 at T5) or vascular dementia (average RR 3.3–4.1, 95% CI 1.1–16.6 at T2, 1.5–7.6 at T5) and was associated with an increased risk of memory decline (average RR 1.4–1.2, 95% CI 1–1.9 at T2, 1–1.5 at T5). A stable cardiovascular risk trajectory appeared to partially mitigate Alzheimer disease dementia risk for APOE ε4 carriers.DiscussionThe findings of the current study show that the longitudinal, cumulative trajectory of cardiovascular risk is predictive of dementia risk and associated with the emergence of memory decline. As a result, clinical practice may benefit from directing interventions at individuals with accelerating cardiovascular risk.

Machine-learning derived algorithms for prediction of radiographic progression in early axial spondyloarthritis.

To compare machine learning (ML) to traditional models to predict radiographic progression in patients with early axial spondyloarthritis (axSpA). We carried out a prospective French multicentric DESIR cohort study with 5 years of follow-up that included patients with chronic back pain for <3 years, suggestive of axSpA. Radiographic progression was defined as progression at the spine (increase of at least 1 point of mSASSS scores/2 years) or at the sacroiliac joint (worsening of at least one grade of the mNY score between 2 visits). Statistical analyses were based on patients without any missing data regarding the outcome and variables of interest (295 patients).Traditional modelling: we performed a multivariate logistic regression model (M1); then variable selection with stepwise selection based on Akaike Information Criterion (stepAIC) method (M2), and Least Absolute Shrinkage and Selection Operator (LASSO) method (M3).ML modelling: using "SuperLearner" package on R, we modelled radiographic progression with stepAIC, LASSO, random forest, Discrete Bayesian Additive Regression Trees Samplers (DBARTS), Generalized Additive Models (GAM), multivariate adaptive polynomial spline regression (polymars), Recursive Partitioning And Regression Trees (RPART) and Super Learner. Accuracy of these models was compared based on their 10-fold cross-validated AUC (cv-AUC). 10-fold cv-AUC for traditional models were 0.79 and 0.78 for M2 and M3, respectively. The three best models in the ML algorithms were the GAM, the DBARTS and the Super Learner models, with 10-fold cv-AUC of: 0.77, 0.76 and 0.74, respectively. Two traditional models predicted radiographic progression as good as the eight ML models tested in this population.

Prediction of inpatient pressure ulcers based on routine healthcare data using machine learning methodology

Despite the relevance of pressure ulcers (PU) in inpatient care, the predictive power and role of care-related risk factors (e.g. anesthesia) remain unclear. We investigated the predictability of PU incidence and its association with multiple care variables. We included all somatic cases between 2014 and 2018 with length of stay ≥ 2d in a German university hospital. For regression analyses and prediction we used Bayesian Additive Regression Trees (BART) as nonparametric modeling approach. To assess predictive accuracy, we compared BART, random forest, logistic regression (LR) and least absolute shrinkage and selection operator (LASSO) using area under the curve (AUC), confusion matrices and multiple indicators of predictive performance (e.g. sensitivity, specificity, F1, positive/ negative predictive value) in the full dataset and subgroups. Analysing 149,006 cases revealed high predictive variable importance and associations between incident PU and ventilation, age, anesthesia (≥ 1 h) and number of care-involved wards. Despite high AUCs (range 0.89–0.90), many false negative predictions led to low sensitivity (range 0.04–0.10). Ventilation, age, anesthesia and number of care-involved wards were associated with incident PU. Using anesthesia as a proxy for immobility, an hourly repositioning is indicated. The low sensitivity indicates major challenges for correctly predicting PU based on routine data.

Increasing microsolar technology adoption: Efficacy of vouchers, cash transfers, and microfinance schemes

Solar lanterns serve as practical substitutes for kerosene as a source of lighting; however, because of liquidity and affordability concerns, the uptake of lanterns has been slow. We conducted a randomized control trial in Bahraich, Uttar Pradesh, amongst 500 households to see whether a voucher, microfinance, or cash program would be most effective. Using the Becker–Degroot–Marschak (BDM) method to elicit the respondents’ willingness to pay (WTP), we found that households receiving the voucher are willing to pay 436 INR (95% CI: 398–473) for a solar lantern (control mean = 326 INR, 95% CI = 291–361), or about 34% more than households in the control group, but still below the retail price of the lanterns. Quantile regression revealed that the voucher program has larger effects on households that are willing to pay least. Finally, we use Bayesian Additive Regression Trees (BART) to show that household wealth or lighting importance perceptions were not significant factors. These results highlight the problem of affordability as a prime obstacle and support the use of voucher-like incentive programs to encourage transitional technologies.

Adaptive Conditional Distribution Estimation with Bayesian Decision Tree Ensembles

We present a Bayesian nonparametric model for conditional distribution estimation using Bayesian additive regression trees (BART). The generative model we use is based on rejection sampling from a base model. Like other BART models, our model is flexible, has a default prior specification, and is computationally convenient. To address the distinguished role of the response in our BART model, we introduce an approach to targeted smoothing of BART models which is of independent interest. We study the proposed model theoretically and provide sufficient conditions for the posterior distribution to concentrate at close to the minimax optimal rate adaptively over smoothness classes in the high-dimensional regime in which many predictors are irrelevant. To fit our model, we propose a data augmentation algorithm which allows for existing BART samplers to be extended with minimal effort. We illustrate the performance of our methodology on simulated data and use it to study the relationship between education and body mass index using data from the medical expenditure panel survey (MEPS). Supplementary materials for this article are available online.

Evaluating ecological uniqueness over broad spatial extents using species distribution modelling

Local contributions to beta diversity (LCBD) can be used to identify sites with high ecological uniqueness and exceptional species composition within a region of interest. Yet, these indices are typically used on local or regional scales with relatively few sites, as they require information on complete community compositions difficult to acquire on larger scales. Here, we investigated how LCBD indices can be predicted over broad spatial extents using species distribution modelling and examined the effect of scale changes on beta diversity quantification. We used Bayesian additive regression trees (BARTs) to predict warbler species distributions in North America based on observations recorded in the eBird database. We then calculated LCBD indices for observed and predicted data and compared the site‐wise difference using direct comparison, a spatial association test and generalized linear regression. We also examined the relationship between LCBD values and species richness in different regions and at various spatial extents. Our results showed that species distribution models provided uniqueness estimates highly correlated with observed data. The form and variance of the LCBD–richness relationship varied according to the region and the total extent size. The relationship was also affected by the proportion of rare species in the communities. Therefore, sites identified as unique over broad spatial extents may vary according to regional characteristics. These results show that species distribution modelling can be used to predict ecological uniqueness over broad spatial extents, which could help identify beta diversity hotspots and important targets for conservation purposes in unsampled locations.

Market premia for renewables in Germany: The effect on electricity prices

Due to the growing share of “green” electricity generated by renewable energy technologies, the frequency of negative price spikes has substantially increased in Germany. To reduce such events, in 2012, a market premium scheme (MPS) was introduced as an alternative to feed-in tariffs for the promotion of green electricity. Drawing on hourly day-ahead spot prices for the time period spanning 2009 to 2016 and employing a nonparametric modeling strategy called Bayesian Additive Regression Trees, this paper empirically evaluates the efficacy of Germany’s MPS. Via counterfactual analyses, we demonstrate that the introduction of the MPS decreased the number of hours with negative prices by some 70%.

Inference in Bayesian additive vector autoregressive tree models

Vector autoregressive (VAR) models assume linearity between the endogenous variables and their lags. This assumption might be overly restrictive and could have a deleterious impact on forecasting accuracy. As a solution we propose combining VAR with Bayesian additive regression tree (BART) models. The resulting Bayesian additive vector autoregressive tree (BAVART) model is capable of capturing arbitrary nonlinear relations between the endogenous variables and the covariates without much input from the researcher. Since controlling for heteroscedasticity is key for producing precise density forecasts, our model allows for stochastic volatility in the errors. We apply our model to two datasets. The first application shows that the BAVART model yields highly competitive forecasts of the U.S. term structure of interest rates. In a second application we estimate our model using a moderately sized Eurozone dataset to investigate the dynamic effects of uncertainty on the economy.

Fair comparisons: Life course selection bias and the effect of father absence on US children

Studies have shown that father absence in opposite-gender couples has detrimental effects on children's wellbeing, net of selection bias. However, life course informed research suggests that the problem of selection bias may be more complex than currently thought. This paper shows the importance of nonparametrically adjusting for the trajectory of confounder covariates for the estimation of these effects. This paper uses data from the Fragile Families and Child Wellbeing Study to estimate the total effect of the departure of the biological father on children's wellbeing, as well as delayed or fade-out effects of this transition. The paper employs Bayesian additive regression trees, a machine learning and causal inference method suited for statistical models involving high-dimensional data sets. After adjusting for multiple time-invariant and -varying confounder covariates, as well as their history, estimates of father absence's effect on children's wellbeing are reduced substantially, a finding which may be referred to as life course selection bias. Results suggest early and middle childhood are not negatively affected by the departure of the biological father. Life course selection bias mostly affects estimates of this effect on adolescence, which is explained by children directly experiencing changes inparent's socioeconomic trajectories that lead to divorce or separation. This would not be the case when father absence is experienced in early childhood. Results suggest father absence is mostly a marker of life course cumulative socioeconomic disadvantage, not a cause of negative effects.

Targeting social safety net programs on human capabilities

• We develop an alternative method for targeting social safety net programs that uses the capabilities approach as a normative framework. • We adapt Bayesian additive regression trees for the estimation of human capabilities and use the resulting estimates as a basis for targeting. • We examine the implications of our method through a variety of simulation exercises and with real data from a field experiment in Indonesia. • Relative to traditional approaches, our method identifies a fundamentally different and arguably more disadvantaged group of beneficiaries. Conventional approaches to targeting social safety net programs select beneficiaries on the basis of income or expenditure levels. We argue that these approaches neglect human diversity and agency, which can lead to counterintuitive targeting outcomes and thus a misallocation of benefits. In light of these issues, we develop an alternative method for targeting that is based on the capabilities approach, which we claim provides a more rigorous normative framework for targeting that respects both human diversity and agency. In particular, we adapt Bayesian additive regression trees for the estimation of human capabilities and demonstrate how the resulting estimates can be used to target social safety net programs. We examine the targeting implications of our method through a variety of simulation exercises and also with real data from a field experiment conducted in Indonesia. Relative to more traditional approaches – including not only the full and proxy means test, but also community-based targeting – we find that our method identifies a fundamentally different and arguably more disadvantaged group of beneficiaries.