The Right Tool for the Job: Choosing Between Covariate-balancing and Generalized Boosted Model Propensity Scores.

Abstract

Estimating the causal effect of an exposure (vs. some control) on an outcome using observational data often requires addressing the fact that exposed and control groups differ on pre-exposure characteristics that may be related to the outcome (confounders). Propensity score methods have long been used as a tool for adjusting for observed confounders in order to produce more valid causal effect estimates under the strong ignorability assumption. In this article, we compare two promising propensity score estimation methods (for time-invariant binary exposures) when assessing the average treatment effect on the treated: the generalized boosted models and covariate-balancing propensity scores, with the main objective to provide analysts with some rules-of-thumb when choosing between these two methods. We compare the methods across different dimensions including the presence of extraneous variables, the complexity of the relationship between exposure or outcome and covariates, and the residual variance in outcome and exposure. We found that when noncomplex relationships exist between outcome or exposure and covariates, the covariate-balancing method outperformed the boosted method, but under complex relationships, the boosted method performed better. We lay out criteria for when one method should be expected to outperform the other with no blanket statement on whether one method is always better than the other.