Propensity Score Matching and Variations on the Balancing Test

Abstract

This paper focuses on the role of balancing tests when employing propensity score matching methods. The idea behind these tests are to check to see if observations with the same propensity score have the same distribution of observable covariates independent of treatment status. Currently, multiple versions of the balancing test exist in the literature. One troubling aspect is that different balancing tests sometimes yield different answers. This paper highlights the importance of distinguishing between balancing tests that are conducted before matching and after matching, and provides a Monte Carlo examination of four commonly employed balancing tests. We highlight the poor size properties of these commonly employed balancing tests and demonstrate how non-parametric versions of before and after matching tests provide much better test sizes. Finally, we illustrate how balancing tests are of little utility if the conditional independence assumption underlying matching estimators is not fulfilled.