Thereʼs a Reason They Call Them Dummy Variables

Abstract

Many research designs and statistical methodologies will be used to conduct comparative effectiveness research (CER). In particular, it is almost certainly the case that the demand for real-world evidence will drive increased demand for CER analyses of observational data. Although a great deal of progress has been made in the development and application of statistical methods for the analysis of observational data, the ordinary least squares multiple regression model remains, by far, the most widely applied multivariate analysis tool. This article begins with a brief review of the interpretation of treatment effects captured through the use of dummy variables in multiple regression models. This review makes clear just how limited this typical estimator of treatment effect is. Structural equation and decomposition methods for CER analyses of observational data are then reviewed. Although these methods have not been commonly used for outcomes research, they offer the opportunity to extract significantly more information regarding treatment effects than the standard dummy variable approach. I have attempted to make the point that traditional dummy variable methods in regression models provide an extremely limited estimate of treatment effects. Structural equation models and decomposition methods provide considerably more information about treatment effects - in particular, the ability to identify how outcomes may vary differentially with respect to patient characteristics and other factors for alternative treatment cohorts. Such an understanding is fundamental to deciphering the heterogeneity of treatment response among patient subpopulations. Structural equation and decomposition methods may be further enhanced by incorporating propensity score matching prior to the analysis. On the other hand, researchers should be wary of the potential pitfalls associated with parametric sample selection bias models. Although tests for selection bias and other forms of endogeneity are an excellent research practice, it is entirely possible that attempts to correct for endogeneity may introduce more bias than they remove. Nonparametric methods, such as differences in differences, while making strong assumptions of their own, avoid the need to identify instrumental variables that are correlated with treatment selection but uncorrelated with residuals in the outcome equation.