Improving causal inference from naturalistic designs: Self-selection and other issues in evaluating alcohol control policies

Abstract

The capacity of naturalistic research designs to contribute to causal understanding is assessed in relation to evaluating alcohol control policies. Recent conceptual writings on evaluation research emphasize the importance of causal fields and complex models in making probabilistic causal inferences. Several new statistical approaches that promise help in drawing generalizable causal inferences from suitably developed naturalistic and quasi-experimental studies are discussed. Structural equation modeling augmented by automated heuristic search procedures, econometric techniques for selection bias adjustment, and theory-driven outcome pattern analysis are considered. Two examples from alcohol control policy evaluations are given. One involved use of selectivity adjustments to evaluate a highly selectivity-conditioned environmentally-based alcohol abuse prevention program. Many naturalistic evaluations are improved by modeling self-selected program exposure as well as program response. A second evaluation, now in progress, assesses impacts of the U.S. national alcohol warning labels policy. Problems facing the investigation are enumerated, highlighting the need for theorizing about mechanisms of influence, developing multiple lines of evidence, and examining patterns of outcomes for fit with theoretical expectations.