Fighting back or fighting themselves?: Community coalitions against substance abuse and their use of best practices

Abstract

Ever the optimists (if not positivists), we continue to look to research to vindicate community health strategies in relation to chronic diseases, complex lifestyle changes and social problems, as they were vindicated in earlier decades against communicable diseases. If we had vaccines to deliver against chronic diseases and substance abuse behaviors, perhaps this hopefulness would be justified. The paper by Hallfors et al. in this issue of the AJPM describes an evaluation of the “Fighting Back” substance abuse initiative supported by the Robert Wood Johnson (RWJ) Foundation. They examined the extent to which coalitions attained selected substance abuse goals in 12 “Fighting Back” communities on which they tracked a variety of indicators through much of the 1990s. They employed a prospective design with multiple comparison communities for each coalition community, quantitative and qualitative methods, and thousands of interviews and observations. Despite all these strengths, lending credibility to the authors’ conclusions, some conceptual and methodologic limitations raise the usual doubts that follow randomized and otherwise controlled community trials, especially when they attempt to isolate one component of complex interventions as a causal explanation for success or failure. Other commentaries and analyses have grappled with the methodologic limitations and pitfalls of such research for purposes of studying community-wide interventions or interventions in communities evaluated at the community level. Some have called specifically into question the limitations of “traditional scientific methodology. . .to capture the outcomes of these complex collaborative organizations” called community coalitions. Others have addressed the problem of unrealistic expectations of community strategies in the short term. Our commentary will address some of the conceptual and procedural issues in understanding community coalitions, best practices, and their evaluation in communities. These are not entirely independent of methodological issues of evaluation, but we believe that we must know what we are attempting to measure and evaluate before we can make strategic and scientific decisions on how best to do so. We will comment, then, on two aspects of the evaluation: (1) the complexity-of-interventions issue in evaluating Fighting Back and the isolation of community coalitions as the common starting point in what became widely varied programs across the communities, and (2) the inflated expectations that follow from isolating community coalitions as the independent variable.