Methodological challenges in evaluating community partnerships & coalitions: Still crazy after all these years

Abstract

The federal Center for Substance Abuse Prevention has made a huge investment in community-based substance abuse prevention. Over the past decade it has provided funding for nearly four hundred community partnerships and coalitions. This article describes a series of methodological challenges in evaluating such programs and in assessing their key processes and outcomes. Evaluation designs face the challenge of an ever-changing array of interventions and the unavailability of traditional no-treatment control groups for testing the effectiveness of these community-wide interventions. Assessment approaches must contend with the often poor, or at least under-specified, connections between the immediate outcomes of the community interventions and the ultimately desired impact of reduced substance abuse. Reporting strategies must forego researchers' penchants for over-analyzing data in favor of getting the information into the hands of practitioners who can use it. A common theme in the resolution of these issues is the need for evaluation professionals to move away from the traditional objective detachment often ascribed to the evaluation enterprise. In the spirit of this prevention approach itself, evaluators must become partners to prevention professionals, adapting their designs, assessment techniques, and reporting strategies to fit the local context and needs. © 2000 John Wiley & Sons, Inc.