Refining (Not Narrowing) Notions of Program Sustainability

Abstract

The recent research agenda set forth by Scheirer and Dearing1 on program sustainability may serve as an important step toward clarifying conceptualizations of the lasting effects of public health interventions. A concern arises, however, in defining sustainability as “continued use of program components and activities for the continued achievement of desirable program and population outcomes.”1(p2060) Linking continued activities with ongoing intended outcomes may direct attention and resources away from exploring the full range of enduring program effects. Program outcomes are not only the result of continuing program activities nor are they only desirable or intended. Public health programs have lasting intended and unintended outcomes2a across multiple levels of an implementation system2b that are at least the result of program-setting interactions over time. The seemingly straightforward notion of a “continued program activity” belies complex and meaningful variation; activities can vary in the degree to which they are continued as well as in the degree to which they are considered program-related.3,4 It may be possible to benefit, therefore, from studying the long-term effects of discontinued program activities in addition to the continued intended or modified activities the authors mention. An excellent example of unintended positive effects is provided by research into the continuation of Drug Abuse Resistance Education (D.A.R.E.) in many schools across the country despite evaluations showing failure to achieve its intended effects.5,6 The explanation for this continuation is three-fold: (1) implementing schools had lower expectations for the program than did its developers; (2) local school staff perceived that D.A.R.E. helped students build relationships with their families, as well as with local law enforcement officials; and (3) personal evidence was valued more highly than scientific evidence. By focusing exclusively on the intended outcomes related to youth substance use, evaluations of D.A.R.E. ignored lasting positive effects that provide insight into alternative strategies for promoting prosocial youth behavior. Finally, these effects revealed discrepant goals and values among actors that have important implications for the design and conduct of enduring public health programs. Scheirer and Dearing are to be commended for their contribution to advancing research on program sustainability; these conceptual issues warrant clarification. However, narrowing conceptualizations of program sustainability may be counterproductive at this point in time. If we are interested in understanding the full range of effects—and not just the desirable intended outcomes—of public health interventions, there may be good reasons to conceptualize intervention activity levels and ongoing outcomes as distinct but related dimensions of multidimensional program sustainability.