Using logic models to facilitate comparisons of evaluation theory

Abstract

Evaluation and Program Planning 38 (2013) 33 Contents lists available at SciVerse ScienceDirect Evaluation and Program Planning jo ur n al ho m ep ag e: www .els evier .c om /lo cat e/evalp r og p lan Using logic models to facilitate comparisons of evaluation theory Marvin C. Alkin *, Anne T. Vo, Mark Hansen University of California, Los Angeles, United States 1. Monograph overview This monograph is motivated by a desire to explore further ways that studying theories of evaluation might be facilitated. We believe that theories can be deconstructed into precisely the same elements we often use to describe the theory or logic of a social or educational program. That is, there are statements concerning the conditions in which a particular approach might have utility, prescriptions concerning the activities that ought to be imple- mented, and claims or hypotheses concerning the effects of implementing these activities. Because of these similarities, we propose that those engaged in the study of evaluation and evaluation theory may enhance their efforts by utilizing some of the same tools that have proven helpful to those studying programs and program theories. One such tool is the program logic model. Many evaluators have found the development of a logic model to be a useful precursor to conducing an evaluation, as it may contribute to greater clarity about the nature of the program and inform decisions concerning the questions that might be addressed in the evaluation study. Here, we draw on the logic model framework to guide our examination of evaluation theory. We view this process as being analogous to the way that we examine program theories. By doing so, we hope to make evaluation theory more accessible and to stimulate greater interest in theory development and refinement. We believe that this volume thus promotes the much-needed development of comparative evalua- tion theory, which in turn leads to better understandings of the applicability of theory to practice given different program contexts. In this volume, we present logic models representing three theories of evaluation practice: Practical Participatory Evaluation, Values-Engaged Evaluation, and Emergent Realist Evaluation. In Chapter 1, Mark Hansen, Marvin Alkin, and Tanner Wallace describe the selection of documents for examination, mode of analysis and the development of these depictions. In the three * Corresponding author. E-mail address: alkin@gseis.ucla.edu (M.C. Alkin). 0149-7189/$ – see front matter s 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.evalprogplan.2012.03.011 subsequent chapters, Anne Vo, Lisa Dillman, and Rebecca Luskin and Timothy Ho demonstrate the use of these logic models in drawing comparisons across the theories they seek to represent. We note that Mark’s (2008) taxonomy for evaluation research provides a useful framework for conducting this examination. Therefore, the papers are organized around three of the categories Mark describes, which also align with the elements presented in the logic models. Specifically, we focus on theoretical claims concerning: evaluation context, activities, and consequences. On these topics, we discuss ways in which the models help to clarify the underlying assumptions and claims of each theory. We also examine connections between existing literature and these theories, including evidence from research on evaluation practice that supports or raises doubts concerning the logic behind the theories presented. In Chapters 5–7, the leading proponents of each of the theoretical views analyzed will comment on the analyses conducted and provide suggestions for the modifications that they might make. J. Bradley Cousins in Chapter 5 critiques the analysis as it relates to Practical Participatory Evaluation. Jennifer Greene, in Chapter 6, provides her comments about the depiction of Values-Engaged Evaluation. Finally, in Chapter 7, Melvin Mark and Gary Henry jointly present a chapter on Emergent Realist Evaluation. In Chapter 8, Robin Miller provides an analysis of the research endeavor and considers whether logic modeling is a useful way to study evaluation theory. Finally, in Chapter 9, John Gargani, examines how practitioners might benefit from viewing theorists’ logic models. Reference Mark, M. M. (2008). Building a better evidence base for evaluation theory: Beyond general calls to a framework of types of research on evaluation. In N. L. Smith & P. Brandon (Eds.), Fundamental issues in evaluation (pp. 111–134). New York, NY: Guilford Press.