Meaningful Evaluation: a holistic and systemic approach to understanding and assessing outcomes

Abstract

The concept and practice of Social Impact and Social Impact assessment has developed and matured over the last decade. Despite this growth, confusion still exists with respect to definitions of social impact as well as which tools and frameworks are most appropriate. Central to the concept and practice of social impact is a focus on outcomes. This paper argues that a discernible albeit problematic pattern and approach has emerged in outcomes measurement within the social impact field. After briefly reviewing some of the key problems with how outcomes measurement is practised, it presents some recent approaches from the evaluation field that attempt to address some of these concerns before introducing a new approach to understanding and evaluating outcomes – Meaningful Evaluation (ME). A pilot Meaningful Evaluation of volunteer ethics teachers in the Primary Ethics program in NSW is used to illustrate the approach. ME combines the Map of Meaning (MoM) with insights from next generation evaluation approaches to understand and assess outcomes. Informed by appreciative inquiry perspectives, it provides a means to bridge the divide between positivist and interpretivist approaches in evaluation. Key to ME is the assumption that it is more likely that immediate outcomes lead to medium and long-term outcomes (changes in behaviour) that are sustainable and lead to impact if participants experience program interventions as ‘meaningful’. Meaning is an important internal outcome that is essential if longer-term external outcomes are to occur. It also shifts the focus to capture unintended outcomes, key to developing holistic and systemic rather than linear and mechanistic Theories of Change.