Towards rigorous practice: A framework for assessing naturalistic evaluations in the development sector

Abstract

This article reflects on the design and application of a framework for assessing the rigour of naturalistic evaluations in the development sector. The framework was employed in a review of a naturalistic evaluation ‘model’. It focused on the quality of evaluation results generated by the application of this model to outcome and impact assessments of development programmes. Naturalistic evaluation research is frequently portrayed as lacking in rigour, so we wanted to know whether the outcomes of development interventions could be assessed with an appropriate level of rigour when a naturalistic evaluation approach is used. The assessment entailed a meta-evaluation of five outcome/impact evaluations; these were undertaken by a Cape Town-based development consultancy over a four-year period. Using Lincoln and Guba’s trustworthiness criteria as the foundation for a meta-evaluation framework, we demonstrated that hallmarks of rigorous practice could be developed and applied to evaluations that employ a naturalistic evaluation model.