Application of an organizational evaluation capacity self-assessment instrument to different organizations: Similarities and lessons learned

Abstract

Organizational evaluation capacity (EC) has received significant attention in the evaluation research literature in the past decade. Much of the focus has been on defining organizational evaluation capacity, which can be thought of as the competencies and structures required to conduct high-quality evaluation studies (capacity to do), as well as the organization's ability to integrate evaluation findings into its decision-making processes (capacity to use). This paper seeks to contribute to this growing body of knowledge through a multiple case study of EC across three different organizations (e.g., non-profit, provincial government and federal government, herein named sectors); the novelty of this particular study is that each case study is based on the use of a common measurement tool developed by Bourgeois, Toews, Whynot and Lamarche (2013). The cross-case analysis presented in the paper reveals that evaluation capacity tends to be higher, both in terms of capacity to do and capacity to use, in organizations that have developed systematic mechanisms to institute an evaluation culture within their walls. Interestingly, however, we also found that capacity to use does not first require capacity to do, as evidenced in the non-profit organization under study.