Why Government and Non-Governmental Policies and Projects Fail Despite ‘Evaluations’: An Indicator to Measure whether Evaluation Systems Incorporate the Rules of Good Governance

Abstract

Background: While much has been written in the evaluation literature on the theory of evaluations and on specific cases, there is still no comprehensive and easy to use indicator that can be used to hold organizations to the principles of effective evaluation, to score their quality in several areas, and to offer an immediate diagnostic for improvements. Purpose: The article offers an easy-to-use indicator for measuring whether organizational evaluation policies for government and non-governmental organization spending actually protect the public interest in line with good governance and management principles or whether they serve, instead, to protect bureaucracies and hide wrongdoing under the cover of an “outside” evaluation. Setting: The primary focus of the piece is on international development evaluations but the author shows how the same indicator and model can be used for other government agencies as well as businesses, with modifications. Intervention: The article examines failures of evaluation systems in light of the principles for quality and shows how an indicator can be used to measure and prevent those failures. Research Design: The piece defines the principles of evaluation systems and accountability using both the frameworks of international agencies, themselves, and professional texts, places them in a model framework for effective evaluations, and then turns this framework into a set of questions to derive an indicator. Data Collection and Analysis: The article offers a sample detailed test of the indicator using the United Nations Development Programme (UNDP) as a case study. Several other organizations of different types are also tested briefly to yield a variety of results on the quality of their evaluation systems. Findings: Use of this indicator on several organizations including those of the UN system, the EC, U.S.A.I.D., reveals that a number of governmental agencies and contractors (and particularly in the field of international development) are actually failing to protect the public interest and are using evaluation processes as tools to cover up abuses and mistakes and to advocate or advertise for more funding.  Keywords: evaluation, international development, transparency, accountability, monitoring, governance