Finding the common thread: The charms and challenges of evaluation for a consortium-run international development program

Abstract

This article explores the opportunities and challenges in designing and implementing a monitoring and evaluation (M&E) framework for a consortium-run, multi-country international development program. The East Asia Vision Program (EAVP), a three-year program of the Vision 2020 Australia Global Consortium, aims to improve capacity in the delivery of eye health and vision care services in Vietnam, Cambodia, and Timor-Leste. Funded by the Australian Government, the EAVP began in 2013 and consists of five Australian organisations working with government and other stakeholders in each country. The Consortium members have been working in collaboration to support national government planning, health professional training, patient treatment, and research capacity through monitoring and evaluation. An M&E Reference Group helped with initial drafts of an M&E framework. Australian and in-country staff were then consulted to ensure availability of data and understanding of the Framework. Early drafts of data entered into the framework were shared between the participating organisations and the evaluator in order to detect errors and share good practice examples using online program management software. Monitoring the program provided an opportunity for Consortium members to improve program implementation and strengthen their capacity for M&E, by sharing examples of evaluation tools and expertise in the monitoring of cross-cutting issues and data collection. The combined work of the organisations within each country provided a rich dataset of outcomes at a health systems strengthening level. Challenges faced during the evaluation included: aligning the evaluation systems of all organisations (including data that were feasible to collect); monitoring a large range of activities; and developing an evaluation tool that was usable by a diverse range of staff. This article reports the perspectives of the M&E Advisor for EAVP and those of a program implementer to share learnings regarding the M&E for a consortium-run program. Consortia are used globally to implement international development programs and they present a unique challenge for evaluators. These consortia also provide the opportunity to build the monitoring and evaluation capacity of participating organisations, leading to improved data quality and better-informed program implementation.