How to evaluate the implementation of complex health programmes in low-income settings: the approach of the Gavi Full Country Evaluations.

Abstract

Vaccination, like most other public health services, relies on a complex package of intervention components, functioning systems and committed actors to achieve universal coverage. Despite significant investment in immunization programmes, national coverage trends have slowed and equity gaps have grown. This paper describes the design and implementation of the Gavi Full Country Evaluations, a multi-country, prospective, mixed-methods approach whose goal was to monitor and evaluate processes, inputs, outputs and outcomes of immunization programmes in Bangladesh, Mozambique, Uganda and Zambia. We implemented the Full Country Evaluations from 2013 to 2018 with the goal of identifying the drivers of immunization programme improvement to support programme implementation and increase equitable immunization coverage. The framework supported methodological and paradigmatic flexibility to respond to a broad range of evaluation and implementation research questions at global, national and cross-country levels, but was primarily underpinned by a focus on evaluating processes and identifying the root causes of implementation breakdowns. Process evaluation was driven by theories of change for each Gavi funding stream (e.g. Health Systems Strengthening) or activity, ranging from global policy development to district-level programme implementation. Mixing of methods increased in relevance and rigour over time as we learned to build multiple methods into increasingly tailored evaluation questions. Evaluation teams in country-based research institutes increasingly strengthened their level of embeddedness with immunization programmes as the emphasis shifted over time to focus more heavily on the use of findings for programme learning and adaptation. Based on our experiences implementing this approach, we recommend it for the evaluation of other complex interventions, health programmes or development assistance.