Methodological Considerations for Use of Routine Health Information System Data to Evaluate Malaria Program Impact in an Era of Declining Malaria Transmission.

Ruth A Ashton,Joshua Yukich,Joseph Keating,Achuyt Bhattarai,Thomas P Eisele,Adam Bennett

doi:10.4269/ajtmh.16-0734

Abstract

.Coverage of malaria control interventions is increasing dramatically across endemic countries. Evaluating the impact of malaria control programs and specific interventions on health indicators is essential to enable countries to select the most effective and appropriate combination of tools to accelerate progress or proceed toward malaria elimination. When key malaria interventions have been proven effective under controlled settings, further evaluations of the impact of the intervention using randomized approaches may not be appropriate or ethical. Alternatives to randomized controlled trials are therefore required for rigorous evaluation under conditions of routine program delivery. Routine health management information system (HMIS) data are a potentially rich source of data for impact evaluation, but have been underused in impact evaluation due to concerns over internal validity, completeness, and potential bias in estimates of program or intervention impact. A range of methodologies were identified that have been used for impact evaluations with malaria outcome indicators generated from HMIS data. Methods used to maximize internal validity of HMIS data are presented, together with recommendations on reducing bias in impact estimates. Interrupted time series and dose-response analyses are proposed as the strongest quasi-experimental impact evaluation designs for analysis of malaria outcome indicators from routine HMIS data. Interrupted time series analysis compares the outcome trend and level before and after the introduction of an intervention, set of interventions or program. The dose-response national platform approach explores associations between intervention coverage or program intensity and the outcome at a subnational (district or health facility catchment) level.

Highlights

With a renewed interest in achieving malaria elimination and funding available from a variety of sources for malaria control, many malaria endemic countries have successfully increased coverage of malaria prevention and control interventions as part of their national strategic plans.[1]
Objections to use of health management information system (HMIS) data are largely due to potential for measurement error and confounding in malaria burden estimates generated from HMIS data, and concern that these limitations may bias subsequent impact estimates.[10–12]
We review the literature to describe malaria impact evaluation designs where the primary outcome of malaria burden was generated using routine HMIS data

Summary

BACKGROUND

With a renewed interest in achieving malaria elimination and funding available from a variety of sources for malaria control, many malaria endemic countries have successfully increased coverage of malaria prevention and control interventions as part of their national strategic plans.[1]. Objections to use of HMIS data are largely due to potential for measurement error and confounding in malaria burden estimates generated from HMIS data, and concern that these limitations may bias subsequent impact estimates.[10–12] This concern was valid before the expansion of malaria diagnostic testing that has occurred in the past 5 years; when most HMIS data from high burden countries were based solely on clinical cases without confirmation. The majority of evaluations assessing impact of malaria interventions applied under routine conditions by national malaria control programs have used population-based, nationally representative crosssectional survey data (e.g., Malaria Indicator Survey [MIS]) to assess the association between malaria interventions and malaria outcomes.[13–15] These data are intermittently gathered after relatively long periods of 2–3 years, limiting our ability to assess impact on longitudinal trends in indicators of malaria morbidity. Cross-cutting methodological issues such as bias and confounding are discussed in relation to generation of outcome estimates from HMIS data, as well as in generation of impact estimates using various analysis approaches

METHODS

CONCLUSIONS