A call for transparency in the evaluation of global maternal health projects

Abstract

In view of frequent discussions surrounding measurable effectiveness of maternal health programmes and learning from failure—as highlighted at both the Women Deliver 2016, and Global Maternal Health 2015 conferences—we wish to draw attention to how practices in global maternal health create perverse incentives to prevent learning that could potentially improve interventional approaches. A key challenge in global maternal health currently is the incongruity between successes—invariably reported at discrete programme level—and the collective lack of progress in global maternal mortality. Evaluations of numerous projects consistently suggest a preponderance of successful interventions,1Bhushan A The Muskoka Initiative and global health financing. The North-South Institute, Ottawa2014www.nsi-ins.ca/wp-content/uploads/2014/05/Muskoka-Final.pdfGoogle Scholar yet collectively 69 of 75 high-burden countries failed to achieve their Millenium Development Goal-5 targets.2WHOTrends in maternal mortality: 1990 to 2013. Estimates by WHO, UNICEF, UNFPA, The World Bank and the United Nations Population Division. World Health Organization, Geneva2014apps.who.int/iris/bitstream/10665/112682/2/9789241507226_eng.pdf?ua=1Google Scholar, 3Victora CG Requejo JH Barros AJ et al.Countdown to 2015: a decade of tracking progress for maternal, newborn, and child survival.Lancet. 2016; 387: 2049-2059Summary Full Text Full Text PDF PubMed Scopus (238) Google Scholar As project evaluators, we have encountered concerning instances in which unwelcome findings were selectively unreported, or led to contractual terminations. Similar experiences have been echoed by colleagues in diverse geographical contexts. This necessitates consideration of three points: first we must consider how to protect the independence of evaluators and prevent the silencing of important evaluation insights. The scientific community has a duty of candour. We suggest an international evaluation registry, along the lines of the International Standard Randomised Controlled Trial Number4ISRCTN Registry. BioMed Central, London2016www.isrctn.com/Google Scholar be established to increase transparency and reduce selective reporting. Second, conflicts of interest prevent funders and implementing agencies from sharing negative, yet important programmatic evaluation results. There is a need to disentangle the associations between project outcomes and individual or institutional credibility, and create safeguards for reporting of negative findings. Third, we should assess whether the current evidence base produced by project evaluations can be trusted, given this positive evaluation bias. It seems likely that the more problematic a project, the less likely we are to learn of its shortcomings. As a result, we are less likely to make substantive changes to our practice. Such goals will require a systematic and thorough exploration of the current practice of global maternal health evaluation, a strong focus on scientific independence, and for all stakeholders to take responsibility for facilitating the reporting of challenges and failures. We declare no competing interests.