Errors in a paper on the Millennium Villages project – Editors' reply

Abstract

The Millennium Villages project is an ambitious decade-long effort to find solutions to predicaments facing millions of people living in extreme poverty. Conceived as a means to harness multiple interventions—across health, education, and agriculture, among other sectors—in the quest to improve the lives and livelihoods of these most disadvantaged populations, the project has attracted considerable attention and controversy. That controversy has continued with the recent publication of an interim analysis of Millennium Villages by Paul Pronyk and colleagues.1Pronyk PM Muniz M Nemser B for the Millennium Villages Study Group et al.The effect of an integrated multisector model for achieving the Millennium Development Goals and improving child survival in rural sub-Saharan Africa: a non-randomised controlled assessment.Lancet. 2012; (published online May 8.)https://doi.org/10.1016/S0140-6736(12)60207-4Summary Full Text Full Text PDF Scopus (56) Google Scholar Almost immediately after the paper was published online on May 8, criticism appeared suggesting that the research team's study design and analysis were weak, and that they were less than transparent with their data and methods.2Editorial With transparency comes trust.Nature. 2012; 485: 147Google Scholar The project will address these criticisms in the fora where they were initially made. Several of these criticisms are misinformed. But in The Lancet, the project team now corrects one error that did appear in the published article. They are withdrawing the comparison they made between reductions in under-5 mortality in Millennium Villages and national rural trends in child mortality. The Millennium Villages project team has quickly and commendably corrected the record after understanding the validity of the challenge it received. But the withdrawal of this element of the paper does not detract from the larger result—namely, that after 3 years Millennium Villages saw falls in poverty, food insecurity, stunting, and malaria parasitaemia, together with increases in access to safe water and sanitation. In addition, this correction does not change the finding that under-5 mortality rates were significantly lower in Millennium Village sites than their baseline rates and those of matched comparison sites. As was made clear in the accompanying Comment by Grace Malenga and Malcolm Molyneux,3Malenga G Molyneux M The Millennium Villages project.Lancet. 2012; (published online May 8.)https://doi.org/10.1016/S0140-6736(12)60369-9Summary Full Text Full Text PDF PubMed Scopus (4) Google Scholar these interim findings should be judged with appropriate caution. The comparison between Millennium and matched villages was not randomised, and so no causal attribution of measured changes can be conclusively ascribed to the interventions. That said, the results presented, in the words of one of the original reviewers of the paper, are “encouraging evidence—a big intervention rapidly leading to measurable results.” And, in fairness to those attempting to evaluate the effects of complex health programmes, non-randomised comparisons do have the power to illuminate—one example being the retrospective non-randomised assessment of UNICEF's Accelerated Child Survival and Development programme in west Africa.4Bryce J Gilroy K Jones G The Accelerated Child Survival and Development programme in west Africa: a retrospective evaluation.Lancet. 2010; 375: 572-582Summary Full Text Full Text PDF PubMed Scopus (66) Google Scholar The next phase of the Millennium Villages project will involve the monitoring of actual vital events (instead of relying on recall). To ensure that all future data from the project are fully and fairly evaluated, Prof Jeffrey Sachs, the Principal Investigator of the Millennium Villages project, is establishing new internal and external oversight procedures, including the creation of an International Scientific Expert Advisory Group, chaired by Prof Robert Black, Chairman of the Department of International Health, Johns Hopkins Bloomberg School of Public Health, which will report to the Principal Investigator and also communicate its findings to The Lancet. The goal is to provide a further independent means of verifying the quality of the project's design and analysis. It is important that this work, which is of considerable significance for understanding how countries scale up multiple complex interventions across sectors, receives proper scientific evaluation before, during, and after publication. Errors in a paper on the Millennium Villages projectIn our paper on the effect of an integrated multisector model for achieving the Millennium Development Goals and improving child survival in rural sub-Saharan Africa (published online May 8),1 we made some erroneous statements and assumptions. Full-Text PDF The effect of an integrated multisector model for achieving the Millennium Development Goals and improving child survival in rural sub-Saharan Africa: a non-randomised controlled assessmentAn integrated multisector approach for addressing the MDGs can produce rapid declines in child mortality in the first 3 years of a long-term effort in rural sub-Saharan Africa. Full-Text PDF