Vitamin A supplementation in Indian children – Authors' reply

Abstract

DEVTA1Awasthi S Peto R Read S Clark S Pande V Bundy D the DEVTA (Deworming and Enhanced Vitamin A) teamVitamin A supplementation every 6 months with retinol in 1 million pre-school children in north India: DEVTA, a cluster-randomised trial.Lancet. 2013; 381: 1469-1477Summary Full Text Full Text PDF PubMed Scopus (109) Google Scholar was a well conducted cluster-randomised trial of the effects of allocation to twice-yearly vitamin A supplementation on pre-school child mortality in a population with low blood retinol, some Bitot's spots, but little xerophthalmia. It had good compliance (about 86%), unbiased assessment of outcome (mortality at ages 1–6 years), and appropriate analysis (as 36 clusters vs 36 clusters). Its effective sample size was twice that of all previous trials combined, but its findings are still subject to some random error. DEVTA should therefore be considered in conjunction with the previous trials, and they with it, as in the figure (adapted from figure 4 in the original Article).1Awasthi S Peto R Read S Clark S Pande V Bundy D the DEVTA (Deworming and Enhanced Vitamin A) teamVitamin A supplementation every 6 months with retinol in 1 million pre-school children in north India: DEVTA, a cluster-randomised trial.Lancet. 2013; 381: 1469-1477Summary Full Text Full Text PDF PubMed Scopus (109) Google ScholarThe Comment2Garner P Taylor-Robinson D Sachdev HS DEVTA: results from the biggest clinical trial ever.Lancet. 2013; 381: 1439-1441Summary Full Text Full Text PDF PubMed Scopus (5) Google Scholar and Correspondence about DEVTA do not sufficiently emphasise how much the play of chance during randomisation can affect the apparent mortality findings in trials. The result from a trial (or meta-analysis) is not only the point estimate of the mortality rate ratio but also its CI, which generally includes a much wider range of possible answers.The figure shows weighted averages of results from eight previous trials of regular vitamin A supplementation of young children in low-income countries, the DEVTA trial, and all 9 trials. From the previous trials, the point estimate of the average proportional mortality reduction was 23%, but the 99% CI ran from 11% to 32%, corresponding to a wide range of purely statistical uncertainty about the average benefit. DEVTA substantially narrows this uncertainty; the point estimate suggested by DEVTA plus previous trials is 11% (95% CI 5–16%), suggesting that the real benefit generally lies around the lower end of the previous range of uncertainty.Assessment of trial evidence should be wholly separated from advocacy, but most responses to DEVTA are from two groups of advocates. The Comment,2Garner P Taylor-Robinson D Sachdev HS DEVTA: results from the biggest clinical trial ever.Lancet. 2013; 381: 1439-1441Summary Full Text Full Text PDF PubMed Scopus (5) Google Scholar coauthored by a critic of routine vitamin A supplementation, over-emphasises the non-significant DEVTA result, somewhat downplaying the meta-analysis. Conversely, correspondents who advocate vitamin A supplementation (or work with organisations delivering it) want to downplay or ignore DEVTA, either by questioning its validity or by using random-effects meta-analyses that give inappropriately little weight to large, statistically reliable trials.3Peto R Interpreting large-scale randomised evidence. John Snow lecture.http://www.youtube.com/watch?v=vybc0PsZ718&list=PL3oyPcbygtxuqmYps-aEj9Sa-ScThzMLhDate: April 12, 2013Google Scholar, 4EBCTCGTreatment of early breast cancer: worldwide evidence, 1985–1990. Oxford University Press, Oxford1990http://www.ctsu.ox.ac.uk/research/meta-trials/ebctcg/original-methods-for-ebctcg-meta-analysesGoogle ScholarInappropriately, such random-effects meta-analyses would almost ignore any major new trial, no matter how good it was or what it showed. For example, even if DEVTA had been an impeccable individually randomised trial in tens of millions of children with 89 000 versus 100 000 deaths (an 11% risk reduction), a standard random-effects meta-analysis would still inappropriately conclude, based mainly on the far smaller previous trials, that vitamin A reduces mortality by about a quarter.Sommer and colleagues,5Sommer A West KP Martorell R Vitamin A supplementation in Indian children.Lancet. 2013; 382: 591Summary Full Text Full Text PDF PubMed Scopus (4) Google Scholar whose prompt response is cited by other correspondents, suggest that DEVTA was not a valid trial. As, however, the population studied was vitamin A deficient by conventional criteria, the key methodological issues are whether there was good compliance with the randomly allocated treatment (yes), and whether there was any material bias between the two treatment groups in ascertainment of mortality (no). All else is irrelevant to trial reliability.The direct evidence we cited of good compliance (despite claims by various correspondents) is supported by airline delivery records of 6 million Roche vitamin A capsules (1·2 million in November, 1998; October, 1999; January, 2001; December, 2001; and October, 2002; all used in DEVTA). Individual child deaths in the past year were sought from several sources at 6-monthly visits to each village, so most deaths were recorded twice; confirming relatively few were missed twice, DEVTA mortality rates slightly exceeded those recorded by the Registrar-General of India for the whole state. The low cost per child (miscalculated tenfold by Sommer and colleagues5Sommer A West KP Martorell R Vitamin A supplementation in Indian children.Lancet. 2013; 382: 591Summary Full Text Full Text PDF PubMed Scopus (4) Google Scholar) merely reflected efficiencies of design and scale.As DEVTA had surprisingly unpromising results we convened a meeting of other vitamin A trialists to discuss their studies and ours, seeking explanations for any heterogeneity. There were no agreed proceedings, but that 2008 meeting engendered for the DEVTA publication extensive further data checks and more robust statistical analysis methods; these did not, however, change the mortality rate ratio.We conclude that the apparent discrepancy between DEVTA and previous trials was probably due mainly to the play of chance (despite being significant, p=0·001). If so, it is medically and statistically appropriate for DEVTA to have twice the weight of all other trials combined when averaging trial results to guide implementation strategies.As indicated in the figure, the real risk reduction could well be about 11% both in the circumstances of DEVTA and in those of some previous trials. To assess such moderate differences reliably, large-scale randomised evidence was needed, and DEVTA has contributed substantially to this. Arguments for or against routine vitamin A supplementation should now be based on appropriate assessment of all mortality trial results, without unduly selective exclusions.3Peto R Interpreting large-scale randomised evidence. John Snow lecture.http://www.youtube.com/watch?v=vybc0PsZ718&list=PL3oyPcbygtxuqmYps-aEj9Sa-ScThzMLhDate: April 12, 2013Google ScholarView Large Image Copyright © 2013 Andy Crump, TDR, WHO/Science Photo LibraryWe declare that we have no conflicts of interest. DEVTA1Awasthi S Peto R Read S Clark S Pande V Bundy D the DEVTA (Deworming and Enhanced Vitamin A) teamVitamin A supplementation every 6 months with retinol in 1 million pre-school children in north India: DEVTA, a cluster-randomised trial.Lancet. 2013; 381: 1469-1477Summary Full Text Full Text PDF PubMed Scopus (109) Google Scholar was a well conducted cluster-randomised trial of the effects of allocation to twice-yearly vitamin A supplementation on pre-school child mortality in a population with low blood retinol, some Bitot's spots, but little xerophthalmia. It had good compliance (about 86%), unbiased assessment of outcome (mortality at ages 1–6 years), and appropriate analysis (as 36 clusters vs 36 clusters). Its effective sample size was twice that of all previous trials combined, but its findings are still subject to some random error. DEVTA should therefore be considered in conjunction with the previous trials, and they with it, as in the figure (adapted from figure 4 in the original Article).1Awasthi S Peto R Read S Clark S Pande V Bundy D the DEVTA (Deworming and Enhanced Vitamin A) teamVitamin A supplementation every 6 months with retinol in 1 million pre-school children in north India: DEVTA, a cluster-randomised trial.Lancet. 2013; 381: 1469-1477Summary Full Text Full Text PDF PubMed Scopus (109) Google Scholar The Comment2Garner P Taylor-Robinson D Sachdev HS DEVTA: results from the biggest clinical trial ever.Lancet. 2013; 381: 1439-1441Summary Full Text Full Text PDF PubMed Scopus (5) Google Scholar and Correspondence about DEVTA do not sufficiently emphasise how much the play of chance during randomisation can affect the apparent mortality findings in trials. The result from a trial (or meta-analysis) is not only the point estimate of the mortality rate ratio but also its CI, which generally includes a much wider range of possible answers. The figure shows weighted averages of results from eight previous trials of regular vitamin A supplementation of young children in low-income countries, the DEVTA trial, and all 9 trials. From the previous trials, the point estimate of the average proportional mortality reduction was 23%, but the 99% CI ran from 11% to 32%, corresponding to a wide range of purely statistical uncertainty about the average benefit. DEVTA substantially narrows this uncertainty; the point estimate suggested by DEVTA plus previous trials is 11% (95% CI 5–16%), suggesting that the real benefit generally lies around the lower end of the previous range of uncertainty. Assessment of trial evidence should be wholly separated from advocacy, but most responses to DEVTA are from two groups of advocates. The Comment,2Garner P Taylor-Robinson D Sachdev HS DEVTA: results from the biggest clinical trial ever.Lancet. 2013; 381: 1439-1441Summary Full Text Full Text PDF PubMed Scopus (5) Google Scholar coauthored by a critic of routine vitamin A supplementation, over-emphasises the non-significant DEVTA result, somewhat downplaying the meta-analysis. Conversely, correspondents who advocate vitamin A supplementation (or work with organisations delivering it) want to downplay or ignore DEVTA, either by questioning its validity or by using random-effects meta-analyses that give inappropriately little weight to large, statistically reliable trials.3Peto R Interpreting large-scale randomised evidence. John Snow lecture.http://www.youtube.com/watch?v=vybc0PsZ718&list=PL3oyPcbygtxuqmYps-aEj9Sa-ScThzMLhDate: April 12, 2013Google Scholar, 4EBCTCGTreatment of early breast cancer: worldwide evidence, 1985–1990. Oxford University Press, Oxford1990http://www.ctsu.ox.ac.uk/research/meta-trials/ebctcg/original-methods-for-ebctcg-meta-analysesGoogle Scholar Inappropriately, such random-effects meta-analyses would almost ignore any major new trial, no matter how good it was or what it showed. For example, even if DEVTA had been an impeccable individually randomised trial in tens of millions of children with 89 000 versus 100 000 deaths (an 11% risk reduction), a standard random-effects meta-analysis would still inappropriately conclude, based mainly on the far smaller previous trials, that vitamin A reduces mortality by about a quarter. Sommer and colleagues,5Sommer A West KP Martorell R Vitamin A supplementation in Indian children.Lancet. 2013; 382: 591Summary Full Text Full Text PDF PubMed Scopus (4) Google Scholar whose prompt response is cited by other correspondents, suggest that DEVTA was not a valid trial. As, however, the population studied was vitamin A deficient by conventional criteria, the key methodological issues are whether there was good compliance with the randomly allocated treatment (yes), and whether there was any material bias between the two treatment groups in ascertainment of mortality (no). All else is irrelevant to trial reliability. The direct evidence we cited of good compliance (despite claims by various correspondents) is supported by airline delivery records of 6 million Roche vitamin A capsules (1·2 million in November, 1998; October, 1999; January, 2001; December, 2001; and October, 2002; all used in DEVTA). Individual child deaths in the past year were sought from several sources at 6-monthly visits to each village, so most deaths were recorded twice; confirming relatively few were missed twice, DEVTA mortality rates slightly exceeded those recorded by the Registrar-General of India for the whole state. The low cost per child (miscalculated tenfold by Sommer and colleagues5Sommer A West KP Martorell R Vitamin A supplementation in Indian children.Lancet. 2013; 382: 591Summary Full Text Full Text PDF PubMed Scopus (4) Google Scholar) merely reflected efficiencies of design and scale. As DEVTA had surprisingly unpromising results we convened a meeting of other vitamin A trialists to discuss their studies and ours, seeking explanations for any heterogeneity. There were no agreed proceedings, but that 2008 meeting engendered for the DEVTA publication extensive further data checks and more robust statistical analysis methods; these did not, however, change the mortality rate ratio. We conclude that the apparent discrepancy between DEVTA and previous trials was probably due mainly to the play of chance (despite being significant, p=0·001). If so, it is medically and statistically appropriate for DEVTA to have twice the weight of all other trials combined when averaging trial results to guide implementation strategies. As indicated in the figure, the real risk reduction could well be about 11% both in the circumstances of DEVTA and in those of some previous trials. To assess such moderate differences reliably, large-scale randomised evidence was needed, and DEVTA has contributed substantially to this. Arguments for or against routine vitamin A supplementation should now be based on appropriate assessment of all mortality trial results, without unduly selective exclusions.3Peto R Interpreting large-scale randomised evidence. John Snow lecture.http://www.youtube.com/watch?v=vybc0PsZ718&list=PL3oyPcbygtxuqmYps-aEj9Sa-ScThzMLhDate: April 12, 2013Google Scholar We declare that we have no conflicts of interest. Vitamin A supplementation every 6 months with retinol in 1 million pre-school children in north India: DEVTA, a cluster-randomised trialDEVTA contradicts the expectation from other trials that vitamin A supplementation would reduce child mortality by 20–30%, but cannot rule out some more modest effect. Meta-analysis of DEVTA plus eight previous randomised trials of supplementation (in various different populations) yielded a weighted average mortality reduction of 11% (95% CI 5–16, p=0·00015), reliably contradicting the hypothesis of no effect. Full-Text PDF Open AccessVitamin A supplementation in Indian childrenShally Awasthi and colleagues' DEVTA investigation (published online March 14)1 purports to have studied 1–2 million children in Uttar Pradesh, India, from 1999 to 2004, half in villages in which vitamin A supplementation reached 86% of the children every 6 months for 5 years and half in which coverage was assumed to be low. About 25 000 deaths were reported; no difference in child mortality was found between the vitamin A group and the control group. Full-Text PDF Vitamin A supplementation in Indian childrenThe results of the DEVTA study presented by Shally Awasthi and colleagues (April 27, p 1469)1 show a surprisingly low effect of vitamin A supplementation on mortality in pre-school children in Uttar Pradesh, India. The study draws attention to the challenges of ensuring optimum delivery and adequacy in evaluation of efficacious interventions in a programmatic setting. It calls on programme implementers to undertake measures to achieve and sustain high coverage for optimum delivery and outcome of such interventions. Full-Text PDF Vitamin A supplementation in Indian childrenThe results of the DEVTA study1 of vitamin A supplementation delivered through an Indian Government programme show that this particular programme did not reduce mortality. This could be a worthwhile finding if the reasons for the failure were elucidated—eg, were supply chains properly set up and implemented and was the supplement delivered to those who could most benefit? Sommer and colleagues2 addressed the methodological shortcomings of the trial, but in our view the most important issue is that programme assessments need more complex designs than do biological efficacy trials. Full-Text PDF Vitamin A supplementation in Indian childrenThe DEVTA trial1 examined a large, twice-yearly vitamin A supplementation programme but did not find the expected 20–30% reduction in overall child mortality. Its publication was delayed for more than 6 years. The Comment2 argues that the contradiction with present assumptions might explain this delay. We agree. In 2011, we completed an individually randomised placebo-controlled trial of vitamin A supplementation delivered at vaccination sessions to more than 7000 children in Guinea-Bissau and identified no overall effect, but strong sex-differential effects. Full-Text PDF Vitamin A supplementation in Indian childrenFor epidemiologists, demographers, and those working in household registration systems, the results and interpretation of the DEVTA study1 are surprising. The reader is left asking why the study—which was certainly large—included neither a surveillance system, nor at minimum a periodic or full household survey before and after implementation of the intervention, to ensure complete and accurate assessment of the primary outcome, mortality. The teams double-checking deaths reported by community workers only verified reported deaths, while leaving unreported deaths entirely unidentified. Full-Text PDF Vitamin A supplementation in Indian childrenThe report of DEVTA1 was the 44th published trial of vitamin A.2 The effect on its primary outcome, mortality, is noticeably smaller than the results of 15 previous trials including more than 200 000 participants.3 In view of problems with the delivery of the intervention in DEVTA,4 meta-analyses of previous high-quality trials might more accurately show the efficacy of vitamin A. Full-Text PDF Vitamin A supplementation in Indian childrenThe DEVTA trial1 of vitamin A supplementation was large and well organised, but did not show the 20–30% reduction in mortality in children aged 1–6 years that had been shown in earlier trials, even though adverse effects were absent. Supplementation might have reduced mortality but only by between 0% and 10%. A meta-analysis with data from other vitamin A supplementation trials showed mortality reductions of 11% (95% CI 5–16; p=0·00015). Methodological errors in the management of the trial were unlikely but non-methodological factors could have confounded this particular trial. Full-Text PDF