Subjecting known facts to flawed empirical testing

Abstract

Creativity is a wonderful thing, but, like anything else, it has its scope. It serves us well in the creation of new knowledge and in vetting claims whose veracity remains uncertain. But what are we to make of creativity applied instead in a revisionist fashion, to argue against facts which frankly remain indisputable? This is probably not the time or the place for such creativity. We already know, with absolute certainty, that excluding from the analyses any randomized patients, in violation of the intent-to-treat principle, renders the trial susceptible to bias [[1]Newell D.J. Intention-to-treat analysis: implications for quantitative and qualitative research.Int J Epidemiol. 1992; 21: 837-841Crossref PubMed Scopus (524) Google Scholar]. This would remain a fact even if we could somehow demonstrate that the potential for bias was never realized in practice. Of course, Dossing et al. [[2]Dossing A. Tarp S. Furst D.E. Gluud C. Wells G.A. Beyene J. Modified intention to treat analysis did not bias trial results.J Clin Epidemiol. 2016; 72: 66-74Abstract Full Text Full Text PDF PubMed Scopus (34) Google Scholar] show no such thing. Instead, they set up the known fact that postrandomization exclusions actually matter, as the alternative hypothesis, and then concluded the null hypothesis by virtue of failing to reject it. The flawed logic in concluding the null is matched only by the commensurate flawed logic of presuming that a bias would manifest as a difference, always in the same direction, in the odds ratios. At issue is that the bias causes shifts away from reality, a reality that not only varies trial to trial but also remains unknown in each trial. It is entirely possible that the 30 intent to treat trials had no bias at all and that the 37 modified intent to treat trials all had large biases, but how would we know that? We do know that the use of modified intent to treat itself exposes these 37 trials to bias, but the authors [[2]Dossing A. Tarp S. Furst D.E. Gluud C. Wells G.A. Beyene J. Modified intention to treat analysis did not bias trial results.J Clin Epidemiol. 2016; 72: 66-74Abstract Full Text Full Text PDF PubMed Scopus (34) Google Scholar] want to sweep this under the carpet by claiming that a lack of observed difference in treatment effect magnitudes rules this bias out? That is nonsequitur, as clearly illustrated by applying the same flawed revisionist logic to other known facts, which can then be made to appear false. Suppose, for example, that we are after a more parsimonious numbering system and want to argue that 66 = 65 and that 74 = 75, so therefore the numbers 66 and 74 are redundant, and should be removed from the number system. We would follow the revisionist paradigm and ignore the fact that this is clearly false. We would gather up some trials conducted in a geriatric population, split them into a test group and a control group, and in the control group leave the data intact. But in the test group, we would change any age of 66 to 65 years and any age of 74 to 75 years. The null hypothesis would be that these two changes do not affect the mean age, and the alternative would be that it does change the mean age. Almost certainly we would fail to reject the null hypothesis, and by the flawed logic used by the authors [[2]Dossing A. Tarp S. Furst D.E. Gluud C. Wells G.A. Beyene J. Modified intention to treat analysis did not bias trial results.J Clin Epidemiol. 2016; 72: 66-74Abstract Full Text Full Text PDF PubMed Scopus (34) Google Scholar], we would then accept the null and conclude that deleting the numbers 66 and 74 does not introduce bias compared to not doing so. The saving grace in making this error would be that along with the numbers 66 and 74, so too would any journal article occupying those page numbers be done away with. Modified intention-to-treat analysis did not bias trial resultsJournal of Clinical EpidemiologyVol. 72PreviewTo investigate whether analysis of the modified intention-to-treat (mITT) population with postrandomization exclusion of patients from analysis is associated with biased estimates of treatment effect compared to the conservative intention-to-treat (ITT) population. Full-Text PDF