When best evidence is rarer than the disease

Abstract

To the Editor, Dr. Bryson raises important questions in his editorial, ‘‘Rarity and review: evidence-based practice for uncommon diseases’’. For many rare disorders, such as carcinoid heart disease, ‘‘Best Evidence’’ is a case series that took 18 years to assemble! Animal models of rare diseases provide another source of evidence. The porcine model of malignant hyperthermia (MH) has been used to develop human diagnostic tests and showed the efficacy of dantrolene therapy at a time when human MH mortality was 80%. Most of what we know about MH comes from such studies and from epidemiologic work. There will never be a human randomized controlled trial (RCT) to determine the optimal dose of dantrolene to treat MH. Instead, MH in susceptible swine was considered similar enough to human MH to extrapolate those results with dramatic effect on mortality. Now new ex vivo models, such as tissue myotubes or cell cultures, provide functional assays to study MH pathophysiology and therapeutics. The Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence (2011) and the PRISMA statement, cited by Dr. Bryson, are silent on such nonclinical studies. The OCEBM lists systematic review of human RCTs as an example of Level 1 evidence but describes ‘‘mechanism-based reasoning’’ as the lowest level (Level 5). It does not dismiss such weaker evidence, but it does not specify how it should be evaluated. Recently, the United States Agency for Healthcare Research and Quality (AHRQ) released a white paper outlining a framework to evaluate ‘‘the evidential weight of mechanistic knowledge’’. It provides a definition of evidence based on inferential effect (how things work), not study design (how information was produced), as is the case in evidence-based medicine (EBM). It suggests that the strength of inferential evidence is ‘‘represented by the degree to which the probability of a clinical effect is modified by evidence from the component steps’’. Those steps are a formalized evaluation of mechanistic evidence that includes seeking a ‘‘translation’’ step that significantly raises the probability that the evidence could be applied to humans. Pharmaceutical companies already practice in a similar way to bring new compounds to market. The AHRQ considers that such a framework could link basic science, EBM, and comparative effectiveness research. The translation of evidence from such mechanistic knowledge could help strengthen systematic reviews of rare disorders.