The Endocrine Society Clinical Practice Guidelines: A Self-Assessment

Abstract

In 2005, after many years of discussion about whether or not The Endocrine Society (TES) should produce Clinical Practice Guidelines (CPGs), TES Council approved a carefully designed program to produce 2 to 4 high-quality CPGs each year. TES has published 20 CPGs to date (1), and several more are expected to be published in the next 2 years. Since its inception, the CPG program has been unique in many ways. First, a highly regulated pathway on the route to publication (Figure 1) permits the selection, review, and vetting of CPGs at multiple levels, including a final peer and editorial review at the Journal of Clinical Endocrinology and Metabolism (JCEM). Second, the guidelines have been selected and produced without support from pharmaceutical or device-manufacturing companies. Third, all CPG expert panels have included at least 1 international member. Fourth, all CPGs have been translated into patient versions. Fifth, the CPGs are routinely reviewed and updated every 3 years. Finally, TES adopted the Grades of Recommendation Assessment, Development and Evaluation (GRADE) approach for evaluating the evidence and producing recommendations (2). The GRADE Working Group began as an international effort to resolve confusion about how to best and most clearly rate evidence and express recommendations. Many governmental organizations and professional societies use the GRADE approach, including the Agency for Healthcare Research and Quality, the American College of Physicians, and UpToDate in the United States; the National Institute for Clinical Excellence and the British Medical Journal in the United Kingdom; and the World Health Organization (3). Although there are multiple systems of grading evidence, the GRADE method is the only one that rates the strength of the recommendation (Strong or Weak) based on a balance of benefits vs harms, the confidence in the magnitude of estimated effect on an outcome, and the weighing of patient values and preferences. Although this method allows for Strong recommendations to be made with low (L) or very low (VL) levels of evidence and vice versa, this should not be a frequent occurrence. TES engaged the Knowledge and Evaluation Research (KER) Unit at the Mayo Clinic to provide methodological assistance to each of the CPG expert panels, whose expertise generally is in clinical medicine and not in methodology or epidemiology. In this issue of JCEM, members of the KER Unit have undertaken a unique exercise—assessing the appropriateness of 357 recommendations contained in all 17 CPGs that were published by TES between 2006 and 2011 (4). Their major finding was that 206 of the 357 recommendations were Strong, of which 121 were based on L/VL levels of evidence. Of the 121, 53 were in categories such as “sensible alternatives do not exist” or recommend “additional research.” KER Unit members then retrospectively applied a recently published schema by the GRADE Working Group of paradigmatic situations where Strong recommendations with L/VL evidence might be appropriate (5). Using this new taxonomy of paradigmatic situations to determine the appropriateness of the remaining 68 Strong recommendations that were based on L/VL levels of evidence, they found that it was appropriately applied to 35 recommendations and inappropriately applied to 33 (6 –16).