A New and Rapid Scoring System to Assess the Scientific Evidence from Clinical Trials

Abstract

Guidelines are based on a scientific analysis from existing data of randomized controlled trials (RCTs), registry trials, simple registries, case reports, and the personal experience of the task force members. Furthermore, meta-analyses and subgroup analyses are used to derive the strengths of recommendations. Fortunately, the major cardiac societies, i.e., the American College of Cardiology (ACC), the American Heart Association (AHA), and the European Society of Cardiology (ESC), are essentially using the same definitions for the levels of recommendations. In the expanding field of cardiology, however, the overwhelming and increasing number of clinical studies reveals the limitations of the traditional ranking of these studies: Applying the standard definitions of the ACC/AHA/ESC criteria for the levels of recommendation, almost every PCI procedure would easily reach the level of evidence A, even with two small, underpowered studies and a surrogate endpoint. Although meta-analyses are important tools for creating an overview of major diagnostic or treatment modalities, they are bound to severe limitations. The compilation of several underpowered, small trials can generate a statistically artificial "significant" result. This is especially important because only two meta-analyses containing almost identical studies could easily yield an evidence level A. RCTs are usually designed and conducted according to a power calculation, for which a primary clinical or surrogate endpoint can be chosen. Surrogate endpoints, however, do not necessarily correlate with the clinical outcome. The history of medicine is full of errors introduced by underpowered studies with surrogate endpoints. Many investigators and companies attempt to tease out treatment effects in specific subpopulations of patients. These subgroup analyses are usually underpowered. Another major limitation of the ACC/AHA/ESC scoring system is that neither the power of a study nor the choice of a primary clinical endpoint is included in their definitions. Yet another limitation of the ACC/AHA/ESC grading system is that two "simple" registries may already lead to a level of evidence B. A new scoring system is presented addressing most of these limitations: a primary clinical endpoint receives three points, whereas all of the following receive one point: double-blind design, evaluation interval of primary endpoint > or = 6 months, multicenter (at least three centers), independent data and safety monitoring, power of > or = 80% for primary endpoint achieved, and follow-up > or = 80% for a surrogate primary endpoint or follow-up of > or = 95% for a clinical primary endpoint. Thus, the maximum achievable points is 10. This scoring system can also be applied for high-quality registry controlled trials using a predefined control group and power calculation. For simple registry studies and subgroup analyses, a modified scoring system has been developed (maximum achievable points is 5). The advantage of the suggested new scoring system is its transparency, reproducibility, and ease of use by quickly answering the key quality questions for clinical trials. The new scoring system suggested here should help make decisions regarding which treatment to use and stimulate discussions.