Evaluation and appraisal of randomized controlled trials in myeloma

Abstract

To critically appraise therapeutic innovations tested in randomized controlled trials (RCTs) in multiple myeloma from 1966-1998. We performed a comprehensive search to identify published RCTs in multiple myeloma. Quality dimensions of the design, conduct, analysis and reporting of each trial were assessed. We identified 136 RCTs reported in 114 papers. Overall, therapeutic efforts in multiple myeloma resulted in a 5% absolute gain in five-year survival at a cost of a 0.35% increase in treatment related deaths. Hence on average a patient enrolled in a RCT in myeloma is 14 (5/35) times more likely to be helped than harmed. However, when the RCTs were critically appraised for key quality dimensions of trials' conduct, we found that only 7%) of the trials (10 of 136) were analyzed according to intention-to-treat (ITT), 9% (12 of 136) reported a power analysis (beta error), 32% (35 of 111) adequately concealed treatment allocation, 78% (106 of 136) provided a detailed description of patient withdrawals, and 83%) (19 of 22) of the double blind RCTs had appropriately described methodology. Therapeutic innovations tested through RCTs have improved the outcomes of patients with multiple myeloma. However, the quality of RCT reporting and methodology in multiple myeloma could be substantially improved. Most therapeutic strategies in multiple myeloma are based on modest quality, low power evidence. Despite these shortcomings our findings suggest patients may often clinically benefit from enrollment in clinical trials when available. Patients on average received modest benefit from innovative therapies tested in RCTs at little additional risk of side effects.