Quality appraisal of clinical validation studies for multigene prediction assays of chemotherapy response in early-stage breast cancer.

Abstract

3082 Background: Despite enthusiasm concerning the potential for genomic predictors of chemotherapy response, considerable uncertainty about optimal research methodology remain. A comprehensive literature search and quality appraisal of multigene signatures predictive of response to chemotherapy in breast cancer were initiated as part of NIH ARRA grant funding. Methods: Validation studies of multigene signatures for prediction of chemotherapy response in breast cancer were sought. A priori inclusion/exclusion criteria and data extraction were applied in a dual blinded fashion. Conflict resolution was based on consensus among three reviewers. Detailed information on the genomic assay, patient population, specific chemotherapeutic agents, clinical prognostic measures, and study outcomes were extracted. In addition, a formal quality appraisal was conducted, utilizing a modified quality scoring system from EGAPP, REMARK, and the STREGA Statements. Results: A total of 37 fully eligible studies were identified. These papers present results on 68 multigene predictors. The primary outcome is treatment response in 51 (75%) analyses and time to recurrence or survival in 17 (25%). A high percentage of studies fail to report the following quality measures: sample size rationale (95%), patient selection (49%), study protocol (81%), quantity/quality of specimen tumor content (33%), assessment of prognostic subgroups (70%), adjustment for false discovery rate (63%), blinding of assay results to clinical outcome (95%), model ROC (56%), complete model specification (96%) or source code for analysis (96%), and clinical utility assessment in univariate (59%) or multivariate (62%) analysis. On a 3-point scale from poor to good, studies were graded as poor to fair for patient characteristics (41%), study endpoints (30%), tissue handling (65%), statistical methods (76%), and statistical model development (81%). Conclusions: Validation studies of multigene signatures of chemotherapy response in patients with early-stage breast cancer vary substantially in terms of study design, methodology, and most studies exhibit major reporting and quality limitations.