Why Are Peer Review Outcomes Less Favorable for Clinical Science than for Basic Science Grant Applications?

Abstract

For decades, concerns have arisen over the robustness of the clinical research enterprise and the erosion of the cadre of new and experienced clinical investigators (1–8). In 1996, the director of the National Institutes of Health (NIH) responded to these concerns by impaneling a group of experienced clinical investigators and teaching hospital administrators to recommend policy changes regarding clinical research (9). Subsequently, NIH implemented a number of the panel’s recommendations, including increased support of clinical research training programs and the establishment of NIH–sponsored educational debt relief programs for clinical investigators (10,11). Re-engineering the clinical research enterprise is a major theme of the NIH Roadmap for Medical Research, launched in 2003 (12). Despite these initiatives in support of clinical research, clinical investigators often perceive that clinical research grant applications may be disadvantaged in the NIH peer review process. Several reports have indicated that priority scores and funding rates are lower for clinical than for non-clinical applications (8, 13–15). Comparable differences between clinical and non-clinical applications were observed in applications reviewed in 1994 and 2004. Anecdotally, it has been suggested that these differences reflect inherent limitations of clinical studies, e.g., clinical studies are more difficult to control and it is difficult to determine causality from the results. Based on an analysis of the relationship of review outcomes in 2004 to study section assignment and the professional backgrounds of study section members, we previously reported the assignment of priority scores for clinical and non-clinical applications did not differ for reviewers with or without experience conducting clinical research (15). The less favorable review outcomes for clinical applications were also not accounted for by the “density” of clinical applications reviewed in a study section or by the greater requested costs for clinical research (15). Our preliminary observations, based on data from two funding cycles, suggested that human subject concerns contributed to the overall less favorable review outcomes for clinical applications (14). We undertook an analysis to further evaluate potential explanations for the difference in peer review outcomes between clinical and non-clinical applications. Specifically, we focused on the impact of rates of submission and outcomes of amended and competing renewal applications submitted by new and established investigators and human subject protection concerns raised at the time of review.