ACEP-SAEM response to the National Institutes of Health (NIH) proposal to simplify the review framework for research project grant applications.

Abstract

The Society for Academic Emergency Medicine (SAEM) and American College of Emergency Physicians (ACEP) are responding jointly to this request for information. SAEM and ACEP promote innovative and imaginative strategy development that will transform prehospital and emergent patient care. The National Institutes of Health (NIH) proposal to modify the current grant scoring system to prioritize these principles is a welcome change and we are supportive of these efforts. Focusing on research importance, feasibility, and scientific rigor will make it easier to interpret reviewer critiques and provide investigators with clearer information on score driving factors related to the science itself. While investigator and environment are important factors for successfully conducting research, reducing their overall importance may help with reducing bias and expanding the pool of meritorious applications. We also support combining significance and innovation into a single “importance” score, as the current format gives equal weight to innovation even though related sections tend to be brief. Furthermore, the existing significance section focuses largely on perceived need and scientific rationale with lesser consideration of the true magnitude of impact. Our societies are hopeful that by broadening the concept of overall importance, this new section can better weight funding decisions toward injuries and illnesses where improved outcomes would benefit the greatest number of people. We also agree that the current scoring system overly emphasizes an investigator's prior grant success and institutional reputation in determining an application's merit. Reliance on these factors detracts from the proposal's true “importance” and may perpetuate existing and ongoing challenges with the development and retention of junior investigators, including investigators from historically underrepresented groups. The proposed changes in this area will allow for increased emphasis on the development of highly novel or paradigm-shifting approaches to problems. Nonetheless, consideration of the capacity to execute the proposed work remains a critically important factor. The proposed method of scoring using a descriptive range of “fully capable” to “additional resources needed” deemphasizes quantitative scoring and we believe accomplishes the stated goals. However, we recommend specific clarification that the proposed scale reflects the entirety of the investigatory team (with due consideration of experience with prior collaboration). An important component of the current scoring system not addressed in the proposed changes is the handling of grant resubmissions. It is not clear why resubmissions have a written narrative that is score-driving and subject to bias. We recommend amending this process for each additional criteria to be marked yes or no as to “all major/moderate concerns appropriately addressed.” Reviewers could also have the option of describing “any new major/moderate concerns identified” in the resubmission process. Finally, while we recognize that these scoring revisions are important, we suggest that the NIH consider a more direct acknowledgment of our nation's need to fund a diverse pool of grant applicants. We believe the approach of the three-factor scoring system that emphasizes capability to perform the work is well aligned with this goal, but we would encourage the NIH to consider an explicit acknowledgment of ongoing disparities in our nation's health research programs addressed by these changes.