Abstract
The primary goal of the peer review of research grant proposals is to evaluate their quality for the funding agency. An important secondary goal is to provide constructive feedback to applicants for their resubmissions. However, little is known about whether review feedback achieves this goal. In this paper, we present a multi-methods analysis of responses from grant applicants regarding their perceptions of the effectiveness and appropriateness of peer review feedback they received from grant submissions. Overall, 56–60% of applicants determined the feedback to be appropriate (fair, well-written, and well-informed), although their judgments were more favorable if their recent application was funded. Importantly, independent of funding success, women found the feedback better written than men, and more white applicants found the feedback to be fair than non-white applicants. Also, perceptions of a variety of biases were specifically reported in respondents’ feedback. Less than 40% of applicants found the feedback to be very useful in informing their research and improving grantsmanship and future submissions. Further, negative perceptions of the appropriateness of review feedback were positively correlated with more negative perceptions of feedback usefulness. Importantly, respondents suggested that highly competitive funding pay-lines and poor inter-panel reliability limited the usefulness of review feedback. Overall, these results suggest that more effort is needed to ensure that appropriate and useful feedback is provided to all applicants, bolstering the equity of the review process and likely improving the quality of resubmitted proposals.
Highlights
IntroductionAt the National Institutes of Health (NIH), after a grant application is peer reviewed, the scores and comments from the reviewers are sent back to the applicant (NIH 2018)
The results of our analysis suggest that while the majority of grant applicants in our survey deemed reviewer feedback to be appropriate across several dimensions—including how fair, well-written and well-informed the feedback was— there were sizable proportions (40–44%) who did not find it appropriate
Reliability concerns intersected with views on fairness; several respondents noted significantly different sets of issues identified in the feedback from panel to panel for resubmissions, revealing important inconsistencies in feedback that can be construed as inequitable to the applicant
Summary
At the National Institutes of Health (NIH), after a grant application is peer reviewed, the scores and comments from the reviewers are sent back to the applicant (NIH 2018). If it is not funded, the applicant must address the reviewer comments if they are to resubmit an updated version of their application (NIH 2020; NIAID 2020). Not listed as a core value of the NIH peer review system, reviewer feedback to applicants for the purposes of improving investigator grantsmanship and the overall quality of applications is an important, if secondary, purpose of grant peer review (NIH 2019). Little empirical data exist that document whether grant review feedback is effective in informing applicants and improving applications
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
