Grant Review Research Articles

Health Research Priority Setting: Do Grant Review Processes Reflect Ethical Principles?

ABSTRACT Most public and non-profit organisations that fund health research provide the majority of their funding in the form of grants. The calls for grant applications are often untargeted, such that a wide variety of applications may compete for the same funding. The grant review process therefore plays a critical role in determining how limited research resources are allocated. Despite this, little attention has been paid to whether grant review criteria align with widely endorsed ethical criteria for allocating health research resources. Here, we analyse the criteria and processes that ten of the largest public and non-profit research funders use to choose between competing grant applications. Our data suggest that research funders rarely instruct reviewers to consider disease burden or to prioritise research for sicker or more disadvantaged populations, and typically only include scientists in the review processes. This is liable to undermine efforts to link research funding to health needs.

Towards inclusive funding practices for early career researchers

Securing research funding is a challenge faced by most scientists in academic institutions worldwide. Funding success rates for all career stages are low, but the burden falls most heavily on early career researchers (ECRs). These are young investigators in training and new principal investigators who have a shorter track record. ECRs are dependent on funding to establish their academic careers. The low number of career development awards and the lack of sustained research funding result in the loss of ECR talent in academia. Several steps in the current funding process, from grant conditions to review, play significant roles in the distribution of funds. Furthermore, there is an imbalance where certain research disciplines and labs of influential researchers receive more funding. As a group of ECRs with global representation, we examined funding practices, barriers, and facilitators to the current funding systems. We also identified alternatives to the most common funding distribution practices, such as diversifying risk or awarding grants on a partly random basis. Here, we detail recommendations for funding agencies and grant reviewers to improve ECR funding prospects worldwide and promote a fairer and more inclusive funding landscape for ECRs.

Grant Review Feedback: Appropriateness and Usefulness

The primary goal of the peer review of research grant proposals is to evaluate their quality for the funding agency. An important secondary goal is to provide constructive feedback to applicants for their resubmissions. However, little is known about whether review feedback achieves this goal. In this paper, we present a multi-methods analysis of responses from grant applicants regarding their perceptions of the effectiveness and appropriateness of peer review feedback they received from grant submissions. Overall, 56–60% of applicants determined the feedback to be appropriate (fair, well-written, and well-informed), although their judgments were more favorable if their recent application was funded. Importantly, independent of funding success, women found the feedback better written than men, and more white applicants found the feedback to be fair than non-white applicants. Also, perceptions of a variety of biases were specifically reported in respondents’ feedback. Less than 40% of applicants found the feedback to be very useful in informing their research and improving grantsmanship and future submissions. Further, negative perceptions of the appropriateness of review feedback were positively correlated with more negative perceptions of feedback usefulness. Importantly, respondents suggested that highly competitive funding pay-lines and poor inter-panel reliability limited the usefulness of review feedback. Overall, these results suggest that more effort is needed to ensure that appropriate and useful feedback is provided to all applicants, bolstering the equity of the review process and likely improving the quality of resubmitted proposals.

Association of Surgeon Representation on NIH Study Sections With Receipt of Funding by Surgeon-scientists.

Our goal was to evaluate the relationship between surgeon representation on NIH study sections and success in grant funding. NIH funding for surgeon-scientists is declining. Prior work has called for increased surgeon participation in the grant review process as a strategy to increase receipt of funding by surgeon-scientists. A retrospective review of surgeon (primary department: General, Urology, Orthopedic, Ophthalmology, Otolaryngology, Neurosurgery) representation on NIH study sections and receipt of funding was performed using NIH Research Portfolio Online Reporting Tools Expenditures and Results (RePORTER) and 2019 Blue Ridge Institute for Medical Research data. NIH chartered study section panels and ad hoc reviewers for each 2019 review date were also obtained. In 2019, 9239 individuals reviewed in at least 1 of the 168 study sections [190 (2.1%) surgeons, 64 (0.7%) standing members, 126 (1.4%) ad-hoc]. Most surgeons on study sections were male (65%) professors (63%). Surgeons most commonly served on bioengineering, technology, and surgical sciences (29.6% surgeons), diseases and pathophysiology of the visual system (28.3%), and surgery, anesthesiology and trauma (21%). In 2019, 773 surgeons received 1235 NIH grants (>$580 M) out of a total of 55,012 awards (2.2%). Funded surgeons were predominantly male (79%), White (68%), non-Hispanic (97%), full professors (50%), and 43% had additional advanced degrees (MPH/PhD/MBA). surgery, anesthesiology and trauma, diseases and pathophysiology of the visual system, and bioengineering, technology, and surgical sciences were the most common study sections that reviewed funded grants to surgeon-scientists. Ninety-two surgeons both received grant funding and served on study section. Study sections with higher surgeon representation were more likely to fund surgeon-scientists (P < 0.001). Surgeon representation on NIH study sections is strongly associated with receipt of funding by surgeon-scientists. Increasing NIH study section representation by surgeons may help to preserve the surgeon-scientist phenotype.

11554 Getting the Grant: Assessment of a Monthly Grant Writing Group for Junior Investigators

ABSTRACT IMPACT: NJ ACTS provides mentored coaching in NIH grant writing for early stage investigators. OBJECTIVES/GOALS: Launching an independent academic careers requires the ability to effectively communicate the purpose and impact of biomedical research in order to obtain extramural funding. We sought to develop and evaluate an interactive grant writing group of junior faculty and senior postdoctoral fellows mentored by trained coaches. METHODS/STUDY POPULATION: Participants meet monthly for 1 hour to peer review Specific Aims pages for grant applications to NIH and private foundations. Sessions are moderated by two senior faculty trained as coaches by the National Research Mentoring Network. Participant grant submission and review of the program are collected annually. RESULTS/ANTICIPATED RESULTS: From 2019-2020, 15 faculty and 2 postdoctoral fellows participated in the grant writing group with an average of 5.7 participants each month. Over the year, participants submitted 53 grant applications (68% submitted to NIH with the majority being R21 or R01 grant mechanisms). Half of grants submitted were discussed during peer review sessions. Of the grants reviewed, 42% were funded or near/below the funding payline. Using a 5-point Likert scale, participants highly rated the quality of coaching (mean/SD: 4.9 ±0.2), time discussing their research (mean/SD: 4.5 ±0.8), and the input from other participants (mean/SD: 4.5 ±0.5). DISCUSSION/SIGNIFICANCE OF FINDINGS: In conclusion, a monthly meeting of junior investigators hosted by two grant writing coaches is an effective means to receive peer review of grant application aims and support submissions for extramural funding.

State Scholarship Programs and Grants as a Factor of the Human Capital Development: Russian and Foreign Experience

Recognizing the importance of the development of human capital, many countries contribute to the growth of international student mobility through various state grants and scholarships for their citizens and foreign students. Nevertheless, there is a lack of research on the state's benefit from sponsoring scholarship programs and study abroad grants in the context of the development of the country's human capital. This article examines scholarship programs and grants to study abroad regarding human capital development, how they can help build human capital, and what can be an obstacle for that. The authors analyze the features and implementation mechanisms and provide typology of the programs in modern conditions. Based on a review of state scholarship programs and grants for studying abroad, and an analysis of scholars' studies and regulatory documents, authors identify existing and potential obstacles to building up and effectively using human capital such as non-return of scholarship holders to their homeland, deskilling, bureaucratic employment procedures, emigration intentions after fulfilling obligations under the program, etc. Exploring of foreign and Russian experience in implementing state scholarship and grant programs, authors determine several ways for the further development of the programs in the context of the idea of human capital building. The results of the study can be applied to the development and improvement of the Russian scholarship programs and grants for studying abroad.

Grants: The Good, the Bad, the Ugly, and the Puzzling

Grants play a major role in higher education, including kinesiology. However, critical commentaries on the role of external funds appear nonexistent in kinesiology. Therefore, the purpose of this paper is to outline the most common criticisms of grants to stimulate a conversation in kinesiology. First, I discuss benefits of grants. Second, I examine the role of grants in higher education. Third, I discuss how external funds are not required to contribute meaningful research. Fourth, I examine how a major reason for grants, to produce research publications, often goes unfullfilled. Fifth, I show how the development of grant applications (especially unsuccessful applications) is an inefficient expenditure of resources. Sixth, I discuss how pursuing grants can be detrimental to other important academy goals. Seventh, I examine how grants may negatively influence faculty and administrator morale and quality of life. Eighth, I report on some common criticisms of the grant review process and discuss some alternative reviewing systems. Finally, I end with a brief summary and some recommendations.

Utilising benefit-risk assessments within clinical trials\u2014a protocol for the BRAINS project

BackgroundDepending on the treatment to be investigated, a clinical trial could be designed to assess objectives of superiority, equivalence or non-inferiority. The design of the study is affected by many different elements including the control treatment, the primary outcome and associated relationships.In some studies, there could be more than one outcome of interest. In these situations, benefit-risk methodologies could be used to assess the outcomes simultaneously and consider the trade-off between the benefits against the risks of a treatment.Benefit-risk is used within the regulatory industry but seldom included within publicly funded clinical trials within the UK. This project aims to gain an expert consensus on how to select the appropriate trial design (e.g. superiority) and when to consider including benefit-risk methods.MethodsThe project will consist of four work packages:A web-based survey to elicit current experiences and opinions,A rapid literature review to assess any current recommendations,A two-day consensus workshop to gain agreement on the recommendations, andProduction of a guidance document.DiscussionThe aim of the project is to provide a guideline for clinical researchers, grant funding bodies and reviewers for grant bodies for how to select the most appropriate trial design and when it is appropriate to consider using benefit-risk methods. The focus of the guideline will be on publicly funded trials however, the vision is that the work will be applicable across research settings and we will connect with other organisations and committees as appropriate.

Using Reverse Auctions to Stretch Broadband Subsidy Dollars: Lessons from the Recovery Act of 2009

Twelve years ago, the federal government awarded hundreds of grants for broadband infrastructure with stimulus funds from the Recovery Act of 2009. In this study, I review the subsidy allocations from the Broadband Technology Opportunities Program and compare actual outcomes with those that a reverse auction or random lottery may have yielded. My analysis shows that a reverse auction might have connected nearly twice as many buildings for the same total subsidy dollars relative to the results from the grant review process. Moreover, the grant review process used by the NTIA did only slightly better in subsidy-dollars-per-building-connected than a random lottery probably would have. Lessons from the Recovery Act are drawn from public data on proposed and awarded projects from application files for stimulus funds. I conclude that the government likely overpaid for broadband by using grant review rather than a reverse auction. The analysis strongly implies that future broadband grants be distributed using market-based methods to get better returns on investment from infrastructure funds.

RADx Tech Viability and Steering Panels: A Model for MedTech Translational Grant Review.

RADxSM Tech’s mission is to rapidly accelerate deployment of SARS-CoV-2 tests and could not utilize typical grant application and review processes that can run 4 to 6 months. Instead, RADx Tech leveraged methodologies developed by CIMIT and utilized by POCTRN as described further in this special issue. RADx Tech uses a multi-stage review with two review panels, a Viability Panel and a Steering Panel, that are supported by subject matter experts and a Deep Dive team. Members of the panels have extensive commercialization and business experience in addition to scientific and technical knowledge. The Viability Panel is responsible for assessing whether the proposal is a good fit with the RADx Tech Program and whether it should be recommended to move into a Deep Dive. Less detailed information is requested in the application than a typical SBIR application since the application is refined and details added during the Deep Dive. The Steering Panel reviews the results from the Deep Dive and decides whether to recommend further funding. Everyone on the Viability Panel and Steering Panel reviews every application, thereby providing consistency and context for the reviewers. Utilization of an “assess, improve, and then select” process with review panels comprised of highly experienced review panel members has resulted in improved timing, efficiency, and effectiveness of reviews and has the potential to be extensible beyond RADx Tech.

A Tribute to Katherine H. Taber, Ph.D., F.A.N.P.A.

A Tribute to Katherine H. Taber, Ph.D., F.A.N.P.A.

The Fallacy of Follow-up: When Orthopaedic Trauma Patients Actually Return to Clinic.

Clinical follow-up in orthopaedic trauma is challenging, yet expectations exist that a 1-year follow-up is the minimum requirement for clinical trials and research publications. The primary purpose of our study was to evaluate the rate of follow-up after operative orthopaedic trauma care and the relationship to clinical care. Our secondary aim was to identify any independent risk factors regarding follow-up completion. A chart review of patients operatively treated for a traumatic injury during the months of January and July 2016 was conducted. Patient demographic characteristics, injury type, severity, and patient distance from the hospital were collected. The final clinical instructions and whether a return visit was requested or as needed were recorded. There were 293 patients in this study, of whom 84 (29%) had follow-up of at least 1 year and 52 (18%) were instructed to follow up only as needed at their last visit prior to the 1-year mark. When removing the latter 52 patients, the 1-year follow-up rate was 35% (84 of 241 patients). Of these 241 patients, 157 (65%) were requested to return for additional clinical care but failed to return prior to 1 year. Logistic regression identified tobacco use (odds ratio [OR], 0.34 [95% confidence interval (CI), 0.15 to 0.77]; p = 0.010), final appointment status (OR, 6.3 [95% CI, 3.4 to 11.6]; p < 0.001), isolated compared with multiple fractures (OR, 2.2 [95% CI, 1.2 to 4.1]; p = 0.013), and distance from the trauma center per mile as a continuous variable (OR, 0.999 [95% CI, 0.998 to 1.0]; p = 0.03) as significant predictors. Our data suggest that a 1-year clinic follow-up requirement may not be feasible. We observed a low rate of patients with a minimum 1-year clinical follow-up. Clinical care had been completed in 18% of patients prior to 1 year. Journal and grant reviewers may need to consider the feasibility and clinical relevance of these follow-up expectations.

Retrospective-prospective study of safety and efficacy of sofosbuvir-based direct-acting antivirals in HIV/HCV-coinfected participants with decompensated liver disease pre- or post-liver transplant.

Direct-acting antiviral (DAA) therapy has transformed the management of human immunodeficiency virus (HIV) and hepatitis C (HCV) coinfected patients with advanced liver disease. STOP-Coinfection was a multicenter prospective and retrospective, open-label study using sofosbuvir-based DAA therapy to treat HIV/HCV-coinfected participants pre- or post-liver transplant (LT). Sixty-eight participants with end-stage liver disease (Child-Turcotte-Pugh score ≥7 and Model for End-Stage Liver Disease score 6-29) were enrolled, 26 had hepatocellular carcinoma. Forty-two participants were treated pre-LT and 26 post-LT. All participants completed therapy without need for dose reduction or transfusion; eight required two or more courses of therapy. Ninety-three percent achieved a sustained virologic response and DAA therapy was well tolerated. Despite HCV cure, 12 end-stage liver disease participants required subsequent LT, 7 for decompensated liver disease. Thirteen participants died, 10 with decompensated liver disease pre-LT and three post-LT. Overall, transplant free survival was 42.8% at 4years and post-LT survival was 87.9% at 5years. We conclude that sofosbuvir-based DAA therapy is safe and highly effective in HCV-HIV patients with decompensated liver disease and post-LT, with post-LT survival rates comparable to other indications. This removes one of the last barriers to liver transplantation in this challenging cohort of recipients.

The Editor's Problem

An editor maximizing quality relies on the qualitative recommendation of a reviewer. The reviewer may be biased to accept or reject independently of quality. Using the minimum principle, an averaging rule is found to best reduce the noise introduced by bias: under this rule, an unbiased reviewer accepts when quality is greater than the average quality selected by the editor. For distributions with heavy tails, the probability of acceptance due to bias is bounded away from zero even if almost all reviewers are unbiased. Standards adopted by the editor may be excessive relative to the social optimum. Environments with multiple reviewers, reviewer histories, detailed reviews and competition between editors do not solve the problem and may worsen it. A dynamic peer review demonstrates the inherent fragility of equilibria with informative reviews. The model applies to many settings, including grant review, evidence selection, medical testing, juries and project selection.

Deciding to post a manuscript preprint

Deciding to post a manuscript preprint

Reviewer training for improving grant and journal peer review

Reviewer training for improving grant and journal peer review

Introductions to the Community: Early-Career Researchers in the Time of COVID-19.

COVID-19 has unfortunately halted lab work, conferences, and in-person networking, which is especially detrimental to researchers just starting their labs. Through social media and our reviewer networks, we met some early-career stem cell investigators impacted by the closures. Here, they introduce themselves and their research to our readers.

Editorial introductions

Current Opinion in Lipidology was launched in 1990. It is part of a successful series of review journals whose unique format is designed to provide a systematic and critical assessment of the literature as presented in the many primary journals. The field of lipidology is divided into six sections that are reviewed once a year. Each section is assigned a Section Editor, a leading authority in the area, who identifies the most important topics at that time. Here we are pleased to introduce the Section Editor for this issue. Section Editor Mohamad NavabMohamad NavabMohamad Navab is a Faculty and a co-project leader at David Geffen school of medicine, University of California Los Angeles. He obtained his PhD in biochemistry from Columbia University, in 1976. Subsequently, he served as an Assistant Professor and Head of Clinical Labs at the National University School of Medicine in Tehran, Iran. He started his research and teaching first at the UCLA Cancer Center in 1981 and subsequently at the Division of Cardiology at UCLA where he has been since 1983. Prof. Navab has received research support from NIH and has served as grant reviewer for the National Heart, Lung & Blood Institute at NIH and the American Heart Association. He serves on the editorial board and advisory board of a dozen scientific journals. Prof. Navab has authored or been a coauthor for 203 papers indexed at PubMed and has presented 171 invited national and international medical lectures. Menno de WintherMenno de WintherMenno de Winther is a Professor of vascular immune cell biology at the AMC of the Amsterdam-University Medical Centers, Amsterdam, Netherlands. Prof. de Winther graduated in biology at Utrecht University, Netherlands, and obtained his PhD in 1999 at Leiden University, Netherlands with a thesis on macrophage scavenger receptors in atherosclerosis. After completing the PhD, he was a postdoc in Oxford University, UK and Maastricht University, Netherlands and became a principal investigator studying the role of macrophages in cardiovascular disease. In 2011, Prof. de Winther was appointed as a Professor of vascular immune cell biology at the Academic Medical Center of the University of Amsterdam. Using cellular and animal models for disease, his group studies the molecular regulation of macrophages in inflammatory disease. His research particularly focuses on the transcriptional regulation of monocytes and macrophages in atherosclerosis, characterizing environmental factors (e.g. dyslipidemia, metabolic triggers, cytokines) that drives signaling pathways influencing cellular function in disease. Prof. de Winther's group was the first to define the role of the transcription factor NF-κB in atherosclerosis and more recently identified the epigenetic enzyme HDAC3 a major regulator of macrophage gene programs in atherosclerosis. Significant efforts in his group are now running on further delineating epigenetic pathways in atherosclerostic diseases. His group thrives on a combination of human and mouse cell- and animal models combined with patient studies through close interactions with the clinic. Dr de Winther is an established investigator of the Netherlands Heart Foundation and together with Dr Glass (UCSD) is currently coordinating a Leducq Transatlantic Network on epigenetics in atherosclerosis.

An online Paper Authoring Tool (PAT) to improve reporting of, and synthesis of evidence from, trials in behavioral sciences.

The Science of Behavior Change project represents a major advance in the way we approach the study of behavior and behavior change; the protocols in this special issue reflect the innovative approach to intervention development. Digital technology is transforming the way we conduct science and can greatly assist with developing protocols, reporting studies, and evidence synthesis. The journal Addiction has developed an online Paper Authoring Tool (PAT) for reporting full-scale randomized controlled trials that could also be used for reporting protocols and early phase trials. PAT aims to help authors ensure that they include all the required information and present it in a way that is clear and consistent in terms of language and phrasing. It also generates a computer-readable record that should greatly facilitate automation of evidence synthesis. PAT is intended to represent the first step in creation of a supportive ecosystem for the conduct of science, linking grant applications, study protocols, study reports, measures, reviews and study data in a coordinated online knowledge base. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Editorial introductions

Editorial introductions