National Institutes Of Health Grant Applications Research Articles

Trends in the Use of Promotional Language (Hype) in National Institutes of Health Funding Opportunity Announcements, 1992-2020

This cross-sectional study examines changes from 1992 to 2020 in the use of promotional language in National Institutes of Health (NIH) funding opportunity announcements in comparison with trends reported in NIH grant applications.

Trends in the Use of Promotional Language (Hype) in Abstracts of Successful National Institutes of Health Grant Applications, 1985-2020

The integrity of the grant application process is important to the success of the entire research enterprise. However, little information is available concerning the prevalence and evolution of subjective or promotional language ("hype") that has the potential to undermine objectivity in the writing and evaluation of grant applications. To assess changes over time in the use of hype in abstracts of National Institutes of Health (NIH) grant applications. This cross-sectional study assessed the prevalence of promotional adjectives in abstracts in the NIH archive from 1985 to 2020. From all abstracts in the NIH RePORTER (Research Portfolio Online Reporting Tools: Expenditures and Results) archive, adjectives were automatically extracted, and their frequencies in the most recent year (2020) were assessed relative to the start year (1985). Adjectives that shifted significantly in frequency and that carried a promotional sense (ie, hype) were retained, and patterns of change were assessed by plotting yearly frequencies (1985-2020). By grouping the adjectives based on shared semantic properties, broad meanings commonly expressed by hype were identified. Absolute change was measured as the difference in normalized frequency between 1985 and 2020. Relative change was measured as the percentage change in normalized frequency in 2020 relative to 1985, or the first year of occurrence. In total, 901 717 abstracts were analyzed and 139 adjective forms were identified as hype. Among these 139 adjective forms, 130 hype adjectives increased in frequency by 7690 words per million (wpm) (mean [SD] relative increase, 1378% [3132%]), while 9 hype adjectives decreased in frequency by 686 wpm (mean [SD] relative decrease, 44% [18%]). The largest absolute increases were for the terms novel (1054 wpm), critical (555 wpm), and key (461 wpm), while the largest relative increases were for the terms sustainable (25 157%), actionable (16 114%), and scalable (13 029%). Hype most often serves to promote the significance, novelty, scale, and rigor of a project; the utility of the expected outcomes; the qualities of the investigators and research environment; and the gravity of the problem; as well as conveying the personal attitudes of the applicants. Levels of hype in successful NIH grant applications have increased over time from 1985 to 2020. The findings in this study should serve to sensitize applicants, reviewers, and funding agencies to the increasing prevalence of subjective, promotional language in funding applications.

Engaging Early Career Researchers

MODERATOR: Kristin Inman Environmental Health Perspectives/NIEHS Research Triangle Park, North Carolina SPEAKERS: Kristin Inman Max Aung University of Southern California, Keck School of Medicine Los Angeles, California @max_aung Kristin Kramer Center for Scientific Review National Institutes of Health Bethesda, Maryland @csrpeerreview REPORTER: Erin Landis Origin Editorial Frederick, Maryland The “Engaging Early Career Researchers” panel centered on the importance of attracting and retaining junior faculty both in research and in the process of scholarly publishing. The panel consisted of speakers all deeply involved with programs and initiatives for early-career researchers (ECRs). Dr Kristin Kramer opened the session by explaining the scope of the National Institutes of Health (NIH) Center for Scientific Review (CSR), which is to provide first-level review of NIH grant applications. A pool of over 20,000 reviewers evaluates more than 66,000 applications annually. Dr Kramer explained that the process of reviewing grant applications varies from that of manuscripts for journals in 2 key ways: 1) the primary audience for the reviews is the NIH, as the goal of the review is to identify the highest impact science for funding consideration at the next stage of review and 2) the process is not iterative—resubmissions do not routinely go back to the same reviewers. The NIH CSR has an early-career reviewer program; the goal of this program is to enrich the peer-reviewer pool and to provide first-hand review experience to early-career scientists as they prepare their own grant applications. Dr Kramer also explained that training is central to the program; […]

The Proposal Preparation Program: A Group Mentoring, Faculty Development Model to Facilitate the Submission and Funding of NIH Grant Applications.

This article describes the University of Minnesota Medical School Proposal Preparation Program (P3). P3 is designed to develop grant-writing skills for assistant professors preparing their first K- or R-series application to the National Institutes of Health (NIH). Three 4-month P3 cycles are conducted annually. For each cycle, a cohort of around 10 assistant professor participants and 5 regular faculty mentors meet for ten ~2-hour group sessions. Participants receive iterative oral and written feedback on their proposals in development within a small, interdisciplinary, group mentoring setting providing structure, accountability, guidance, and support. Between sessions, 1 peer and 1 mentor are assigned (on a rotating basis) to critique each participant's developing application. The sessions include a brief mentor-led presentation on a particular grant section followed by discussion of each participant's application conducted by the assigned reviewers. The cycle concludes with a mock NIH review session, in which each participant is matched with a University of Minnesota faculty content expert who critiques their completed application using NIH guidelines. In a survey sent to all past P3 participants as of 2018 (n = 194), 88% of respondents reported having submitted their P3-developed NIH grant, and 35% of these submitters reported funding success. A separate analysis of institutional data for all past P3 participants as of 2016 (n = 165) showed that 73% submitted at least 1 NIH proposal since completing P3 and that 43% of these had acquired NIH funding, for a combined total of $193 million in funding awarded. The estimated rate at which participants obtained funding for their P3-developed grant application (~35%) exceeds the national annual NIH grant funding rates (~20%) by approximately 50%. This article provides the practical information needed for other institutions to implement a P3-like program and presents a cost-benefit analysis showing the advantages of doing so.

Low agreement among reviewers evaluating the same NIH grant applications

Obtaining grant funding from the National Institutes of Health (NIH) is increasingly competitive, as funding success rates have declined over the past decade. To allocate relatively scarce funds, scientific peer reviewers must differentiate the very best applications from comparatively weaker ones. Despite the importance of this determination, little research has explored how reviewers assign ratings to the applications they review and whether there is consistency in the reviewers' evaluation of the same application. Replicating all aspects of the NIH peer-review process, we examined 43 individual reviewers' ratings and written critiques of the same group of 25 NIH grant applications. Results showed no agreement among reviewers regarding the quality of the applications in either their qualitative or quantitative evaluations. Although all reviewers received the same instructions on how to rate applications and format their written critiques, we also found no agreement in how reviewers "translated" a given number of strengths and weaknesses into a numeric rating. It appeared that the outcome of the grant review depended more on the reviewer to whom the grant was assigned than the research proposed in the grant. This research replicates the NIH peer-review process to examine in detail the qualitative and quantitative judgments of different reviewers examining the same application, and our results have broad relevance for scientific grant peer review.

Surgeon Scientists Are Disproportionately Affected by Declining NIH Funding Rates

Obtaining National Institutes of Health (NIH) funding over the last 10 years has become increasingly difficult due to a decrease in the number of research grants funded and an increase in the number of NIH applications. National Institutes of Health funding amounts and success rates were compared for all disciplines using data from NIH, Federation of American Societies for Experimental Biology (FASEB), and Blue Ridge Medical Institute. Next, all NIH grants (2006 to 2016) with surgeons as principal investigators were identified using the National Institutes of Health Research Portfolio Online Reporting Tools Expenditures and Results (NIH RePORTER), and a grant impact score was calculated for each grant based on the publication's impact factor per funding amount. Linear regression and one-way ANOVA were used for analysis. The number of NIH grant applications has increased by 18.7% (p= 0.0009), while the numbers of funded grants (p < 0.0001) and R01s (p < 0.0001) across the NIH have decreased by 6.7% and 17.0%, respectively. The mean success rate of funded grants with surgeons as principal investigators (16.4%) has been significantly lower than the mean NIH funding rate (19.2%) (p= 0.011). Despite receiving only 831 R01s during this time period, surgeon scientists were highly productive, with an average grant impact score of 4.9 per $100,000, which increased over the last 10 years (0.15 ± 0.05/year, p= 0.02). Additionally, the rate of conversion of surgeon scientist-mentored K awards to R01s from 2007 to 2012 was 46%. Despite declining funding over the last 10 years, surgeon scientists have demonstrated increasing productivity as measured by impactful publications and higher success rates in converting early investigator awards to R01s.

Multidisciplinary Mentoring Programs to Enhance Junior Faculty Research Grant Success.

Junior faculty face challenges in establishing independent research careers. Declining funding combined with a shift to multidisciplinary, collaborative science necessitates new mentorship models and enhanced institutional support. Two multidisciplinary mentorship programs to promote grant success for junior faculty were established at the Duke University School of Medicine beginning in 2011. These four-month programs-the Path to Independence Program (PtIP) for National Institutes of Health (NIH) R applicants and the K Club for NIH K applicants-use multiple senior faculty mentors and professional grant-writing staff to provide a 20-hour joint curriculum comprising a series of lectures, hands-on workshops, career development counseling, peer groups, and an internal study section. In March 2016, the authors analyzed the success rate for all NIH grants submitted by participants since program enrollment. In a 2015 postprogram survey, participants rated their feelings of support and competency across six skill factors. From October 2011 to March 2016, the programs engaged 265 senior faculty mentors, 145 PtIP participants, and 138 K Club participants. Success rates for NIH grant applications were 28% (61 awards/220 decisions) for PtIP participants-an increase over the 2010 Duke University junior faculty baseline of 11%-and 64% (38/59) for K Club participants. Respondents reported significantly increased feelings of support and self-ratings for each competency post program. The authors plan to expand the breadth of both the mentorship pool and faculty served. Broad implementation of similar programs elsewhere could bolster success, satisfaction, and retention of junior faculty investigators.

‘Your comments are meaner than your score’: score calibration talk influences intra- and inter-panel variability during scientific grant peer review

In scientific grant peer review, groups of expert scientists meet to engage in the collaborative decision-making task of evaluating and scoring grant applications. Prior research on grant peer review has established that inter-reviewer reliability is typically poor. In the current study, experienced reviewers for the National Institutes of Health (NIH) were recruited to participate in one of four constructed peer review panel meetings. Each panel discussed and scored the same pool of recently reviewed NIH grant applications. We examined the degree of intra-panel variability in panels' scores of the applications before versus after collaborative discussion, and the degree of inter-panel variability. We also analyzed videotapes of reviewers' interactions for instances of one particular form of discourse-Score Calibration Talk-as one factor influencing the variability we observe. Results suggest that although reviewers within a single panel agree more following collaborative discussion, different panels agree less after discussion, and Score Calibration Talk plays a pivotal role in scoring variability during peer review. We discuss implications of this variability for the scientific peer review process.

Local Citation Analysis of National Science Foundation and National Institutes of Health Grant Applications: Meeting the Needs of Researchers

Supporting and meeting the research needs of faculty members is a priority at the University of Kansas because of the necessity for external grant-funding. For this study, the authors analyzed the citations from successful National Institutes of Health (NIH) and National Science Foundation (NSF) grant applications submitted between 2005 and 2013. The purpose was to identify the types of resources used by researchers and determine if KU Libraries are currently providing access. In addition to access, the authors analyzed the age, format, whether journals were provided in a journal package or aggregator, subject area, and open access status. Overall, the authors found KU Libraries’ collection provides substantial support for researchers who submitted successful NSF and NIH grant applications.

‘Kicking the can’: science, Congress and a Ponzi scheme

AIDS researcher Yuntao Wu, Professor of microbiology and infectious diseases at George Mason University (Manassas, VA USA), is finding imaginative ways to maintain his cell lines, not to mention his life's work on developing therapeutics against HIV infection. As his funding from the US National Institutes of Health (NIH) has dried up, Wu has become dependent on the kindness of strangers and friends to cover the US$3,000 a month he needs to keep his lab running. To help repay a US$35,000 loan from his university, he has so far raised more than US$21,000 on CrowdRise, a crowd‐sourced fundraising website. In previous years, Wu had little trouble funding his research the ‘usual’ way. He received more than US$1 million in grant money from NIH over the past 4 years, and his lab published 18 papers in that period. But his application to renew the grant for another 5 years was rejected this past spring. Wu blames the tight NIH budget. Desperate for funds, he is now applying to other sections of the US National Institute for Allergy and Infection Diseases to see whether they will be more forthcoming than the molecular biology section that did not renew his grant. He expects to hear this month whether he will receive any NIH money, as the 16‐day government shutdown in October last year delayed the grant review process. Wu's grant applications are not the only ones to be affected by the shutdown: The closure of the government temporarily delayed the assessment of 300,000 extramural research groups for funding. Another 14,000 NIH scientists—designated as ‘non‐essential’ federal employees—were sent home, virtually closing some laboratories. Though the obvious effects of the shutdown were temporary, there are concerns about the longer‐term consequences of the unstable funding environment. In an 11th hour deal, leaders from the conservative‐dominated House of Representatives …

Perspective: Is NIH Funding the “Best Science by the Best Scientists”? A Critique of the NIH R01 Research Grant Review Policies

Clinical and experimental biomedical research provides the foundation for advances in medicine, health, and the welfare of the public. The National Institutes of Health (NIH) is the major agency providing funding for biomedical research. The stated objectives of the NIH for funding research grants (R01s) are to "fund the best science, by the best scientists" and "to see that NIH grant applications receive fair, independent, expert, and timely reviews-free from inappropriate influences-so NIH can fund the most promising research." The NIH recently reviewed and identified issues involved with the study section peer review process that compromise the achievement of these laudable and important objectives. Consequently, the NIH has and continues to issue new guidelines and requirements relating to the R01 grant review process. The author argues that some of these NIH directives conflict with and counteract the achievement of the NIH's stated objectives. The author further contends that the directives introduce discrimination into the review process. Such conditions impede the funding of the best science by the best scientists, while funding lesser-quality research. The NIH should eliminate all directives that prevent R01 grants from being awarded solely to the highest-quality research. This is in the best interest of the biomedical community and the health and welfare of the public at large.

The vanishing physician-scientist?

The vanishing physician-scientist?

Francis Collins: Director of the US National Institutes of Health

Francis Collins: Director of the US National Institutes of Health

Anesthesiology physician scientists in academic medicine: a wake-up call.

Physician scientists perform biomedical research ranging from fundamental molecular studies to clinical trials. As such, physician scientists have a pivotal role in the biomedical research enterprise. The foundation for physician scientist leadership in the subspecialty of anesthesiology is highly dependent on the success of young investigators as they develop independent careers involving basic and clinical investigation. Hence, a leading priority of anesthesiology must be to nurture the academic careers of physician scientists within the specialty. Over the past decade, during a period of budgetary expansion by the National Institutes of Health (NIH), the number of NIH grant applications from physicians, in contrast to the number of grant proposals from Ph.D. scientists, has been slow to increase. Likewise, growth in NIH awards to physicians working in anesthesiology departments has also been slow.1 Junior clinical anesthesiology faculty across the country find their ability to develop NIH-funded research programs limited by inadequate or overly fragmented research start-up time and by inconsistent approaches to research mentoring. In concert, the recent abrupt decrease in rate of growth of NIH funding now threatens not only the pace of our progress, but survival of physician scientist activity within anesthesiology. Based on these observations, anesthesiology is at risk of losing its status as a respected academic discipline within the broader biomedical community. Based on historic analysis of periods of limited NIH budgets, critical analysis of anesthesiology research training programs relative to peer academic specialties,1 and considering the forecasted continued high demand for clinical manpower in our academic medical centers, we present the case for radical change in our approach to training within the specialty. Although effective solutions will require significant sacrifice by all sectors of the specialty, we propose that modest steps are far too late and will no longer succeed. This article, while seeking to inform and educate, is above all a passionate call for decisive action.

How will we solve the critical shortage of clinician-scientists in psychiatry?

The number of graduating psychiatry residents who choose a career in academic medicine is remarkably small, and the percentage who become National Institutes of Health (NIH)-funded investigators is even smaller. Although this trend for a reduced number of physician-scientists is true in all branches of medicine and has reached criticialproportions, the perception is that this shortage is even more severe in psychiatry. The purpose of this essay is to increase awareness of the problem and begin a discussion of the reasons why the field finds itself in this problematic situation.First, however, it is important to more clearly define the nature of the problem. At the current time, NIH funding for research and training grants (Research Career Development Awards) has attained unprecedentedly high levels, with the percent of submitted applications that are funded higher than ever thought possible. Physicians who submit NIH grant applications are as successful as their PhD colleagues, dispelling the myth that physicians cannot compete with PhDs for such funding. Others have raised the question as to whether physicians are necessary or even desirable as investigators. In other words, the entire concept of a clinician-scientist is considered by some to be simply passé.

Federal training requirements for responsible research: not going far enough.

In what is clearly an important development related to research integrity and the protection of human research subjects, the U.S. government has instituted two new training requirements as a condition of receiving federal financial support. First, the National Institutes of Health (NIH) is requiring, as a condition of funding, that key research personnel involved in human subject research complete education “in the protection of human subjects.” Evidence that key personnel have completed this training must be provided in NIH grant applications or contract proposals.The NIH education policy will eventually be superseded by a more broadly applicable instructional policy for the “responsible conduct of research,” which will be promulgated by the Department of Health and Human Service's Office of Research Integrity and the Public Health Service (PHS). The instructional policy will apply to all persons engaged in any research or research training with PHS support. Presently, the only version of the policy is in draft form.

NIH Funding and Career Development Opportunities for Women in Reproductive Immunology.

The National Institutes of Health (NIH) has a number of mechanisms available to support research, research training and research career development. The various institutes, centers and special offices within the NIH have, as part of their missions, a diverse array of basic and clinical research goals. Much of the scientific spectrum of reproductive immunology, however, falls within the mission of the National Institute of Child Health and Human Development (NICHD). There have been a number of recent innovations in the preparation, review and funding of NIH grant applications. Specific opportunities, such as fellowships, career awards, and new investigator status, are available for investigators at different career stages. A wealth of information is available on the NIH website at www.nih.gov. NIH staff are available to answer your questions and to provide administrative guidance in preparing grant applications.

Research and emergence of the Lifeline Foundation

Research and emergence of the Lifeline Foundation

Research Proposals: A Guide to Success

The author is a veteran investigator whose career as a scientist includes participation in the National Institutes of Health (NIH) member study section, associate deanship at the University of Southern California, and directing research development at the Doheny Eye Institute. Such experiences have caused him to recognize the need for a no-nonsense, straightforward set of instructions on how to prepare an NIH research grant. Today, when medical research funding is said to be in a crisis, when only a small fraction of NIH grant applications are funded, and when a career in biomedical research is much less attractive to young people, this is a much-needed text. Dr Ogden's theme can be expressed in the terms of form and substance. Good science—substance—can be hampered in the review process by a poorly prepared proposal—form. Poor or ordinary science cannot be helped. Thus, how to put one's best foot forward is the theme.