Thought Exercises on Accountability and Performance Measures at the National Heart, Lung, and Blood Institute (NHLBI): An Invited Commentary for Circulation Research

Abstract

Recently, popular magazines and newspapers broadcast skeptical headlines, such as, “Desperately Seeking Cures”1; “A Decade Later, Genetic Map Yields Few Cures”2; “Faltering Cancer Trials”3; and “Grant System Leads Cancer Researchers to Play It Safe.”4 Some patients, physicians, advocacy groups, journalists, scholars, and policymakers openly wonder about the value of billions of dollars in government-supported research. During the past 10 years, the NIH budget has doubled, yet cardiovascular disease, while decreasing in incidence and severity, is still the leading cause of death and cancer incidence and death rates have declined little. Along with “Where Are the Cures?” critics ask “Who Is Accountable?”5 There is a rapidly growing body of scholarship on the metrics of science, methods by which scientists, scientific organizations, policymakers, and funding agencies can gauge the value and impact of their work and investments.6 To assess the impact of the 2009 American Recovery and Reinvestment Act (ARRA), the National Science Foundation, the National Institutes of Health, and the White House Executive Office are engaged in an ambitious “STAR METRICS” project.7 Metrics often focus on publications and citations (“bibliometrics”) but also consider commercial products and quantitative impacts on practice or intellectual thought. Although scientists are typically eager to publish their work and see it cited, many worry about seeing their value to society, employers, and funders graded by black-box numbers.8 Some professional groups have harshly criticized bibliometrics, noting their inherent flaws and the risk that the act of measuring science will damage innovation, which is at the heart of the scientific enterprise.9 At NHLBI, we increasingly recognize that we need to be held accountable for our performance, which means that it is incumbent on us to assess the performance of the scientists and projects we choose to support. …