Bibliometrics approach to evaluating the research impact of CTSAs: A pilot study.

Fei Yu,Gaurav Dave,Tanha Patel,Allison Alicia Van,Nandita Mani,Giselle M Corbie-Smith,John Buse,Andrea Carnegie,Timothy Carey

doi:10.1017/cts.2020.29

Abstract

To enhance the performance evaluation of Clinical and Translational Science Award (CTSA) hubs, we examined the utility of advanced bibliometric measures that go beyond simple publication counts to demonstrate the impact of translational research output. The sampled data included North Carolina Translational and Clinical Science Institute (NC TraCS)-supported publications produced between September 2008 and March 2017. We adopted advanced bibliometric measures and a state-of-the-art bibliometric network analysis tool to assess research productivity, citation impact, the scope of research collaboration, and the clusters of research topics. Totally, 754 NC TraCS-supported publications generated over 24,000 citation counts by April 2017 with an average of 33 cites per article. NC TraCS-supported research papers received more than twice as many cites per year as the average National Institute of Health-funded research publications from the same field and time. We identified the top productive researchers and their networks within the CTSA hub. Findings demonstrated the impact of NC TraCS in facilitating interdisciplinary collaborations within the CTSA hub and across the CTSA consortium and connecting researchers with right peers and organizations. Both improved bibliometrics measures and bibliometric network analysis can bring new perspectives to CTSA evaluation via citation influence and the scope of research collaborations.

Highlights

To enhance the performance evaluation of Clinical and Translational Science Award (CTSA) hubs, we examined the utility of advanced bibliometric measures that go beyond simple publication counts to demonstrate the impact of translational research output
While the CTSA consortium primarily focuses on clinical and translational research conducted within the USA, the findings showed that collaborations occurred worldwide
Since the research output productivity depends on the funding scale and citation counts build up over time, the bibliometric comparison across CTSAs can be biased. This pilot study adopted bibliometrics and bibliometric network analysis to measure the research output and collaboration impact of a CTSA program hub. Both the traditional measures and the improved new fieldand time-normalized measures developed by Scopus and National Institutes of Health (NIH) iCite, i.e., Field-Weighted Citation Impact (FWCI), Citation Benchmarking (CB), and Relative Citation Ratio (RCR), were applied to evaluate NC TraCS-supported publications from 2008 to 2017

Summary

Introduction

To enhance the performance evaluation of Clinical and Translational Science Award (CTSA) hubs, we examined the utility of advanced bibliometric measures that go beyond simple publication counts to demonstrate the impact of translational research output. We adopted advanced bibliometric measures and a state-of-the-art bibliometric network analysis tool to assess research productivity, citation impact, the scope of research collaboration, and the clusters of research topics. The National Center for Advancing Translational Sciences (NCATS) at NIH has since funded and supported CTSA hubs at more than 50 medical research institutes across the United States [2]. These CTSA hubs engage in a wide variety of institutional activities, all aiming to increase the quality, transparency, translation, and reproducibility of scientific research and to “get more treatments to more patients more quickly” [3]. At each CTSA hub, evaluation is conducted to assess its activities, outcomes, and impact on the translational research enterprise

Objectives

Methods

Results

Discussion

Conclusion