Abstract
Background: There is a growing interest in using and analysing Altmetrics data for quantifying the impact of research, especially societal impact (Haunschild & Bornmann 2017, Thelwall et al 2016, Bornmann 2014). This study therefore aimed to explore the value of Altmetrics for assessing policy impact by analysing research articles from a single centre (the University of Sheffield) that were identified by Altmetrics as being cited in at least one policy document. Method: This study has only included published research articles from authors at the University of Sheffield and indexed in the Altmetric.com database. Altmetric data on policy impact were sourced from Altmetric.com following a data request and included citations up until February 2017. Supplementary Altmetric data including news media, blogs, Mendeley saves and Wikipedia citations were also gathered. Results: Altmetrics did enable an assessment of whether a relevant research article had had an impact on policy documents and, if so, how many such documents. In total, 1463 pieces of published research from authors at the University of Sheffield were found to be cited in between 1 and 13 policy documents. Twenty-one research articles were listed as being cited in five or more policy documents; 21 in four policy documents; 50 in three documents; 186 in two documents; and 1185 in one document. Of those 1463 outputs, 1449 were journal articles, 13 were books and one was a book chapter, and the time lag from publication to citation in policy documents ranged from 3 months to 31 years. Analysis of the 92 research articles cited in three or more policy documents indicated that the research topics with the greatest policy impact were medicine, dentistry and health, followed by social science and pure science. The Altmetrics data enabled an in-depth assessment of the 21 research articles cited in five or more policy documents. Conclusion. Within the limitations of the current text-mining system, Altmetrics can offer important and highly-accessible data on the policy impact of an organization’s published research articles.
Highlights
Altmetrics have only been in existence since 2010 and are already starting to highlight useful pieces of information on how a research output is communicated and shared on the web
The analysis of the available data principally consisted of the tabulation and discussion of descriptive statistics and frequencies, e.g., the reporting of numbers of relevant research articles cited in policy documents; the categorization of these articles by academic faculty or field; the numbers and sources of policy documents
Details of the Total Sample Altmetric.com did enable an assessment of whether a relevant research article had an impact on policy documents and, if so, how many such documents
Summary
Altmetrics have only been in existence since 2010 and are already starting to highlight useful pieces of information on how a research output is communicated and shared on the web. Citation analysis and peer review are still the principal approaches taken in measuring impact (Booth, 2016), but they fail to take into account that research is being communicated, shared, downloaded, and saved to reference management tools across the web. These traditional metrics have focused on journals and authors, and not article-level outputs. This study aimed to explore the usefulness of Altmetric.com data as a means of identifying and categorizing the policy impact of research articles from a single center (the University of Sheffield)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
