Using open infrastructures for alternative quality assessment of research outputs

Abstract

Decision-makers are not able to follow all the changes in the open science (OS) on the policy level as fast as needed. This affects the way how research outputs are assessed. Due to this, not all research outputs are recognised, and there are no mechanisms to trace and appraise all outputs. This paper presents open infrastructures and supporting methods which are capable of overcoming these deficiencies. The Scopus and Web of Science citation indexes have a significant impact on the quality assessment of research outputs, especially when it comes to research promotion and career development. At the same time, researchers produce scientific outputs that cannot be assessed using traditional approaches (Pravilnik o sticanju istraživačkih i naučnih zvanja, 2020).Open infrastructures can support research communities by providing methods which are verifiable and available for integration with institutional infrastructures (such as repositories and CRIS systems). There are also some services that do not fall into the category of open infrastructure but support the concept by providing technical solutions that can be open access. Also, there is a tendency to make open infrastructures as compliant with OS principles as possible. Some of these infrastructures implement other closely related principles such as the FAIR data principles (GO FAIR initiative, 2022). When an institutional infrastructure adopts these principles and integrates services provided by open infrastructures, then benefits are seen by both researches and decision-makers. Dimensions is an alternative to and more open and inclusive than a traditional citation database. Also, Dimensions is integrated with other infrastructures that embrace the concept of openness (Dimensions, 2019). Thanks to functionalities available through various open application programming interfaces (APIs), Dimensions has played an important part in technical adoption of OS principles in Serbia.Alternative metrics like Altmetric could serve as another solution (Altmetric, 2018). Like Dimensons, Altmetric is not an open infrastructure, but it contributes to open infrastructures through open APIs that can track the impact of research outputs across the Internet. The main focus is on tracking research papers on social media (Facebook, Twitter) and multimedia platforms (YouTube) or their mentions in Internet communities. An important issue for each country is how to assess research outputs in humanities. Knowing that research outputs in this area focus mainly on locally relevant topics, it is difficult to track the impact of such results by relying on traditional citation databases.Unlike Dimensions and Altmetric, OpenCitations is an open infrastructure which fully embraces the concept of OS and is compliant with the FAIR data principles inasmuch as the data it collects are findable, accessible, interoperable, and re-usable. Furthermore, OpenCitations has adopted the Principles of Open Scholarly Infrastructure (POSI) (Bilder, Lin and Neylon, 2020) and its infrastructure has been upgraded with a system that provides persistent identifiers (PIDs) (Peroni, Shotton, 2019). All these solutions are mostly dealing with citations. However, there are research outputs like peerreview reports and editorial work that cannot be assessed only by the number of citations and require other metrics. Publons is far from an open infrastructure but at least provides some open access services for researchers. More relevant in this context is the recently launched OpenAlex, an open infrastructure that can serve as a search engine, research output monitor, and assessment tool. OpenAlex aggregates data from sources like Crossref and ORCID and seeks to present data in a standardised way