Peer Review of Research Data Submissions to ScholarsArchive@OSU: How can we improve the curation of research datasets to enhance reusability?

Clara Llebot,Steven Van Tuyl

doi:10.7191/jeslib.2019.1166

Clara Llebot, Steven Van Tuyl

Open Access

https://doi.org/10.7191/jeslib.2019.1166

Copy DOI

Abstract

Objective: Best practices such as the FAIR Principles (Findability, Accessibility, Interoperability, Reusability) were developed to ensure that published datasets are reusable. While we employ best practices in the curation of datasets, we want to learn how domain experts view the reusability of datasets in our institutional repository, ScholarsArchive@OSU. Curation workflows are designed by data curators based on their own recommendations, but research data is extremely specialized, and such workflows are rarely evaluated by researchers. In this project we used peer-review by domain experts to evaluate the reusability of the datasets in our institutional repository, with the goal of informing our curation methods and ensure that the limited resources of our library are maximizing the reusability of research data. Methods: We asked all researchers who have datasets submitted in Oregon State University’s repository to refer us to domain experts who could review the reusability of their data sets. Two data curators who are non-experts also reviewed the same datasets. We gave both groups review guidelines based on the guidelines of several journals. Eleven domain experts and two data curators reviewed eight datasets. The review included the quality of the repository record, the quality of the documentation, and the quality of the data. We then compared the comments given by the two groups. Results: Domain experts and non-expert data curators largely converged on similar scores for reviewed datasets, but the focus of critique by domain experts was somewhat divergent. A few broad issues common across reviews were: insufficient documentation, the use of links to journal articles in the place of documentation, and concerns about duplication of effort in creating documentation and metadata. Reviews also reflected the background and skills of the reviewer. Domain experts expressed a lack of expertise in data curation practices and data curators expressed their lack of expertise in the research domain. Conclusions: The results of this investigation could help guide future research data curation activities and align domain expert and data curator expectations for reusability of datasets. We recommend further exploration of these common issues and additional domain expert peer-review project to further refine and align expectations for research data reusability. The substance of this article is based upon a panel presentation at RDAP Summit 2019.

Highlights

Managing research data with care, and sharing research outcomes with the public is becoming an expectation in academia
Domain experts expressed a lack of expertise in data curation practices and data curators expressed their lack of expertise in the research domain
Domain experts and data curators coincide in describing difficulties when data curation guidance requires that the same information will be duplicated and when some of the information is given via links

Summary

Introduction

Managing research data with care, and sharing research outcomes with the public (including publications and datasets) is becoming an expectation in academia. Journals that require publication of data underlying the research are becoming increasingly common (Dearborn et al 2018). The goal of these policies and recommendations is clear: maximize access to research data to ensure that research is more reproducible, and make data that is more reusable. Requiring a data management plan with research grant proposals and requiring or recommending sharing the data generated during the research will usually achieve this. Research suggests that funders and journals are ineffective at convincing researchers to share their data (Savage and Vickers 2009), and that when data is shared, it often has issues (e.g., lack of documentation, inadequate formats) that make it difficult to reuse the data (Van Tuyl and Whitmire 2016, Naudet et al 2018)

Methods

Results

Discussion

Conclusion