Abstract

In response to widespread concerns about the integrity of research published in scholarly journals, several initiatives have emerged that are promoting research transparency through access to data underlying published scientific findings. Journal editors, in particular, have made a commitment to research transparency by issuing data policies that require authors to submit their data, code, and documentation to data repositories to allow for public access to the data. In the case of the American Journal of Political Science (AJPS) Data Replication Policy, the data also must undergo an independent verification process in which materials are reviewed for quality as a condition of final manuscript publication and acceptance.
 Aware of the specialized expertise of the data archives, AJPS called upon the Odum Institute Data Archive to provide a data review service that performs data curation and verification of replication datasets. This article presents a case study of the collaboration between AJPS and the Odum Institute Data Archive to develop a workflow that bridges manuscript publication and data review processes. The case study describes the challenges and the successes of the workflow integration, and offers lessons learned that may be applied by other data archives that are considering expanding their services to include data curation and verification services to support reproducible research.

Highlights

  • Recent initiatives such as ‘Data Access and Research Transparency (DA-RT): A Joint Statement by Political Science Journal Editors’ (Lupia and Elman, 2014) and the ‘Guidelines for Transparency and Openness Promotion in Journal Policies and Practice (TOP Guidelines)’ (Nosek et al, 2015) have demonstrated the scientific community’s renewed focus on the replication standard of data quality

  • Successful integration of data review activities into the scholarly communication workflow requires a great deal of consideration regarding the roles and responsibilities of the editorial staff that issues data policy, the author who must comply with the data policy, and the data curator who enforces the data policy

  • The feasibility of journals and data archives to do this is very much dependent on collaborative efforts to develop tools and processes that streamline and automate an integrated data review/manuscript publication workflow. Such a collaboration is a reflection of the narrowing gap among the work of authors, journal editors and data curators – authors being the source of research reported in submitted articles, along with the data used to conduct the research

Read more

Summary

Introduction

Recent initiatives such as ‘Data Access and Research Transparency (DA-RT): A Joint Statement by Political Science Journal Editors’ (Lupia and Elman, 2014) and the ‘Guidelines for Transparency and Openness Promotion in Journal Policies and Practice (TOP Guidelines)’ (Nosek et al, 2015) have demonstrated the scientific community’s renewed focus on the replication standard of data quality. Editor of the American Journal of Political Science (AJPS), issued a statement on March 26, 2015 announcing this new addition to the AJPS replication policy. He wrote, “Research transparency and replicability of results are standards to which the discipline traditionally has paid lip service. The new AJPS replication policy requires scholars to ‘practice what we preach’ and adhere to these standards in a meaningful way” (Jacoby, 2015) True to this pronouncement, the new AJPS policy requires replication files to undergo a successful independent verification process as a condition of final manuscript acceptance and publication in addition to the existing policy’s requirement that authors upload replication files to the AJPS Dataverse repository. We offer our view of the challenges and opportunities for other data repositories anticipating or exploring this potential data quality assurance role for data archives

The Replication Standard
Operationalizing the Replication Standard
The Data Review Workflow
Lessons Learned and Recommendations
Data Review Requires Commitment from All Stakeholders
The Data Archive Remains Neutral in the Manuscript Publication Process
Data Review Requires Specialized Expertise
Disparate Systems Call for Desperate Measures
Producing Quality Replication Datasets Presents Challenges
Data Review Services May Not Be Scalable Given Existing Resources
Conclusion
Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.