Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals

Mark J Mccabe,Frank Mueller-Langer

doi:10.2139/ssrn.3329272

Abstract

Does data disclosure have an impact on citations? Four leading economics journals introduced a data disclosure policy between 2004 and 2006. We use panel data consisting of 17,135 article citing-year observations from 1996 to 2015 for articles published in these journals. Empirical articles that did not disclose data (46% of the sample) serve as a control group. Evidence for a positive open data citation effect is weak (6% and not statistically significant). On the other hand, the citation impacts of publication are substantial and precisely estimated. Pure theory, hybrid and purely empirical articles enjoy citations benefits of 22%, 32% and 44%, respectively. Our pre- and post-publication citation data allow us to identify the citation effects of data disclosure and publication, while controlling for intrinsic article quality.

Highlights

Our evidence of negligible aggregate open data benefits may provide an additional explanation for why empirical researchers are typically reluctant to disclose their data voluntarily
Our econometric analysis indicates that citation effects due to publication vary by article type; these publication effects are greater for empirical articles, for purely empirical articles
We examine the citation effect of data disclosure on empirical articles published in the American Economic Review, Econometrica, the Journal of Political Economy, and the Review of Economic Studies

Summary

Introduction

Open access to research data has attracted attention from economists (Dewald et al, 1986; Glandon, 2011; Hamermesh, 2007; McCullough et al, 2006; McCullough and Vinod, 2003) and policymakers (Burgelman et al, 2010; Doldirina et al, 2015; European Commission, 2012, 2016 & 2017; ESRC, 2019; NIH, 2003; NSF, 2014; OECD, 2007; US House of Representatives, 2007). Horizon 2020 projects must deposit their research data in a research data repository.1 This subject sparks a fierce debate, whether due to data fraud (see Levelt Committee et al (2012) for a discussion of the scandal surround the social psychologist Diederik Stapel) or mistakes, e.g. the coding errors made by two leading economists, Reinhart and Rogoff, in a pair of 2010 papers (see Herndon et al (2014) and Bell et al (2015)).. Data disclosure is essential for the academic community and science policy It improves the quality of research results, increases the efficiency of the academic system and pushes subsequent research (Anderson et al, 2008; Furman and Stern, 2011; McCullough et al, 2008; Nature, 2009).

Objectives

Methods

Findings

Discussion

Conclusion