Abstract
We investigate how often replication studies are published in empirical economics and what types of journal articles are replicated. We find that between 1974 and 2014 0.1% of publications in the top 50 economics journals were replication studies. We consider the results of published formal replication studies (whether they are negating or reinforcing) and their extent: Narrow replication studies are typically devoted to mere replication of prior work, while scientific replication studies provide a broader analysis. We find evidence that higher-impact articles and articles by authors from leading institutions are more likely to be replicated, whereas the replication probability is lower for articles that appeared in top 5 economics journals. Our analysis also suggests that mandatory data disclosure policies may have a positive effect on the incidence of replication.
Highlights
In times of increasing publication rates, cases of inadvertent errors and failure to fully report the fragility or robustness of results the mechanisms of quality control in science are put under scrutiny (McNutt, 2015)
In Sheshinksi (1971) which is ranked 99th in terms of being a replication study the key word "note" appears in the title whereas all other key stem words do not appear in either title or abstract. 20 http://replication.uni-goettingen.de/wiki/index.php/Main_Page 21 In the supplementary materials, we provide an overview of the replication studies and respective replicated articles
Exploring the subsample of articles published in or after 2004, we find evidence for a statistically significant positive effect of mandatory data disclosure policies on the replication probability
Summary
In times of increasing publication rates, cases of inadvertent errors and failure to fully report the fragility or robustness of results the mechanisms of quality control in science are put under scrutiny (McNutt, 2015). Considering that an increasingly powerful science infrastructure makes it possible to store and retrieve data and code online at low costs, replications become more feasible. This is reinforced by the growing demand from publishers, science funders, and policy makers to make these materials available (e.g., Bohannon, 2015; Hoeffler, 2017). In 1982, the Journal of Money, Credit and Banking (JMCB) initiated the JMCB Data Storage and Evaluation Project (Dewald et al, 1986) Within this project, the JMCB required authors to make the data and code used in their articles available to other researchers upon request. These results raised concerns with respect to the technical and factual reproducibility of empirical work in economics
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
