Are Replications in Management Research a Wasted Effort? An Analysis of Co-Citation Patterns

Abstract

The past decade has witnessed great interest in replication research caused by widely publicized failures to replicate a significant share of research in psychology. However, much of the discussion implicitly assumes that replications lead to reevaluations of the original findings. We examined this assumption by drawing on a sample of 137 original study-replication pairs in management and organization studies. Study 1 analyzed the direct impact of replication by studying original-replication co-citation patterns. An analysis of data for 6,965 unique citing sources showed that over 92% of journal articles and conference proceedings published after the replication was performed failed to co-cite it with the original study, whereas papers published in higher ranking journals were more likely to co-cite replications. Dependent replications were more likely to be co-cited, whereas no support was found for the hypothesis that failed replications are more likely to be co-cited. Study 2 analyzed the indirect impact of replication through meta-analyses. Half of the replications in our sample had never been meta-analyzed. For the remaining cases, we used data for 3,501 citing sources, showing that 90% of them failed to co-cite meta-analysis when citing the original study, while the rank of citing journals was positively associated with co-citations. However, the replication-covering meta-analyses were cited 30,138 times, thus having an immense impact on the field. Our results show that replications tend to be ignored when original studies are referenced but have a considerable impact through meta-analyses. We offer suggestions on editorial policy intended to increase the use of replications.