Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature

Abstract

The past decade has been marked by concerns regarding the replicability and reproducibility of published research in the social sciences. Publicized failures to replicate landmark studies, along with high-profile cases of research fraud, have led scholars to reconsider the trustworthiness of both findings and institutionalized research practices. This paper considers two questions: (1) Relative to psychology and economics, what is the state of replication and reproduction research in management? (2) Are the disciplines equally advanced in the use of methods applied to study the replication problem? A systematic literature review identified 67 studies pertinent to these questions. The results indicate that the replication prevalence rate in management studies lies almost exactly between those of psychology and economics, while a high level of variation between management and other business-related disciplines can be noted. Further, similarly to psychology, but unlike economics, the surveys of published replications tend to report high replication success rates for management and other business-related disciplines. However, a comparison with recently obtained results in preregistered multi-study replications in psychology and economics suggests that these rates are almost certainly inflated. Method and data transparency are medium to low, often rendering attempts to reproduce or replicate studies impossible. Finally, the understanding of the replicability problem in management is held back by the underutilization of methods developed in other disciplines. The review also reveals that management, psychology, and economics exhibit strikingly different practices and approaches to replication, despite facing similar incentive structures. Disciplines in which replication and reproduction attempts are rare and which frequently involve authors of the original study in replication attempts lack strong deterrents against questionable research practices; thus, they are less likely to deliver replicable results.