ARE REPLICATION STUDIES INFREQUENT BECAUSE OF NEGATIVE ATTITUDES?

Kevin Mcmanus

doi:10.1017/s0272263121000838

Abstract

AbstractReplication is a research methodology designed to verify, consolidate, and generalize knowledge and understanding within empirical fields of study. In second language studies, however, reviews share widespread concern about the infrequency of replication. A common but speculative explanation for this situation is that replication studies are not valued because they lack originality and/or innovation. To better understand and respond to the infrequency of replication in our field, 354 researchers were surveyed about their attitudes toward replication and their practices conducting replication studies. Responses included worldwide participation from researchers with and without replication experience. Overall, replications were evaluated as relevant and valuable to the field. Claims that replication studies lack originality/innovation were not supported. However, dissemination issues were identified: half of published replication studies lacked explicit labeling and one quarter of completed replications were unpublished. Explicit labeling of replication studies and training in research methodology and dissemination can address this situation.

Highlights

Replication is a research methodology designed to verify, consolidate, and generalize knowledge and understanding within empirical fields of study
In a survey of 73 researchers in computer science education (Ahadi et al, 2016), respondents agreed that “original studies are more prestigious than replication studies” (p. 7) and thought that replications contributed little toward citation and grant success
Results reported for specific survey questions are cross-referenced to the supplementary materials

Summary

Introduction

Replication is a research methodology designed to verify, consolidate, and generalize knowledge and understanding within empirical fields of study. To better understand and respond to the infrequency of replication in our field, 354 researchers were surveyed about their attitudes toward replication and their practices conducting replication studies. As reviews of the field have repeatedly noted, replication studies are critically needed to consolidate and strengthen the field’s evidence base (Marsden et al, 2018; Porte, 2012) This is because replication allows us to better understand how a study’s research data were collected, measured, and analyzed, as well as the extent to which unexpected and/or unanticipated factors potentially influenced the results (Porte & McManus, 2019; Schmidt, 2009).

Methods

Results

Discussion

Conclusion