Abstract
AbstractDespite its critical role for the development of the field, little is known about replication in second language (L2) research. To better understand replication practice, we first provide a narrative review of challenges related to replication, drawing on recent developments in psychology. This discussion frames and motivates a systematic review, building on syntheses of replication in psychology, education, and L2 research. We coded 67 self‐labeled L2 replication studies found across 26 journals for 136 characteristics. We estimated a mean rate of 1 published replication study for every 400 articles, with a mean of 6.64 years between initial and replication studies and a mean of 117 citations of the initial study before a replication was published. Replication studies had an annual mean of 7.3 citations, much higher than averages in linguistics and education. Overlap in authorship between initial and replication studies and the availability of the initial materials both increased the likelihood of a replication supporting the initial findings. Our sample contained no direct (exact) replication attempts, and changes made to initial studies were numerous and wide ranging, which likely obscured, if not undermined, the interpretability of replication studies. To improve the amount and quality of L2 replication research, we propose 16 recommendations relating to rationale, nomenclature, design, infrastructure, and incentivization for collaboration and publication.Open PracticesThis article has been awarded Open Materials and Open Data badges. All materials and data are publicly accessible via the IRIS Repository athttps://www.iris-database.org/iris/app/home/detail?id=york:934328. Learn more about the Open Practices badges from the Center for Open Science:https://osf.io/tvyxz/wiki.
Highlights
The driving force behind this battery of calls, commentaries, infrastructure, and metascience is a perceived crisis in the state of replication research
The current synthesis did not aim to evaluate the reproducibility of L2 research, partly determined by the fact that we found no direct replications
B) What were the designs and contexts of the initial studies? c) What were the participant characteristics in the initial studies? d) To what extent were the initial study’s materials accessible? RQ3 To what extent and how did researchers change the initial L2 studies? a) To what extent did the amount of change between initial and replication studies relate to nomenclature of replications? RQ4 To what extent did L2 replications support the initial studies’ findings?
Summary
Note. aThe year 2015 was the last complete year captured by our synthesis. bTo calculate the denominator (total articles published), we used a start date of either 1973 (the date of our first replication) or the start of the journal if that fell after 1973. Two of the four journals that stated they accepted replications emphasized originality in the first sentence of their aims/scope sections Three of these four journals reserved specific strands for replications (SSLA, LT, SLR). Two journals explicitly mentioned that null findings would not be grounds for rejection per se (SLR and Language Testing, both journals that encouraged originality and explicitly accepted replications).3 Note that beyond this analysis of the number of replications published by journals and the explicit and implicit messages that journals send to authors, we cannot know how the replication rate of L2 research reflects the extent to which authors submit replications that are rejected versus the extent to which replications are not submitted. Article citation counts and journal impact factors for replication and initial studies
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
