Towards a replication culture in phonetic research: Speech production research in the classroom

Timo B Roettger,Dinah Baer-Henney

doi:10.3765/pda.v1art4.13

Timo B Roettger, Dinah Baer-Henney

Open Access

https://doi.org/10.3765/pda.v1art4.13

Copy DOI

Abstract

Our understanding of human sound systems is increasingly shaped by experimental studies. What we can learn from a single study, however, is limited. It is of critical importance to evaluate and substantiate existing findings in the literature by directly replicating published studies. Our publication system, however, does not reward direct replications in the same way as it rewards novel discoveries. Consequently, there is a lack of incentives for researchers to spend resources on conducting replication studies, a situation that is particularly true for speech production experiments, which often require resourceful data collection procedures and recording environments. In order to sidestep this issue, we propose to run direct replication studies with our students in the classroom. This proposal offers an easy and inexpensive way to conduct large-scale replication studies and has valuable pedagogical advantages for our students. To illustrate the feasibility of this approach, we report on two classroom-based replication studies on incomplete neutralization, a speech phenomenon that has sparked many methodological debates in the past. We show that in our classroom studies, we not only replicated incomplete neutralization effects, but our studies yielded effect magnitudes comparable to laboratory experiments and meta analytical estimates. We discuss potential challenges to this approach and outline possible ways to help us substantiate our scientific record.

Highlights

Our understanding of human speech and its cognitive underpinnings is increasingly shaped by experimental data
The Open Science Collaboration (2015) tried to replicate 100 studies that were published in three high-ranking psychology journals, assessing whether the replications and the original experiments
Concerns about the replicability of findings have been raised for many other disciplines including the medical sciences (e.g. Ioannidis 2005), cancer research (Errington et al 2014), the neurosciences (Wager, Lindquist, Nichols, Kober, & van Snellenberg 2009), economics (Camerer et al 2016), and genetics (Hewitt 2012)

Summary

Introduction

Our understanding of human speech and its cognitive underpinnings is increasingly shaped by experimental data. Mitleb 1981; Roettger, Winter, Grawunder, Kirby, & Grice 2014). These findings have led to important discussions about cognitive representations of speech sounds With a rapidly growing body of experimental evidence, it is of critical importance to evaluate and substantiate existing findings in the literature. This is important because the evidence provided by a single study is limited to the concrete research method and context. In recent coordinated efforts to replicate published results, the quantitative sciences have uncovered unexpectedly low replicability rates, a state of affairs that has been coined the ‘replication crisis’. Concerns about the replicability of findings have been raised for many other disciplines including the medical sciences (e.g. Ioannidis 2005), cancer research (Errington et al 2014), the neurosciences (Wager, Lindquist, Nichols, Kober, & van Snellenberg 2009), economics (Camerer et al 2016), and genetics (Hewitt 2012)

Methods

Results

Conclusion