Abstract
Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.
Highlights
Over the last several years, impressive efforts have been made to estimate the reproducibility of various empirical literatures
Studies came from several different sub-areas of experimental philosophy: 8 from Action Theory, 1 from Aesthetics, 4 from Causation, 5 from Epistemology, 8 from Free Will, 8 from Moral Psychology, 1 from Philosophy of Language, 2 from Philosophy of Mind, 3 uncategorized
Our goal was to reach a rough estimate of the reproducibility of experimental philosophy studies
Summary
Over the last several years, impressive efforts have been made to estimate the reproducibility of various empirical literatures. Notable examples include the Open Science Collaboration’s (OSC) attempt to estimate the reproducibility of psychological science (Open Science Collaboration 2015), the Reproducibility Project’s. 9, 1202 Geneva, Switzerland analogous initiative for cancer biology (Nosek and Errington 2017), meta-scientist John Ioannidis’s modeling efforts in biomedicine and beyond (e.g., Ioannidis 2005) and a 2015 estimate produced by the Board of Governors of the Federal Reserve System for research in economics (Chang and Li 2015). Health Care, University of Ghent, Ghent, Belgium Institute of Philosophy, University of Bern, Bern, Switzerland Queen’s University Belfast, Belfast, UK debate about what the optimal replication rate should be for a given field in light of trade-offs between, e.g., innovation and confirmation (Gilbert et al 2016; Makel & Plucker, 2014), many scientists regard the estimates that have been generated—less than 50% in each of the above cases—as worryingly low. Promising ideas for improvement—including the recent push toward norms of pre-registration—are gaining traction among leading scientists (Chambers and Munafò 2013; Munafò et al 2017; Nosek et al in press; but see Lash and Vandenbroucke 2012; Scott 2013)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
