Abstract
Recent replication crises in psychology and other fields have led to intense reflection about the validity of common research practices. Much of this reflection has focussed on reporting standards, and how they may be related to the questionable research practices that could underlie a high proportion of irreproducible findings in the published record. As a developing field, it is particularly important for Experimental Philosophy to avoid some of the pitfalls that have beset other disciplines. To this end, here we provide a detailed, comprehensive assessment of current reporting practices in Experimental Philosophy. We focus on the quality of statistical reporting and the disclosure of information about study methodology. We assess all the articles using quantitative methods (n = 134) that were published over the years 2013–2016 in 29 leading philosophy journals. We find that null hypothesis significance testing is the prevalent statistical practice in Experimental Philosophy, although relying solely on this approach has been criticised in the psychological literature. To augment this approach, various additional measures have become commonplace in other fields, but we find that Experimental Philosophy has adopted these only partially: 53% of the papers report an effect size, 28% confidence intervals, 1% examined prospective statistical power and 5% report observed statistical power. Importantly, we find no direct relation between an article’s reporting quality and its impact (numbers of citations). We conclude with recommendations for authors, reviewers and editors in Experimental Philosophy, to facilitate making research statistically-transparent and reproducible.
Highlights
Philosophers have recently started to adopt empirical methods to address research questions of philosophical relevance
We considered the affiliations listed on the published article and in cases in which authors were affiliated to an interdisciplinary center or no clear information about their institution was available, the subject of their PhD was considered to determine the relevant subject area
Unable to estimate the number of philosopher authors who had obtained relevant training, but we note again that statistics and experimental design
Summary
Philosophers have recently started to adopt empirical methods to address research questions of philosophical relevance This practice is often referred to as Experimental Philosophy (Knobe and Nichols 2008; Alexander 2012; Knobe et al 2012; Machery and O’Neill 2014; Sytsma and Buckwalter 2016), it incorporates both experimental and correlational studies. It seems that what best characterizes this recent trend is an attempt to employ quantitative methods to make progress in philosophy (Knobe 2015). Experimental philosophers should be concerned with ongoing discussions, in several empirical fields, about whether common scientific practices in design, analysis, and reporting ought to be revised (Begley and Ellis 2012; Ioannidis 2005; Miguel et al 2014; Simmons et al 2011)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
