Abstract
Four data sets from studies included in the Reproducibility Project were re-analyzed to demonstrate a number of flawed research practices (i.e., “bad habits”) of modern psychology. Three of the four studies were successfully replicated, but re-analysis showed that in one study most of the participants responded in a manner inconsistent with the researchers’ theoretical model. In the second study, the replicated effect was shown to be an experimental confound, and in the third study the replicated statistical effect was shown to be entirely trivial. The fourth study was an unsuccessful replication, yet re-analysis of the data showed that questioning the common assumptions of modern psychological measurement can lead to novel techniques of data analysis and potentially interesting findings missed by traditional methods of analysis. Considered together, these new analyses show that while it is true replication is a key feature of science, causal inference, modeling, and measurement are equally important and perhaps more fundamental to obtaining truly scientific knowledge of the natural world. It would therefore be prudent for psychologists to confront the limitations and flaws in their current analytical methods and research practices.
Highlights
Written in the 13th Century, Questions V and VI of St
We present four additional, perfidious habits that may be preventing modern psychologists from developing truly scientific knowledge of human experience
A simple type of induction found within the vast majority of published psychological studies is the inference from an observed sample statistic to an unobserved population parameter
Summary
Written in the 13th Century, Questions V and VI of St. Thomas Aquinas’ Expositio super Librum. A simple type of induction (reasoning from the specific to the general) found within the vast majority of published psychological studies is the inference from an observed sample statistic to an unobserved population parameter This inference often occurs within the context of a Null Hypothesis Significance Test (NHST), in which the researcher judges a computed p-value to reach or exceed some low threshold (e.g., p ≤ 0.05). Schmidt and Besner argued that the best explanation for this phenomenon is a contingency threshold that is altered when frequencies of congruent words increase The details of this mechanism need not be repeated here, but it is important to emphasize that the mechanism is meant to provide a causal account of the observed data. Inference to best explanation sought by the study authors is not generally supported by the data from the proposed change in threshold not consistent a majority the replication study. This provides conclusionevidence was missed inference which hasprovides no necessary bearing a proposed operating the level of because evidence for anon inference whichcausal has nomechanism necessary bearing on a at proposed causal mechanism operating the individuals in a given study.at the level of the individuals in a given study
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
