Replication Crisis In Psychology Research Articles

On reproducibility and replicability: Arguing for open science practices and methodological improvements at the South African Journal of Industrial Psychology

Problematisation: In recent years, psychology has been going through a crisis of sorts. Research methods and practices have come under increased scrutiny, with many issues identified as negatively contributing to low replicability and reproducibility of psychological research.Implications: As a consequence, researchers are increasingly called upon to overhaul and improve their research process. Various stakeholders within the scientific community are arguing for more openness and rigor within industrial and organisational (I-O) psychological research. A lack of transparency and openness further fuels criticisms as to the credibility and trustworthiness of I-O psychology which negatively affects the evidence-based practices which it supports. Furthermore, traditional gate-keepers such as grant agencies, professional societies and journals, are adapting their policies, reflecting an effort to curtail these trends.Purpose: The purpose of this opinion paper is, therefore, to stimulate an open dialogue with the South African Journal of Industrial Psychology (SAJIP) contributing authors, its editorial board and readership about the challenges associated with the replication crisis in psychology. Furthermore, it attempts to discuss how the identified issues affect I-O psychology and how these could be managed through open science practices and other structural improvements within the SAJIP.Recommendations: We enumerate several easily implementable open science practices, methodological improvements and editorial policy enhancements to enhance credibility and transparency within the SAJIP. Relying on these, we recommend changes to the current practices that can be taken up by researchers and the SAJIP to improve reproducibility and replicability in I-O psychological science.

Can hypnotic suggestibility be measured online?

Hypnosis and hypnotic suggestions are gradually gaining popularity within the consciousness community as established tools for the experimental manipulation of illusions of involuntariness, hallucinations and delusions. However, hypnosis is still far from being a widespread instrument; a crucial hindrance to taking it up is the amount of time needed to invest in identifying people high and low in responsiveness to suggestion. In this study, we introduced an online assessment of hypnotic response and estimated the extent to which the scores and psychometric properties of an online screening differ from an offline one. We propose that the online screening of hypnotic response is viable as it reduces the level of responsiveness only by a slight extent. The application of online screening may prompt researchers to run large-scale studies with more heterogeneous samples, which would help researchers to overcome some of the issues underlying the current replication crisis in psychology.

Redefine or justify? Comments on the alpha debate.

Benjamin et al. (Nature Human Behaviour 2, 6-10, 2017) proposed improving the reproducibility of findings in psychological research by lowering the alpha level of our conventional null hypothesis significance tests from .05 to .005, because findings with p-values close to .05 represent insufficient empirical evidence. They argued that findings with a p-value between 0.005 and 0.05 should still be published, but not called "significant" anymore. This proposal was criticized and rejected in a response by Lakens et al. (Nature Human Behavior 2, 168-171, 2018), who argued that instead of lowering the traditional alpha threshold to .005, we should stop using the term "statistically significant," and require researchers to determine and justify their alpha levels before they collect data. In this contribution, I argue that the arguments presented by Lakens et al. against the proposal by Benjamin et al. are not convincing. Thus, given that it is highly unlikely that our field will abandon the NHST paradigm any time soon, lowering our alpha level to .005 is at this moment the best way to combat the replication crisis in psychology.

The misattribution of emotions and the error-related negativity: A registered report

A growing body of work in social and affective neuroscience suggests that emotion plays an instrumental role in error monitoring processes, rather than only a moderating one. High-powered replications of studies that support this idea, however, are lacking. Here, we attempted a preregistered replication of our own study that had provided evidence for the functional role of emotions in error monitoring: that a neural signal of error monitoring—the error-related negativity—is reduced when participants undergo a misattribution of arousal procedure (Inzlicht & Al-Khindi, 2012). Like a previous replication attempt (Rodilla et al, 2016), our misattribution procedure failed to reduce the amplitude of the ERN. However, it also failed its manipulation check to reduce state anxiety, limiting the conclusions we can draw. Nonetheless, these findings are consistent with the view that our original study may have been a false positive. We discuss these findings in the context of the replication crisis in psychology and of work on the emotional properties of the ERN.

Is There A Replicability Crisis In Psychology? New Study Says: Its Complicated

Is There A Replicability Crisis In Psychology? New Study Says: Its Complicated

A Review of Rozeboom’s Ideas with an Analysis of Issues in Null Hypothesis Significance Testing

Reexamining William Rozeboom’s recommendations for the future direction of disciplines such as psychology and philosophy is imminent due to the pressing issues in null hypothesis significance testing (NHST). An overreliance on NHST forms the basis of the replication crisis in psychology. Likewise, the discipline’s stringent guidelines on significance levels convey a pressure to publish, which is also significantly contributing to the replication crisis. As researchers’ careers are staked on the extent to which they publish, reassessing the fundamental issues with NHST within the context of Rozeboom’s ideas is paramount. In this brief review, I focus specifically on Rozeboom’s critiques of NHST and on a proposed alternative for evaluating the depth of findings within the framework of theoretical psychology. Thereafter, I compare his foreboding concerns with modern issues facing the validity of psychology and posit Rozeboom prophesized the failings of the discipline years ahead of his time. Keywords: Theoretical Psychology; NHST; William Rozeboom; P-Hacking; Explanatory Induction

Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future

Over the last few years, psychology researchers have become increasingly preoccupied with the question of whether findings from psychological studies are generally replicable. The debates have originated from some unfortunate events of scientific misconduct in the field, and they have reached a climax with the recent discovery of a relatively weak rate of replicability of published literature, leading to the so-called replicability crisis in psychology. The present paper is concerned with examining the issue of replicability in the field of social psychology. We begin by drawing a state of the art of the crisis in this field. We then highlight some possible causes for the crisis, discussing topics of statistical power, questionable research practices, publication standards, and hidden auxiliary assumptions of context-dependency of social psychological theories. Finally, we argue that given the absence of absolute falsification in science, social psychology could greatly benefit from adopting McGuire’s perspectivist approach to knowledge construction.

A tragedy of the (academic) commons: interpreting the replication crisis in psychology as a social dilemma for early-career researchers.

Several proposals for addressing the “replication crisis” in social psychology have been advanced in the recent literature. In this paper, we argue that the “crisis” be interpreted as a disciplinary social dilemma, with the problem facing early-career researchers being especially acute. To resolve this collective action problem, we offer a structural solution: as a condition of receiving their Ph.D. from any accredited institution, graduate students in psychology should be required to conduct, write up, and submit for publication a high-quality replication attempt of at least one key finding from the literature, focusing on the area of their doctoral research. We consider strengths, weaknesses, and implementation challenges associated with this proposal, and call on our colleagues to offer critical response.

Change starts with journal editors: In response to Makel (2014).

The editors of the Journal of Advanced Academics comment on Makel (2014). The replicability crisis in psychology is summarized in terms of three focal issues: the “file drawer” problem, lack of replication studies, and the null hypothesis significance testing paradigm. The authors argue that journal editors are uniquely positioned to address all three of these problems via the adoption of new policies for review and publication.