New Guidelines for Null Hypothesis Significance Testing in Hypothetico-Deductive IS Research

Abstract

The objective of this research perspectives article is to promote policy change among journals, scholars, and students with a vested interest in hypothetico-deductive information systems (IS) research. We are concerned about the design, analysis, reporting, and reviewing of quantitative IS studies that draw on null hypothesis significance testing (NHST). We observe that although debates about misinterpretations, abuse, and issues with NHST have persisted for about half a century, they remain largely absent in IS. We find this to be an untenable position for a discipline with a proud quantitative tradition. We discuss traditional and emergent threats associated with the application of NHST and examine how they manifest in recent IS scholarship. To encourage the development of new standards for NHST in hypothetico-deductive IS research, we develop a balanced account of possible actions that are implementable in the short-term or long-term and that incentivize or penalize specific practices. To promote an immediate push for change, we also develop two sets of guidelines that IS scholars can adopt immediately.