Paradigm Shift in Social Science Research: A Significance Testing and Effect Size Estimation Rapprochement?Paradigm Shift in Social Science Research: A Significance Testing and Effect Size Estimation Rapprochement?

Abstract

During the past 40 years, the vast majority of students trained in the social sciences have been taught the of null- hypothesis significance-testing (NHST). Social science researchers following the received review have approached their work with a heavy reliance on statistical tests, which have been strongly identified with probability or testing. The process of rejecting or retaining the null hypothesis has dictated this research, leaving the hope of accepting an alternative, explanatory hypothesis to the mercies of somewhat arbitrary designations of chance probabilities (e.g., p <.05). However, over the years a growing number of researchers have argued against the received view of statistical inference (e.g., Carver, 1978; Cohen, 1994; Hunter, 1997; Loftus, 1996). In fact, the American Psychological Association's (APA) Board of Scientific Affairs has recently considered prohibiting the reporting of significance tests in APA journals (Shrout, 1997). However, there are strong supporters of NHST, who state that critiques of NHST tend to be metatheoretical assertions that do not take into account the fact that rejection of H0 is only one factor in the practice of testing a theoretical hypothesis (Abelson, 1997; Chow, 1996; Cortina, & Dunlap, 1997; Frick, 1996). Rex B. Kline, in his recent book Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research, takes issue with the received view and foresees a future in which the NHST in social science research will diminish. Kline is calling for a paradigm shift, in which social science will more closely reflect the hard sciences. Kline concisely explicates his vision: Most studies in the future will not use statistical tests as the primary decision criterion, and those that do will concern only very specific problems for which variations of NHST may be appropriate, such as equivalence testing or inferential confidence intervals. It is also envisioned that the social sciences will become more like the natural sciences. That is, one will report the directions and magnitudes of one's effects; determine whether they