Abstract
This article explores whether the null hypothesis significance testing (NHST) framework provides a sufficient basis for the evaluation of statistical model assumptions. It is argued that while NHST-based tests can provide some degree of confirmation for the model assumption that is evaluated—formulated as the null hypothesis—these tests do not inform us of the degree of support that the data provide for the null hypothesis and to what extent the null hypothesis should be considered to be plausible after having taken the data into account. Addressing the prior plausibility of the model assumption is unavoidable if the goal is to determine how plausible it is that the model assumption holds. Without assessing the prior plausibility of the model assumptions, it remains fully uncertain whether the model of interest gives an adequate description of the data and thus whether it can be considered valid for the application at hand. Although addressing the prior plausibility is difficult, ignoring the prior plausibility is not an option if we want to claim that the inferences of our statistical model can be relied upon.
Highlights
One of the core objectives of the social sciences is to critically evaluate its theories on the basis of empirical observations
The basis of the null hypothesis significance testing (NHST) framework goes back to the statistical paradigm founded by Fisher in the 1930s (Fisher, 1930, 1955, 1956, 1960; Hacking, 1976; Gigerenzer, 1993), as well as the statistical paradigm founded by Neyman and Pearson in that same period (Neyman, 1937, 1957; Pearson, 1955; Neyman & Pearson, 1967; Hacking, 1976; Gigerenzer, 1993)
Starting from the 1950s, elements from both approaches were incorporated in the hybrid NHST framework as it exists today in the social and behavioral sciences (Gigerenzer & Murray, 1987; Gigerenzer, Swijtink, Porter, Daston, Beatty, & Kruger, 1989; Gigerenzer, 1993; Lehmann, 2006): While this framework proposes to evaluate a null hypothesis in contrast to an alternative hypothesis—in line with Neyman and Pearson—the focus lies on attempting to reject the null hypothesis, in line with Fisher’s methodology
Summary
One of the core objectives of the social sciences is to critically evaluate its theories on the basis of empirical observations. 1993, 2004), with alternatives such as using confidence intervals (Cumming & Finch, 2005; Fidler & Loftus, 2009; Cumming, 2014) or Bayesian approaches (Wagenmakers, 2007) being proposed as more informative and more appropriate tools for statistical inference These criticisms have been unanimous in their rejection of the use of (solely) NHST-based methods for evaluating substantive hypotheses and have had an important impact on both recommended and actual statistical practices, the uncritical and inappropriate use of NHST still appears to be quite common in practice. Standard methods for assessing whether statistical model assumptions hold often rely on NHST as well (e.g., Tabachnick & Fidell, 2001; Field, 2009), and are even employed when the substantive analyses do not use NHST (e.g., item response theory; Lord, 1980). Similar to the way in which Popper suggests to evaluate theories (Popper, 1959), null hypotheses are considered hypotheses that have not yet successfully been rejected, but should not be considered to be likely to be true
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
