Underestimation of type I errors in scientific analysis

Abstract

Statistical significance is the hallmark, the main thesis and the main argument to quantitatively verify or falsify scientific hypothesis in science. The correct use of statistic increases the reproducibility of the results and the is recently a widely wrong use of statistics and a reproducibility crisis in science. Today, most scientists can’t replicate the results of their peers and peer-review hinders them to reveal it. Next to an estimated 80% of general misconduct in the practices, above 90% of non-reproducible results and on top of this there is also a very high portion of generally false statistical assessment using statistical significance tests like the otherwise valuable t-test. The 5% confidence interval, a significance threshold of the t-test, is regularly and very frequently used without assuming the natural rate of false positives within the test system, referred to as the stochastic alpha or type I error. Due to the global need of a clear-cut clarification, this statistical research paper didactically shows for the scientific fields that are using the t-test, how to prevent the intrinsically occurring pitfalls of p-values and advises a random testing of alpha, beta, SPM and data characteristics to determine the right p-level and testing.