Abstract
It is well recognised that low statistical power increases the probability of type II error, that is it reduces the probability of detecting a difference between groups, where a difference exists. Paradoxically, low statistical power also increases the likelihood that a statistically significant finding is actually falsely positive (for a given p-value). Hence, ethical concerns regarding studies with low statistical power should include the increased risk of type I error in such studies reporting statistically significant effects. This paper illustrates the effect of low statistical power by comparing hypothesis testing with diagnostic test evaluation using concepts familiar to clinicians, such as positive and negative predicative values. We also note that, where there is a high probability that the null hypothesis is true, statistically significant findings are even more likely to be falsely positive.
Highlights
Significance tests cannot determine whether a null hypothesis is true or not, they can only indicate the probability of observing the data collected assuming the null hypothesis is true
We illustrate how interpretation of hypothesis tests may vary with statistical power and the probability of the null hypothesis being true through comparison with methods of diagnostic test evaluation
There are similarities between this representation of Hypothesis testing (HT) with that of diagnostic test evaluation (Table 2) [14] and revising the common representation of HT illustrated in Table 1 to that commonly used for diagnostic tests further highlights the similarity in logic between these methods (Table 3)
Summary
Significance tests cannot determine whether a null hypothesis is true or not, they can only indicate the probability of observing the data collected assuming the null hypothesis is true. It is well recognised that low statistical power increases the probability of a type II error. If two experiments are conducted and each of the null hypotheses are rejected (with the statistical tests used having the same p-value), it is sometimes assumed that the type I error rates are the same in each case. If one of the studies had substantially lower statistical power, this study has an increased probability of incorrectly rejecting the null hypothesis. We illustrate how interpretation of hypothesis tests may vary with statistical power and the probability of the null hypothesis being true through comparison with methods of diagnostic test evaluation
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
