Abstract
Statistical analyses are often conducted with α= .05. When multiple statistical tests are conducted, this procedure needs to be adjusted to compensate for the otherwise inflated Type I error. In some instances in tabletop gaming, sometimes it is desired to roll a 20-sided die (or 'd20') twice and take the greater outcome. Here I draw from probability theory and the case of a d20, where the probability of obtaining any specific outcome is (1)/ 20, to determine the probability of obtaining a specific outcome (Type-I error) at least once across repeated, independent statistical tests.
Highlights
In scientific research, it is important to consider the issue of conducting multiple statistical tests and the likelihood of spuriously obtaining a ‘significant’ effect
To make some events more likely, there are times when players roll a d20 ‘with advantage’, meaning that they roll the d20 twice and take the greater value1. (There are instances where a d20 is rolled ‘with disadvantage’, where the lesser value is taken, but here I will only focus on the former case.) This parallels the use of null-hypothesis significance testing (NHST) without any correction for multiple comparisons, as it is more likely to get a significant effect due to chance (i.e., Type-I error) if many tests are conducted without a correction for multiple comparisons
While it is widely understood that multiple comparisons need to be corrected for, many would underestimate the degree of inflation in Type-I error associated with additional, uncorrected statistical tests
Summary
It is important to consider the issue of conducting multiple statistical tests and the likelihood of spuriously obtaining a ‘significant’ effect. Within a null-hypothesis significance testing (NHST) framework, statistical tests are usually conducted with α = .05, i.e., the likelihood of falsely rejecting the null hypothesis as .05 This value coincides with the probability of obtaining a specific outcome on a 20-sided dice (or ‘d20’), as 1 = .05. I wondered how much the probability of obtaining a 20, on a d20, would increase due to multiple tests–i.e., obtaining at least one 20 across n die This approach assumes that each statistical test is wholly independent from each other, and is likely to over-estimate the effect related to conducting multiple statistical tests using variations in how the measures are calculated or the use of different, but correlated, measures. Following from the same approach of calculating the complementary event, the probability of obtaining not obtaining any two specific outcomes across multiple dice is. O n=1 n=2 n=3 n=4 n=5 1 1.0000 1.0000 1.0000 1.0000 1.0000 2 .9500 .9975 .9999 1.0000 1.0000 3 .9000 .9900 .9990 .9999 1.0000 4 .8500 .9775 .9966 .9995 .9999 5 .8000 .9600 .9920 .9984 .9997 6 .7500 .9375 .9844 .9961 .9990 7 .7000 .9100 .9730 .9919 .9976 8 .6500 .8775 .9571 .9850 .9947 9 .6000 .8400 .9360 .9744 .9898 .5500 .7975 .9089 .9590 .9815 .5000 .7500 .8750 .9375 .9688 .4500 .6975 .8336 .9085 .9497 .4000 .6400 .7840 .8704 .9222 .3500 .5775 .7254 .8215 .8840 .3000 .5100 .6570 .7599 .8319 .2500 .4375 .5781 .6836 .7627 .2000 .3600 .4880 .5904 .6723 .1500 .2775 .3859 .4780 .5563 .1000 .1900 .2710 .3439 .4095 .0500 .0975 .1426 .1855 .2262
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
