Null Hypothesis Significance Testing: Effect Size Matters

Abstract

A statistically significant outcome only indicates that it is likely that there is a relationship between variables. It does not describe the extent (strength) of that relationship. In this article, emphasis is placed on the importance of assessing the strength of the relationship between the independent and dependent variables using effect size indices. Effect size indices for the d family and r family are introduced, along with formulas for their direct and indirect computation for both the t test and chi-square test. A subset of the variables and concepts examined in the Whittaker and Manfredo study are reported here to demonstrate why an effect size index should be computed. Statistical analyses (either t test or chi-square test) were performed on the original sample of 796 and three smaller sample sizes (398, 200, and 100) randomly selected from the initial sample. Effect size indices were computed for each statistical test. The results indicated that the size of the sample directly affects the t or chi-square statistic and p , but the effect size was independent of the sample size. Effect sizes should, therefore, accompany reported p values.