An effect size index for comparing two independent alpha coefficients

Abstract

Since Cronbach proposed the alpha coefficient in 1951, researchers have contributed to the derivation of its sampling distribution and the testing of related statistical hypotheses. Yet, there has been no research on effect size index relevant to coefficient alpha to our knowledge. Considering the importance of effect size in understanding quantitative research findings, we therefore developed an effect size index Delta for the comparison of two independent alphas with equal test length based on the asymptotic distribution of (1/2)ln(1 - alphahat) under the assumptions of normality and compound symmetry. Simulations indicated that the index was applicable when the sample size was at least 100. The robustness of the derived index when the required assumptions were violated was also explored. It is suggested that the index should be applicable in most cases of unequal test lengths and could be extended to non-normally distributed component scores. Moreover, a small simulation was conducted to explore the behaviour of Delta with correlated errors, a frequently studied situation violating the assumption of compound symmetry. The proposed index was found to be robust unless a large number of highly correlated errors were present in the data.