Classical and Bayesian Uncertainty Intervals for the Reliability of Multidimensional Scales

Abstract

The reliability of a multidimensional test instrument is commonly estimated using coefficients ωt (total) and ωh (hierarchical). However, point estimates for the coefficients are rarely accompanied by uncertainty estimates. In this study, we compare bootstrap and normal-theory confidence intervals. In addition, we develop Bayesian versions of coefficients ωt and ωh by sampling from a second-order factor model. Results from a comprehensive simulation study show that the studied confidence intervals performed well when the sample size was sufficiently large (). The Bayesian estimates performed well across most studied conditions. When the sample size was small and the reliability low, only the bias-corrected and accelerated bootstrap confidence interval approached a satisfactory coverage among all intervals. This study guides on ωt and ωh confidence intervals and introduces ωt and ωh credible intervals that are easy to use and come with the benefits of Bayesian parameter estimation.