The accuracy of reliability coefficients: A reanalysis of existing simulations.

Abstract

Controversy over which reliability estimators should be used persists due to a lack of knowledge about their accuracy. Simulation is an effective tool to obtain an answer, but existing simulation studies yield contradictory results regarding which reliability estimators are the best. The causes of these inconsistent conclusions have yet to be discussed. This study reanalyzes existing studies to understand these contradictions. The most important reason is that previous studies consider only a few reliability estimators. This study examines approximately 30 reliability estimators and finds that there is no single, most accurate reliability estimator across all data types. Instead, several reliability estimators are accurate to comparable levels for unidimensional data (congeneric reliability, Guttman's lambda2, and ten Berge-Zegers's mu). Likewise, multiple reliability estimators perform similarly for multidimensional data (multidimensional parallel reliability, correlated factors reliability, and second-order factor reliability). Whereas many recent studies support factor analysis (FA) reliability estimators, this study shows that not all FA reliability estimators are accurate and that some cause severe overestimation. (PsycInfo Database Record (c) 2024 APA, all rights reserved).