Integrating Bifactor Models into a Generalizability Theory Based Structural Equation Modeling Framework

Abstract

Although generalizability theory (GT) designs have traditionally been analyzed within an ANOVA framework, identical results can be obtained with structural equation models (SEMs) but extended to represent multiple sources of both systematic and measurement error variance, include estimation methods less likely to produce negative variance components, and correct for score coarseness. In research reported here, we integrated principles of bifactor modeling into a GT-SEM framework to partition systematic variance for subscale and composite scores into general and group factor effects and measurement error into multiple components. GT-bifactor modeling allowed for partitioning of variance at different levels of aggregation, clearer definitions of constructs within sampled domains, extended indices of score consistency, insights into composite and subscale score viability, and markers for best enhancing score consistency. Results for domain and facet scores from the recently updated form of the Big Five Inventory highlighted the importance of taking all sources of measurement error into account and the diagnostic benefits of GT-bifactor designs over conventional univariate and multivariate GT designs. Findings further revealed that corrections for scale coarseness noticeably enhanced overall score consistency by reducing specific-factor and random-response measurement error. We provide code in R for applying all illustrated designs in a detailed online supplement.