Investigating the performance of level-specific fit indices in multilevel confirmatory factor analysis with dichotomous indicators: A Monte Carlo study.

Abstract

We conducted a Monte Carlo study to examine the performance of level-specific χ2 test statistics and fit regarding their capacity to determine model fit at specific levels in multilevel confirmatory factor analysis with dichotomous indicators. Five design factors-numbers of groups (NG), group size (GS), intra-class correlation (ICC), thresholds of dichotomous indicators (THR), and factor loadings (FL)-were considered in this study. According to our simulation results, we recommend that practitioners should be aware that the performance of between-level-specific (b-l-s) χ2 and fit indices was mainly influenced by ICC and FL, followed by NG. At the same time, THR could slightly weigh in the performance of b-l-s fit indices in some conditions. Both b-l-s χ2 and fit indices were more promising indicators to correctly indicate model fit when ICC or FL increased. A small to medium NG (50-100) might be sufficient for b-l-s χ2 and fit indices only if both ICC and factor loadings were high, while in remaining conditions, an NG of 200 was needed. Moreover, practitioners could use within-level-specific (w-l-s) χ2 and fit indices (except for RMSEAW) along with traditional cut-off values to evaluate within-level models comprising dichotomous indicators. W-l-s χ2 and fit indices were more promising to determine model fit when FL increased. THR had a slight impact and could weigh in the performance of [Formula: see text], RMSEAW, CFIW, and TLIW. Unfortunately, RMSEAW was heavily affected by FL and THR and could determine model fit only when FL was high and THR was symmetric.