Abstract
In educational settings, researchers are likely to encounter multilevel data with cross-classified structure. However, due to the lack of familiarity and limitations of statistical software for cross-classified modeling, most researchers adopt less optimal approaches to analyze cross-classified multilevel data in testing measurement invariance. We conducted two Monte Carlo studies to investigate the performances of testing measurement invariance with cross-classified multilevel data when the noninvarinace is at the between-level: (a) the impact of ignoring crossed factor using conventional multilevel confirmatory factor analysis (MCFA) which assumes hierarchical multilevel data in testing measurement invariance and (b) the adequacy of the cross-classified multiple indicators multiple causes (MIMIC) models with cross-classified data. We considered two design factors, intraclass correlation (ICC) and magnitude of non-invariance. Generally, MCFA demonstrated very low statistical power to detect non-invariance. The low power was plausibly related to the underestimated factor loading differences and the underestimated ICC due to the redistribution of the variance component from the ignored crossed factor. The results demonstrated possible incorrect statistical inferences with conventional MCFA analyses that assume multilevel data as hierarchical structure for testing measurement invariance with cross-classified data (non-hierarchical structure). On the contrary, the cross-classified MIMIC model demonstrated acceptable performance with cross-classified data.
Highlights
In educational and other social science research, multilevel data are commonly encountered
Goodness-of-fit indices Considering the sensitivity of the χ2 test statistic to sample size, we have examined the performance of the following difference ( ) of the goodness-of-fit indices in comparing the two competing invariance models: (a) IC (i.e., AIC and BIC; (b) SRMR between and within; (c)
We examined cross-classified multiple indicators multiple causes (MIMIC) model to fit a correct model in testing measurement invariance with cross-classified data using Mplus
Summary
In educational and other social science research, multilevel data are commonly encountered. Hierarchical linear models (HLMs) assume that in multilevel data, the levels are strictly nested or hierarchical, which means that a lower-level observation belongs to one and only one higher-level cluster. In education settings, a student belongs to only a particular classroom while that classroom belongs to only a particular school. Measurement Invariance in Cross-Classified Data a strictly nested or hierarchical structure, especially in education settings. Schools and neighborhoods are cross-classified with each other at the same level. This type of non-hierarchical multilevel data is called cross-classified multilevel data. Some researchers did not fully consider the cross-classified structure of the data by ignoring one of the cross-classified factors in the data and used HLMs in their analyses (e.g., George and Thomas, 2000; Ma and Ma, 2004)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
