A comparison of model selection indices for nested latent class models

Abstract

Latent Class analysis has been used to study the hierarchical relationship among sets of categorical variables. Researchers routinely use chi-squared statistics as model-selection criteria. Due to the limitation of chi-squared statistics, it is desirable to develop other model selection indices. In this study, we compared the performance of chi-squared statistics with three information criteria, Akaike's AIC, Schwarz's BIC and Bozdogan's CAIC. The factors actually manipulated in this study were types of latent class model and conditional response probabilities including intrusion and omission error rates for certain models. In terms of the performance of the six models, BIC and CAIC were most accurate for the Independence, Proctor and Goodman models, AIC was most accurate for the Extended Intrusion-Omission Error model, while chi-squared difference statistics showed their best performance in some settings of the Extended Proctor model. As for the Intrusion-Omission Error model and some situations in the Extend Proctor model, the performance of the five indices depended on sample sizes and error rates. AIC was preferred in Intrusion-Omission Error model when sample size was small and BIC and CAIC were preferred when sample size was medium. In Proctor model, AIC was more accurate than chi-squared difference statistics when error rate was small. A similar effect occurred in Hierarchy I in Extended Proctor model. With a small error rate, AIC was superior to chi-squared difference statistics but worse with a large error rate. None of the five criteria performed well in Extended Proctor and Extended Intrusion-Omission Error models. In conclusion, the information criteria had superior performance than both chi-squared statistics. Among the information criteria, BIC and CAIC were more appropriate than AIC for the Independence and Proctor models while AIC was more accurate for Extended Proctor model.