Model Selection Strategies and the Use of Association Models to Detect Group Differences

Abstract

Listen

Translate article

This article discusses the use of association models to detect group differences and the choice of model selection criteria under different conditions. The performances of several commonly used model selection criteria in log-linear modeling are examined using Monte Carlo simulations. The results suggest that no single criterion can play the role of panacea in model selection. L2/ df, normed fit index (NFI) (or 1 - L12/L02), and Akaike's information criterion (AIC) are systematically biased toward models incorporating group differences. The log-likelihood ratio test (LRT), in conjunction with the nested chi-square difference test, is useful for samples with moderate sizes. Bayesian information criterion (BIC) is most reliable among the measures tested. It consistently provides correct information about group differences in association, especially when the sample size is large. Although BIC may give equivocal results under certain conditions, the ambiguities can be mitigated by an inclusion of competing models and a cautious interpretation of any small improvement in BIC.