Item Fit Indices Research Articles

A Psychometric Assessment of the Newest Vital Sign among Youth in Guatemala City

Background and Objective: Psychometrically sound health literacy assessments are widely available in the US for both adults and youth. However, among international youth populations there is a dearth of reliable instruments. This study was undertaken in order to assess the psychometric viability of using the Spanish version of the Newest Vital Sign (NVS; previously validated in the US) among a sample of youth living in Guatemala City. Methods: Analysis is based on 230 youth who were attending various schools within Guatemala City, Guatemala. Rasch modeling was used to estimate item difficulty and discrimination parameters, item fit indices, item-total correlations, and differential item functioning between male and female youth. Results: The majority of the items on the NVS exhibit acceptable properties. The last item demonstrated statistically significant DIF of a practically important magnitude (DIF > |0.43|) suggesting that females are more likely than males to answer the item correctly after controlling for the ‘amount’ of health literacy that the students possess. Conclusion: Those using the NVS among international Spanish-speaking youth populations should be aware that the NVS appears to be appropriate to use in Latin America. Females may tend to have slightly higher scores than males. Additional psychometric testing of the instrument among similar cross-national samples of youth is needed.

Model Data Fit Comparison between DINA and G-DINA in Cognitive Diagnostic Models

In this study, item and model data fit indices, calculated by DINA and G-DINA Models using the same sample and Q matrix, are analyzed. Fit indices for these two models from Cognitive Diagnostic Models are analyzed using 2LL, AIC and BIC statistics. Item fit indices are analyzed using residual correlations and probabilities. Analysis results showed G-DINA model had better fit results than DINA model. DINA model could give rather better results to estimate student profile in tests where higher level and progressive behaviors are used together. On the other hand, G-DINA model weights required attributes for an item when estimating student profile. Therefore in items requiring more than one attributes, contributions of attributes to probability that a student answers the item correctly are not equal. This provides an important advantage to testers to evaluate multiple choice items in assessing complex and prerequisite forming patterns.

A Comparison of Item Fit Statistics for Mixed IRT Models

In this study we examined procedures for assessing model‐data fit of item response theory (IRT) models for mixed format data. The model fit indices used in this study include PARSCALE's G2, Orlando and Thissen's S − X2 and S − G2, and Stone's χ2* and G2*. To investigate the relative performance of the fit statistics at the item level, we conducted two simulation studies: Type I error and power studies. We evaluated the performance of the item fit indices for various conditions of test length, sample size, and IRT models. Among the competing measures, the summed score‐based indices S − X2 and S − G2 were found to be the sensible and efficient choice for assessing model fit for mixed format data. These indices performed well, particularly with short tests. The pseudo‐observed score indices, χ2* and G2*, showed inflated Type I error rates in some simulation conditions. Consistent with the findings of current literature, the PARSCALE's G2 index was rarely useful, although it provided reasonable results for long tests.

Performance of the generalized S-X2 item fit index for the graded response model

The utility of Orlando and Thissen’s (2000, 2003) S-X2 fit index was extended to the model-fit analysis of the graded response model (GRM). The performance of a modified S-X2 in assessing item-fit of the GRM was investigated in light of empirical Type I error rates and power with a simulation study having various conditions typically encountered in applied testing situations. The results show that the Type I error rates were controlled adequately around the nominal alpha by S-X2. The power of the S-X2 statistic was much lower when the source of misfit was multidimensionality than when it was due to discrepancy from the true GRM curves. Once the data size increased sufficiently, however, appropriate power was obtained regardless of the source of the item-misfit. In summary, the generalized S-X2 appears to be a promising index for investigating item fit for polytomous items in educational and psychological assessments.

Item-fit evaluation in biased tests: a study under Rasch model

In this paper, the power rates and distributional properties of the Outfit, Infit, Lz, ECI2z and ECI4z statistics when they are used in tests with biased or differential item functioning (DIF) were explored. In this study, different conditions of sample size, sample size ratio focal and reference group, impact between groups, DIF effect size, and percentage of DIF items were manipulated. In addition, examinee responses were generated to simulate uniform DIF. Results suggest that item fit statistics generally detected medium percents of DIF in large samples (1000/500 or 1000/1000) only when DIF effect size was relatively high and when the mean of focal and reference group was different. Moreover, when groups had equal mean, low correct identification rates were found in the five item-fit indices. In general, the results showed adequate control of false positive rates. These findings lead to the conclusion that all indices used in this study are partially adequate fit measures for detecting biased items, mainly when impact between groups is present and sample size is large.

An Examination of Item Response Theory Item Fit Indices for the Graded Response Model

The current study examined the Type I error rates and power of several item response theory (IRT) item fit indices used in conjunction with the graded response model (GRM). Specifically, S — χ2, χ2*, and adjusted χ 2 degrees of freedom ratios (χ2/dfs) were examined. Model misfit was introduced by manipulating item parameters and by using a different IRT model to generate item data. Results indicated lower than expected Type I error rates for S — χ2 and χ 2*. Adjusted χ2/dfs resulted in large Type I error rates when used with cross validation and very low Type I error rates when used without cross validation. χ2* and adjusted χ 2/dfs without cross validation were the most powerful overall.

Modified Likelihood-Based Item Fit Statistics for the Generalized Graded Unfolding Model

Orlando and Thissen (2000) developed an item fit statistic for binary item response theory (IRT) models known as S - X2. This article generalizes their statistic to polytomous unfolding models. Four alternative formulations of S - X2 are developed for the generalized graded unfolding model (GGUM). The GGUM is a unidimensional IRT model for unfolding polytomous responses. It yields single-peaked, nonmonotonic item characteristic curves that predict a higher item score to the extent that an individual is located close to an item on the underlying latent continuum. A simulation was performed to examine the characteristics of these new item fit indices under the GGUM, as well as a traditional likelihood ratio χ2 test ( G2). All variants of S - X2 exhibited reasonable Type I error rates, but that for G2 was more erratic. The new indices exhibited variable power to detect misfit. Two new item fit tests are recommended for use based on simulation results.

Further Investigation of the Performance of S - X2: An Item Fit Index for Use With Dichotomous Item Response Theory Models

This study presents new findings on the utility of S - X2 as an item fit index for dichotomous item response theory models. Results are based on a simulation study in which item responses were generated and calibrated for 100 tests under each of 27 conditions. The item fit indices S - X2 and Q1 - X2 were calculated for each item. ROC curves were constructed based on the hit and false alarm rates of the two indices. Examination of these curves indicated that in general, the performance of S - X2 improved with test length and sample size. The performance of S - X2 was superior to that of Q1 - X2 under most but not all conditions. Results from this study imply that S - X2 may be a useful tool in detecting the misfit of one item contained in an otherwise well-fitted test, lending additional support to the utility of the index for use with dichotomous item response theory models. Index Terms: item response theory, S - X2, Q1 - X, model = data fit, item fit index.

A Conditional Item-Fit Index for Rasch Models

A new item-fit index is proposed that is both a descriptive measure of deviance of single items and an index for statistical inference. This index is based on the assumptions of the dichotomous and polytomous Rasch models for items with ordered categories and, in particular, is a standardization of the conditional likelihood of the item pattern that does not depend on the item parameters. This approach is compared with other methods for determining item fit. In contrast to many other item-fit indexes, this index is not based on response-score residuals. Results of a simulation study illustrating the performance of the index are provided. An asymptotically normally distributed Z statistic is derived and an empirical example demonstrates the sensitivity of the index with respect to item and person heterogeneity. Index terms: appropriateness measurement, item discrimination, item fit, partial credit model, Rasch model.

Relationships among IRT item discrimination and item fit indices in criterion-referenced language testing

This study investigates relationships among the IRT one-parameter fit statistics, the two-parameter slope parameter and traditional biserial correla tions in terms of the role these indices play in criterion-referenced language test construction. It discusses the assumptions of the two models and how these assumptions can affect criterion-referenced test construction and interpreta tion. The study then specifically examines how the indices interrelate as indices of item discrimination. Examinees in Mexico, Saudi Arabia and Japan were administered one of two forms of a functional test (Form A n = 430, k = 94: Form B n = 400, k = 95). The data were analysed using the two IRT models and the results were compared. The results indicate strong relationships among biserial correlation, two-parameter slope, and one-parameter infit and outfit. These results indicate the need to employ the two-parameter model when con ditions allow, and to take item discrimination and item difficulty indices into account when conditions do not. Further implications for interpreting the strong relationships between the indices are discussed.

A Comparison of Item- and Person-Fit Methods of Assessing Model-Data Fit in IRT

Many item-fit statistics have been proposed for assessing whether the responses to test items ag gregated across examinees conform to IRT test models. Conversely, person-fit statistics have been proposed for assessing whether an examinee's re sponses aggregated across items are congruent with a specified IRT model. Statistical procedures to as sess item fit have differed from those to assess per son fit. This research compared a χ 2 item-fit index with a likelihood-based person-fit index. Eight 0,1 data matrices were simulated under the three- parameter logistic test model. Both the likelihood- based and χ2 fit statistics were then computed for examinees and items, and Type I and Type II error rates were analyzed. With data simulated to fit the IRT model, the χ 2 test overidentified examinees and items as being misfitting, while the likelihood- based fit index held closer to the specified α levels. The two fit indices gave consistent (mis)fit-to- model results in 94 and 97 percent of cases for items and examinees, respectively, across simula tions. Under simulated conditions of data misfit, the χ2 statistic detected misfit at a higher rate than the likelihood-based statistic, indicating that the χ2 statistic was slightly more sensitive to response pat tern aberrancy. However, other considerations led to a recommendation for employing the likelihood- based index in applied fit analyses to evaluate both examinee and item model-data (mis)fit.