Dichotomous Item Response Models Research Articles

The Effect of Sample Size on Type I error of the S-X2 Item Fit Index

본 연구에서는 카이제곱 분포를 따르는 S-χ2문항 적합도 지수가 제1종 오류에 있어서 표본크기의 영향을 어느 정도 받는지에 관하여 모의실험을 통해 살펴보았다. 추가적으로, 문항반응이론 분야에서 전통적으로 활용되어 온 문항 적합도 지수인 Q1-χ2의 수행도 함께 조사하였다. 모의실험에서는 여러 이분 문항반응모형과 다양한 수준의 문항 수를 고려하여, 제1종 오류와 통계적 검정력 측면에서 문항 적합도 검정이 각 조건에 따라 어떻게 이루어지는지 확인하였다. 연구 결과, 검사 자료 분석을 위하여 문항반응이론을 활용하고자 하는 연구자는 BILOG-MG와 같이 일반적으로 널리 쓰이는 프로그램에서 제공하는 문항 적합도 지수보다 S-χ2 지수를 사용하는 것이 훨씬 더 바람직하다고 볼 수 있었다. 또한 표본크기가 매우 클 때에도 S-χ2 지수를 사용한다면 무선적 표집 절차 없이 모든 검사 응답 자료를 활용하여 문항 적합도 검정을 수행할 수 있는 것으로 나타났다.The main purpose of this study is to examine the extent to which the S-χ2 item fit index that follows chi-square distributions are affected by sample sizes in terms of type I error rates through a simulation study. In addition, the performance of Q 1 -χ 2 index which has been traditionally used in the area of item response theory were also investigated. In the simulation study, various dichotomous item response models and a few different numbers of items were considered to check these indices could perform adequately in terms of type I error and statistical power at each simulation condition. The study results showed that the S-X 2 index was able to control the type I error without excessive inflation even when the sample sizes are extremely large. The Q 1 -χ2 index, however, calculated by BILOG-MG program could provide the empirical type I error rates around the nominal significance level, 5%, only in very restricted conditions. In conclusion, a researcher who wants to use item response theory for the purpose of analyzing given test data needs to use the S-χ2 index rather than resorting to traditional item fit indices such as the Q 1 -χ 2 provided by commonly used commercial program like BILOG-MG. It was also shown that, if the S-χ2 index could be used even when the sample size was very large, then, no sampling process is required to lessen the size of data set.

DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework

This paper presents DFIT, an R package that implements the differential functioning of items and tests framework as well as the Monte Carlo item parameter replication approach for producing cut-off points for differential item functioning indices. Furthermore, it illustrates how to use the package to calculate power for the NCDIF index, both post hoc, as has regularly been the case in differential item functioning empirical and simulation studies, as well as a priori given certain item parameters. The version reviewed here implements all DFIT indices and Raju's area measures for tests comprised of items modeled with the same parametric item response unidimensional model (1-, 2-, and 3-parameters, generalized partial credit model or graded response model), the Mantel-Haenszel statistic with an underlying dichotomous item response model, and the item parameter replication method for any of the estimated indices with dichotomous item response models.

Efficient Standard Error Formulas of Ability Estimators with Dichotomous Item Response Models.

This paper focuses on the computation of asymptotic standard errors (ASE) of ability estimators with dichotomous item response models. A general framework is considered, and ability estimators are defined from a very restricted set of assumptions and formulas. This approach encompasses most standard methods such as maximum likelihood, weighted likelihood, maximum a posteriori, and robust estimators. A general formula for the ASE is derived from the theory of M-estimation. Well-known results are found back as particular cases for the maximum and robust estimators, while new ASE proposals for the weighted likelihood and maximum a posteriori estimators are presented. These new formulas are compared to traditional ones by means of a simulation study under Rasch modeling.

Estimation of Ability Using Pseudocounts in Item Response Theory

A method for estimation of ability using pseudocounts in dichotomous item response models is given when associated item parameters are known or estimated by a separate calibration sample of examinees with the size of an appropriate order. The pseudocount minimizing the asymptotic mean square error is algebraically obtained. Though the pseudocount depends on unknown ability, a fixed lower bound for the pseudocount is derived under the logistic model with equivalent items. The lower bound is numerically shown to be reasonable under the 3-parameter logistic model with and without model misspecification.

A Note on Weighted Likelihood and Jeffreys Modal Estimation of Proficiency Levels in Polytomous Item Response Models

Warm (in Psychometrika, 54, 427-450, 1989) established the equivalence between the so-called Jeffreys modal and the weighted likelihood estimators of proficiency level with some dichotomous item response models. The purpose of this note is to extend this result to polytomous item response models. First, a general condition is derived to ensure the perfect equivalence between these two estimators. Second, it is shown that this condition is fulfilled by two broad classes of polytomous models including, among others, the partial credit, rating scale, graded response, and nominal response models.

On the asymptotic standard error of a class of robust estimators of ability in dichotomous item response models

In item response theory, the classical estimators of ability are highly sensitive to response disturbances and can return strongly biased estimates of the true underlying ability level. Robust methods were introduced to lessen the impact of such aberrant responses on the estimation process. The computation of asymptotic (i.e., large-sample) standard errors (ASE) for these robust estimators, however, has not yet been fully considered. This paper focuses on a broad class of robust ability estimators, defined by an appropriate selection of the weight function and the residual measure, for which the ASE is derived from the theory of estimating equations. The maximum likelihood (ML) and the robust estimators, together with their estimated ASEs, are then compared in a simulation study by generating random guessing disturbances. It is concluded that both the estimators and their ASE perform similarly in the absence of random guessing, while the robust estimator and its estimated ASE are less biased and outperform their ML counterparts in the presence of random guessing with large impact on the item response process.

Testing manifest monotonicity using order-constrained statistical inference.

Most dichotomous item response models share the assumption of latent monotonicity, which states that the probability of a positive response to an item is a nondecreasing function of a latent variable intended to be measured. Latent monotonicity cannot be evaluated directly, but it implies manifest monotonicity across a variety of observed scores, such as the restscore, a single item score, and in some cases the total score. In this study, we show that manifest monotonicity can be tested by means of the order-constrained statistical inference framework. We propose a procedure that uses this framework to determine whether manifest monotonicity should be rejected for specific items. This approach provides a likelihood ratio test for which the p-value can be approximated through simulation. A simulation study is presented that evaluates the Type I error rate and power of the test, and the procedure is applied to empirical data.

A Unified Approach to IRT Scale Linking and Scale Transformations

Abstract. This paper examines IRT scale transformations and IRT scale-linking methods used in the nonequivalent groups with anchor test (NEAT) design to equate two tests, X and Y. It proposes a unifying approach to the commonly used IRT linking methods: mean-mean, mean-var linking, concurrent calibration, Stocking and Lord, and Haebara characteristic curves approaches, and fixed-item parameters scale linkage. The main idea is to view any linking procedure as a restriction on the item parameter space. Then, a rewriting of the log-likelihood function together with an appropriately implemented maximization procedure of the log-likelihood function under linear (or nonlinear) restrictions will accomplish the linking. The proposed method is general enough to cover both the dichotomous item response models (the one parameter logistic (1PL) model, 2PL, and 3PL) and the polytomous unidimensional IRT models such as the generalized partial credit model.

A UNIFIED APPROACH TO IRT SCALE LINKING AND SCALE TRANSFORMATIONS

ABSTRACTThis paper examines item response theory (IRT) scale transformations and IRT scale linking methods used in the Non‐Equivalent Groups with Anchor Test (NEAT) design to equate two tests, X and Y. It proposes a unifying approach to the commonly used IRT linking methods: mean‐mean, mean‐var linking, concurrent calibration, Stocking and Lord and Haebara characteristic curves approaches, and fixed‐item parameters scale linkage. The main idea is to view any linking procedure as a restriction on the item parameter space. Then a rewriting of the log‐likelihood function together with an appropriately implemented maximization procedure of the log‐likelihood function under linear (or nonlinear restrictions) will accomplish the linking. The proposed method is general enough to cover both the dichotomous item response models (the one parameter logistic (1PL) model, 2PL, and 3PL) and the polytomous unidimensional IRT models like the generalized partial credit model.

Asymptotic null distribution of person fit statistics with estimated person parameter

Person fit statistics are considered for dichotomous item response models. The asymptotic null distribution is derived for statistics which are linear in the item responses, and in which the ability parameter is replaced by an estimate. This allows the asymptotically correct standardization of linear person fit statistics with estimated ability parameter. The fact that the ability parameter is estimated usually decreases the asymptotic variance.

Stochastic order in dichotomous item response models for fixed, adaptive, and multidimensional tests

Dichotomous IRT models can be viewed as families of stochastically ordered distributions of responses to test items. This paper explores several properties of such distributions. In particular, it is examined under what conditions stochastic order in families of conditional distributions is transferred to their inverse distributions, from two families of related distributions to a third family, or from multivariate conditional distributions to a marginal distribution. The main results are formulated as a series of theorems and corollaries which apply to dichotomous IRT models. One part of the results holds for unidimensional models with fixed item parameters. The other part holds for models with random item parameters as used, for example, in adaptive testing or for tests with multidimensional abilities.

Information Functions of the Generalized Partial Credit Model

The concept of information functions developed for dichotomous item response models is adapted for the partial credit model. The information func tion is explained in terms of the model parameters and scoring functions. The relationship between the item information function and the item response function also is discussed. The informa tion function then is used to investigate the effect of collapsing and recoding categories of polytomously-scored items of the National Assess ment of Educational Progress (NAEP). The NAEP writing items were calibrated and the item and test information is used to discuss desirable properties of polytomous items.

INFORMATION FUNCTIONS OF THE GENERALIZED PARTIAL CREDIT MODEL

ABSTRACTThe concept of information functions developed for dichotomous item response models is adapted for the partial credit model. The information function is explained in terms of the model parameters and scoring functions. The relationship between the item information function and the expected score function is also discussed. The information function is then used to investigate the effect of collapsing and recoding categories of polytomously‐scored items of the National Assessment of Educational Progress (NAEP). Finally, the NAEP writing items are calibrated and the item and test information is used to discuss desirable properties of polytomous items.item response model polytomous item response model partial credit model information function NAEP