Item Information Functions Research Articles

Fundamentals of item response theory

Background Concepts, Models, and Features Ability and Item Parameter Estimation Assessment of Model-Data Fit The Ability Scale Item and Test Information and Efficiency Functions Test Construction Identification of Potentially Biased Test Items Test Score Equating Computerized Adaptive Testing Future Directions of Item Response Theory

An Application of Item Response Theory in the Comparison of Four Conventional Item Discrimination Indices for Criterion-Referenced Tests

Several recent papers have argued for the usefulness of item response theory (IRT) methods of assessing item discrimination power for criterion-referenced tests (CRTs). Conventional methods continue to be used more widely, however, for reasons that include some practical constraints associated with the use of IRT methods. To provide users with information that may help them to decide on which conventional indices to employ in evaluating CRT items, Spearman rank-order correlations were computed between IRT-derived item information functions (llFs) and four conventional discrimination indices: the phi-coefficient, the B-index, phi/phi max, and the agreement statistic. The rank-order correlations between the phi-coefficient and the llFs were very high, with a median of .96. The remaining conventional indices, with the exception of phi-over-phi-max, also correlated well with the IIF. Theoretical explanations for these relationships are offered.

A rasch model for continuous ratings

A unidimensional latent trait model for continuous ratings is developed. This model is an extension of Andrich's rating formulation which assumes that the response process at latent thresholds is governed by the dichotomous Rasch model. Item characteristic functions and information functions are used to illustrate that the model takes ceiling and floor effects into account. Both the dichotomous Rasch model and a linear model with normally distributed error can be derived as limiting cases. The separability of the structural and incidental parameters is demonstrated and a procedure for estimating the parameters is outlined.

A Latent Trait Look at Pretest-Posttest Validation of Criterion-referenced Test Items

Since Cox and Vargas (1966) introduced their pretest-posttest validity index for criterion-referenced test items, a great number of additions and modifications have followed. All are based on the idea of gain scoring; that is, they are computed from the differences between proportions of pretest and posttest item responses. Although the method is simple and generally considered as the prototype of criterion-referenced item analysis, it has many and serious disadvantages. Some of these go back to the fact that it leads to indices based on a dual test administration- and population-dependent item p values. Others have to do with the global information about the discriminating power that these indices provide, the implicit weighting they suppose, and the meaningless maximization of posttest scores they lead to. Analyzing the pretest-posttest method from a latent trait point of view, it is proposed to replace indices like Cox and Vargas’ Dpp by an evaluation of the item information function for the mastery score. An empirical study was conducted to compare the differences in item selection between both methods.

CARIF: A computational program for item information functions and item characteristic curves

CARIF: A computational program for item information functions and item characteristic curves

A Latent Trait Look at Pretest-Posttest Validation of Criterion-Referenced Test Items

Since Cox and Vargas (1966) introduced their pretest-posttest validity index for criterion-referenced test items, a great number of additions and modifications have followed. All are based on the idea of gain scoring; that is, they are computed from the differences between proportions of pretest and posttest item responses. Although the method is simple and generally considered as the prototype of criterion-referenced item analysis, it has many and serious disadvantages. Some of these go back to the fact that it leads to indices based on a dual test administration- and population-dependent item p values. Others have to do with the global information about the discriminating power that these indices provide, the implicit weighting they suppose, and the meaningless maximization of posttest scores they lead to. Analyzing the pretest-posttest method from a latent trait point of view, it is proposed to replace indices like Cox and Vargas’ Dpp by an evaluation of the item information function for the mastery score. An empirical study was conducted to compare the differences in item selection between both methods.