2 - Robustness of Item Response Models and Effects of Test Length and Sample Size on the Precision of Ability Estimates

Abstract

This chapter presents two studies to discuss the robustness of item response models and effects of test length and sample size on the precision of ability estimates. The purpose of Study 1 was to study systematically the goodness of fit of the one-, two-, and three-parameter logistic models. Using computer-simulated test data, the effects of four variables were studied: (1) the variation in item discrimination parameters; (2) the average value of the pseudochance-level parameters; (3) the test length; and (4) the shape of the ability distribution. Artificial or simulated data representing the departures of varying degrees from the assumptions of the three-parameter logistic test model were generated and the goodness of fit of the three test models to the data was studied. The goodness-of-fit measures used were chosen for their practical significance. The Study 2 was designed to investigate two practical questions that are important to test developers: (1) the effects of examinee sample size and test length on the standard errors of ability estimation (SEE) curves and (2) the effects that the statistical characteristics of an item pool have on the precision of standard errors of ability estimation curves. A study of the use of SEE curves and factors that affect their stability was motivated by item response model test development procedures. When item statistics are available they are commonly used by test developers for selecting test items from a pool of test items to produce a test with a desired SEE curve.