Does the Use of Test Assembly Procedures Proposed in Legislation Make Any Difference in Test Properties and in the Test Performance of Black and White Test Takers?

Abstract

In this study, four simulated verbal and mathematical test forms were produced using variants of test assembly procedures of the type proposed in 1986 California and New York legislative bills. These procedures were intended to minimize score differences between majority and minority groups by manipulating differences in the proportions correct of test items. Item response theory procedures were used to estimate the effects of administering the verbal and mathematical forms to approximately 22,000 Black and 28,000 White high-school juniors and seniors. Content and statistical characteristics differed considerably from form to form when these characteristics were not controlled explicitly in test assembly. In addition, several of the simulated test forms manifested reduced differences between Blacks and Whites in scaled score means. These differences were apparently due, not to the manipulation of differences in item proportions correct, but rather to the indirect effect of including items that functioned ...