Item Bias in a Test of Reading Comprehension

Abstract

The possibility that certain features of items on a reading comprehension test may lead to biased esti mates of the reading achievement of particular sub groups of students was investigated. Eight nonover lapping subgroups of students were defined by the combinations of three factors: student grade level (fifth or sixth), income level of the neighborhood in which the school was located (low and middle or above), and race of the student (black or white). Es timates of student ability and item parameters were obtained separately for each of the eight subgroups using the three-parameter logistic model. Bias in dices were computed based on differences in item characteristic curves for pairs of subgroups. A cri terion for labeling an item as biased was developed using the distribution of bias indices for subgroups of the same race that differed only in income level or grade level. Using this criterion, three items were consistently identified as biased in four independent comparisons of subgroups of black and white stu dents. Comparisons of content and format charac teristics of items that were identified as biased with those that were not, or between items biased in dif ferent directions, did not lead to the identification of any systematic content differences. The study did provide strong support for the viability of the esti mation procedure; item characteristics, estimated with samples from different populations were very similar. Some suggestions for improvements in methodology are offered.