A Comprehensive Framework for Evaluating Hypotheses About Cultural Bias in Educational Testing

Abstract

The purpose of this article is to present a working definition of the term culture, as well as to describe and demonstrate a comprehensive framework for evaluating hypotheses about cultural bias in educational testing. The framework is demonstrated using 5th-grade reading and language arts data from the Terra Nova test (CTB/ McGraw-Hill, 1999). The example begins at the broadest level of analyzing bundles of items that are thought to illustrate aspects of the Hispanic or Black cultures in the correct or incorrect option and reduces to the analysis of individual distractors. The results show that although Hispanic and Black examinees did not differ from matched White examinees in their bundle scores, Black examinees were differentially drawn to incorrect options that illustrated aspects of their culture in comparison to matched White examinees. Implications of the results are discussed, as well as suggestions for future research.