Detecting Gender-Based Differential Item Functioning on a Constructed- Response Science Test

Abstract

In this study, I explored methods for detecting gender-based differential item functioning on a 12th-grade constructed-response (CR) science test administered as part of the National Education Longitudinal Study of 1988 (NELS:88). The primary difficulty encountered with many CR tests is the absence of a reliable and appropriate measure of ability on which to condition. In this study, several combinations of conditioning variables were explored, and results were supplemented with evidence from interviews of students who completed the test items. The study revealed that 1 item in particular displayed a large male advantage and contributed to the gender difference on total score. Results were similar to those obtained with the NELS:88 multiple-choice test. In both cases, gender differences were largest on items that involved visualization and called on knowledge acquired outside of school. Implications for users of large-scale assessment results are discussed.