SMALL‐SAMPLE DIF ESTIMATION USING LOG‐LINEAR SMOOTHING: A SIBTEST APPLICATION

Abstract

ABSTRACTThe purpose of the current study was to examine whether log‐linear smoothing of observed score distributions in small samples results in more accurate differential item functioning (DIF) estimates under the simultaneous item bias test (SIBTEST) framework. Data from a teacher certification test were analyzed using White candidates in the reference group and African American candidates in the focal group. Smoothed and raw DIF estimates from 100 replications under seven different sample‐size conditions were compared to a criterion to determine the effect of smoothing on small‐sample DIF estimation. Root‐mean‐squared deviation and bias were used to evaluate the accuracy of DIF detection in the smoothed versus raw data conditions. Results indicate that, for most studied items, smoothing the raw score distributions reduced variability and bias of the DIF estimates especially in the small‐sample‐size conditions. Implications of these results for actual testing programs and future directions for research are discussed.