Test fairness: a DIF analysis of an L2 vocabulary test

Abstract

The purpose of this study is to analyse gender-uniform differential item functioning (DIF) in a second language (L2) vocabulary test with the tools of item response theory (the separate calibration t-method) and to study potential gender impact on the test performance measured by different item composites. The results of the study show that despite the fact that there are test items with indications of DIF in favour of either females or males, the test as a whole is not gender-biased. In spite of this, it was demonstrated that some item composites are gender-biased. In view of item bank building and use, it means that some of the tests constructed on the basis of an item bank might be biased if the item bank contains items with indication for DIF. Although the results of some empirical research suggest that the requirements for items with DIF to be excluded from the final test version may on the whole be too restrictive, this study demonstrated that the traditional advice of excluding biased items gains new significance in the light of item bank building and use since doing so will prevent possible biased item composites.