Investigating Different Theory-Based Differential Item Functioning Methods Under Different Scoring Situations

Abstract

This study intended to investigate the operations of various theory-based Differential Item Functioning (DIF) methods under different scoring situations. In this context, data obtained from conducting a Verbal Reasoning Skills Test (scored both in dichotomous and weighted scores) on 1593 individuals was examined via Mantel-Haenszel/Mantel, Standardization and Likelihood Ratio Test techniques and items that point to DIF and numbers of items with DIF for each technique and scoring situation were identified. Results show that Classical Test Theory (CTT) based DIF methods generally provided results that were consistent in themselves and that these results were not significantly different from each other. Agreement between techniques based on Item Response Theory (IRT) and Classical Test Theory (CTT) was found to be low under different scoring situations and results were significantly different from each other.