Assessing Measurement Invariance in ASWB Exams: Regulatory Research Proposal to Advance Equity

Abstract

ABSTRACT Purpose Social workers from minoritized racial, ethnic, linguistic, and age groups are far less likely to pass licensing examinations required to practice. Using a simulated data set, our study investigates measurement equivalence, or invariance, of social work licensing exams. Materials For this analysis, we simulated responses to 15 multiple-choice questions which were scored as either correct or incorrect using the R mirt package and used mirt to fit a 2-parameter logistic model (2PL) to the response data. We generated the data so that five items could demonstrate DIF and calculated their impact on the test characteristic curves and item characteristic curves. Results Small amounts of differential item functioning added up into differential test functioning, but the effect size was small. This result is one potential outcome of an analysis of ASWB exams. Discussion Most studies evaluating test characteristic curves demonstrate small effect sizes. Measuring the test characteristic curve and the test information curve will help to investigate content-irrelevant sources of variance in the exams, including unfairness, unreliability, and invalid pass scores. Conclusion Differential test functioning is a core part of measurement invariance studies. Psychometric standards require test developers to assess measurement invariance at both the item-level and test-level to protect themselves from accusations of bias.