Untangling complex effects of disabilities and accommodations within a multilevel IRT framework

Abstract

A small number of previous studies on setting accommodation suggest that this accommodation is beneficial for students with special needs; a majority of the research has been survey type studies concentrating on participants’ subjective judgements. To test previous assumptions that the use of this accommodation is valid, and disentangle the complex person- and item-level effects of the data, this study extended the conventional multilevel item response theory model by explicitly modeling multiple student-level predictors and the interactions between the predictors to examine the differential difficulties of math and reading items for students participating in provincial assessments, specifically in relation to whether they received setting-only accommodation and had a learning disability. Both group differences in difficulty across all items and for individual items [differential item functioning (DIF)] were examined. The results show that math and reading items were relatively more difficult for accommodated than non-accommodated students in learning disabilities and non-disabled groups; several items were found to exhibit DIF. These empirical findings demonstrate that there is good potential to apply the methodology to structured data at and beyond accommodation studies.