Equating Test Forms for Progress in International Reading Literacy Study (PIRLS 2011) in Sultanate of Oman Using Item Response Theory

Abstract

The study aims to compare between true score of mixed item formats (dichotomous and polytomous items), using single group design and nonequivalent group with common items design. The study used PIRLS test, which consists of 146 items, (74 multiple-choice items and 72 short answer items), distributed over 13 booklets. Each booklet has common items with some other booklets and some uncommon items, as well as two passages: reading for literary experience and reading to acquire and use information. The 13 booklets were distributed to a random sample of 10394 students from grade 4 in the Sultanate of Oman. MULTILOG 7.03 software was used to estimate item and ability parameters using the graded response model and Poly Equate (V0.5) software. The 13 booklets were equated using true score equating and observed score equating. The results showed differences in the descriptive characteristics for the 13 booklets before equating, and reduction in these differences after equating with both equating methods. Paired samples t-test showed statistical significant differences between the two equating methods with all booklets towards the observed score equating except with booklet 10 where there are differences towards true score equating and booklets 4 and 5 where there is no statistical difference between the two equating methods.