Reporting Differentiated Literacy Results in PISA by Using Multidimensional Adaptive Testing

Abstract

Multidimensional adaptive testing (MAT) allows for substantial increases in measurement efficiency. It was examined whether this capability can be used to report reliable results for all 10 subdimensions of students’ literacy in reading, mathematics and science considered in PISA. The responses of N = 14,624 students who participated in the PISA assessments of the years 2000, 2003 and 2006 in Germany were used to simulate unrestricted MAT, MAT with the multidimensional maximum priority index method (MMPI), and MAT with MMPI taking typical res­trictions of the PISA assessments (treatment of link items, treatment of open items, grouping of items to units) into account. For MAT with MMPI the reliability coefficients for all subdimensions were larger than .80, as opposed to sequential testing based on the booklet design of PISA 2006. These advantages slightly lessened with the incorporation of PISA-typical restrictions. The findings demonstrate that MAT with MMPI can successfully be used for subdimensional reporting in PISA.