A Rasch modeling approach to analyzing students’ incorrect answers on multiple-choice questions

Abstract

An analysis of students’ difficulties for a curricular topic may help the educator to gain better insight into students’ reasoning about that topic which is a prerequisite for high-quality teaching. The purpose of this study was to demonstrate how distractor analysis may be used for identifying students’ difficulties in a certain topic. In order to be in position to perform invariant measurement and to easily relate students’ difficulties to their achievement levels, we decided to take a Rasch modeling approach. Our study included 14 wave optics items and 286 students from five universities in Slovenia, Croatia, and Bosnia and Herzegovina. Rasch modeling was used to estimate item and student measures, as well as to create option probability curves which allowed us to relate students’ achievement levels to the choice of individual distractors. It has been found that all 14 included items function in line with the Rasch model. In 10 out of 14 items there were distractors chosen by at least 25% of students. For several out of these 10 items, the option probability curves indicated that attractiveness of individual distractors depended on students’ ability levels. We could conclude that the Rasch-based distractor analysis may provide very useful information for differentiation of physics instruction.