Abstract
Two-tier multiple-choice (TTMC) items are used to assess students’ knowledge of a scientific concept for tier 1 and their reasoning about this concept for tier 2. But are the knowledge and reasoning involved in these tiers really distinguishable? Are the tiers equally challenging for students? The answers to these questions influence how we use and interpret TTMC instruments. We apply the Rasch measurement model on TTMC items to see if the items are distinguishable according to different traits (represented by the tier), or according to different content sub-topics within the instrument, or to both content and tier. Two TTMC data sets are analyzed: data from Singapore and Korea on the Light Propagation Diagnostic Instrument (LPDI), data from the United States on the Classroom Test of Scientific Reasoning (CTSR). Findings for LPDI show that tier-2 reasoning items are more difficult than tier-1 knowledge items, across content sub-topics. Findings for CTSR do not show a consistent pattern by tier or by content sub-topic. We conclude that TTMC items cannot be assumed to have a consistent pattern of difficulty by tier—and that assessment developers and users need to consider how the tiers operate when administering TTMC items and interpreting results. Researchers must check the tiers’ difficulties empirically during validation and use. Though findings from data in Asian contexts were more consistent, further study is needed to rule out differences between the LPDI and CTSR instruments.
Highlights
Two-tier multiple-choice (TTMC) items are used to assess students’ knowledge of a scientific concept for tier 1 and their reasoning about this concept for tier 2
Despite the notable progress in development and use of two-tier items, much of the work on TTMC items has focused on the diagnostic use of these instruments
Because each two-tier item consists of two items, a tier-1 item and a tier-2 item, the term item will refer to the tier-1 or tier-2 item only, whereas when we refer to the two-tier-item as a whole we will use the term two-tier item
Summary
Two-tier multiple-choice (TTMC) items are used to assess students’ knowledge of a scientific concept for tier 1 and their reasoning about this concept for tier 2. The lack of research to answer these questions hinders the field because it threatens the validity of our use of two-tier items to examine these different cognitive skills To address this lack of research, we conducted a study of students’ responses to two-tier items to consider how the tiers are related in terms of their respective difficulty—whether identifying one’s reasoning is more or less difficult than showing one’s knowledge—and whether this pattern in difficulty is consistent with the distinct but related abilities the tiers are intended to assess. We do this through the application of the Rasch measurement model on both first and second tiers. We review research on two-tier items to describe their types and uses, development and analysis of such items, and move on to discuss a gap in the literature on two-tier items that motivates our research questions for this study
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
