Reverse Engineering a Multiple-Choice Test Blueprint to Improve Course Alignment

Abstract

Large introductory classes typically reflect courses that cover a broad landscape of learning and necessitate assessment formats that are efficient and reliable, making multiple-choice (MC) tests a good option. Designing a good quality MC test requires reflection on the appropriateness of the assessment format, designing a test blueprint, and writing MC questions following item-writing guidelines that align with course learning objectives and teaching/learning strategies. It is a time-consuming endeavour that often requires a group of question writers who are experts in the domain—both of which can be challenging considering the demands of teaching in higher education. Sometimes we want to adopt or adapt existing MC tests that we have access to. I suggest a five-step “reverse engineering” analytical process for selecting questions from existing MC tests to utilize in our course assessment, emphasizing the alignment process. In one of the steps, we will refer to the Taxonomy Table, a practical two-dimensional aid proposed by Anderson and Krathwohl (2001), to select the questions that cover a cognitive dimension landscape. The outcome of this process provides an effective visual representation of the alignment and enables us to decide on the utility of questions for the context of our unit and assessment objectives.
             Keywords: course alignment, assessment, multiple-choice test, taxonomy table.