Abstract
Objective examination questions are used for assessment in many disciplines and levels. They allow extensive coverage, reliability, re-usability and administrative ease, but the amount of information they gather about student knowledge is not widely known. In this paper, I show that small changes in question format make large differences to the information gained, and therefore to the efficiency of the assessment process. The basic objective question (BOQ) can be extended beyond the true/false response to test probabilities in complex situations. When objective questions are used to assess students’ abilities, the information gained should be sufficient in quantity and coverage and at the appropriate DOK (Depth of Knowledge) to give a robust and valid assessment. Objective questions take time and expertise to set and validate, so we should seek methods that meet these criteria with the least administrative load. The basic objective question is a statement followed by two or more responses; one response must be marked as correct. The Type 1 MCQ (Multiple Choice Question), multiple true/false question (MTFQ), extended matched question (EMQ) and others are all variants of the BOQ, but they differ widely in the amount of information they gain. For a given input of statements and responses, the Type 1 MCQ delivers the least information about the student’s knowledge, the MTFQ and the BOQ deliver almost four times as much and the EMQ about six times as much. The expected level of student knowledge also affects the information gained. If questions are set so that most (say 90%) of students get the correct answers, then the information gained, and the discriminatory power of the test is much less than when the test is set for lower expected levels. The format used for objective questions and the expected level of correct responses govern their efficiency in collecting pass/fail information. The quality of that information such as depth of knowledge (DOK) depends on the wording and situations modeled for the questions and this is independent of their format.
Highlights
Objective questions are a major form of testing in education, including medical education
The formats are all variations of the basic objective question (BOQ)
All objective questions use the basic form of statement followed by a selection of responses, but these can be assembled in different ways and conditions may be used to limit the choice of responses
Summary
Objective questions are a major form of testing in education, including medical education. Medical teachers have been imaginative and prolific in designing new formats of objective questions to meet different needs and to test a wide range of clinical situations and knowledge. These techniques can be used in other areas. The formats are all variations of the basic objective question (BOQ). We can compare their properties, and their information contents to select the most efficient and valid formats. I shall demonstrate the formats of various formats of objective questions, calculate their information content and administrative efficiency and suggest appropriate formats for specific purposes
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
