Measuring Student Knowledge, Ignorance and Risk Aversion with Objective Examination Questions

Alan Dugdale

doi:10.4236/ce.2016.715227

Abstract

This paper applies deductive logic to the structure and outcomes of objective questions. When all terms are clearly defined and deductive logic is correctly used, then the results must be true, even if they are surprising or counter-intuitive. The defined premises are 1) student’s true knowledge, 2) student’s false knowledge, 3) student’s admitted ignorance and 4) student’s level of risk aversion. Definitions for 5) correct answer, 6) wrong answer, and 7) abstain are also needed. Some logical processes are numerical and involve simple equations. The main outcomes include: i) the standard scoring of correct answers, the number of wrong answers and the number of abstain can result from varying levels of true knowledge, false knowledge and admitted ignorance; ii) the level of risk aversion is measured by the proportion of answers where the student abstains from answering. High risk aversion lowers standard score, but this effect can be easily corrected.

Highlights

Objective questions have been used for more than seventy years, and scores form a large part of student assessment
The main outcomes include: i) the standard scoring of correct answers, the number of wrong answers and the number of abstain can result from varying levels of true knowledge, false knowledge and admitted ignorance; ii) the level of risk aversion is measured by the proportion of answers where the student abstains from answering
Objective questions have been a major form of student assessment for many years

Summary

Introduction

Objective questions have been used for more than seventy years, and scores form a large part of student assessment. 2) in assessment for future professional careers, we should be testing the application of that knowledge, which is a higher cognitive taxonomy This can be done by changing the stem of objective questions from a simple statement to a clinical scenario, so the student needs both relevant correct knowledge and the ability to apply it. Under these conditions, admitted ignorance (DK don’t know) is not dangerous, but misinformation (FK false knowledge) or the wrong use of true knowledge (TK) can lead to clinical error and patient harm. If all premises are true, the terms are clear, and the rules of deductive logic are followed, the conclusions reached are necessarily true

The Basic Structure of Objective Questions

Premises and Definitions

Deductions from Basic Equations

The Logic of Type 1 MCQs

Findings

Discussion