Inversions in true/false and in multiple choice questions -- a new form of item analysis.

Abstract

Summary Total ignorance is expressed not by consistently selecting the wrong answers in a true/false or multiple choice examination, but rather by making a random choice from all the alternatives available, resulting in a predictable proportion of correct responses. When a significant majority of students answers incorrectly in a true/false question, it must therefore be concluded that this is not the result of ignorance, but of some factor which has led to their deliberately choosing the wrong answer. This situation has been termed an ‘inversion’. Of 2057 true/false examination questions set in this department over the past 3 years, 140 (6.8%) were found to be ‘inverted’. In about one third of these cases the ‘inversion’ was found to be due to a wrong model answer, an ambiguous or misleading question, or to incorrect teaching. The remaining two-thirds of the ‘inversions’ could be ascribed to attempts to work out the answer from inadequate or inappropriate first principles. It has been our policy to delete all ‘inverted’ questions from the examination (unless the model answer needed to be changed), because the score obtained when the students deliberately chose the wrong answer is less than the score they would have obtained had they been totally ignorant and had ventured a guess or had chosen not to answer the question. This concept can also be applied to multiple choice questions.