Dealing with flawed items in examinations: Using the compensation of disadvantage as used in German state examinations in items with partial credit scoring.

Abstract

In the written part of German state exams, multiple choice questions which inherit a flaw are not always excluded but taken into consideration when awarding a grade to any individual candidate, if this is advantageous to the candidate. This process of elimination and compensation of disadvantage in flawed questions, as applied by the Institut für medizinische und pharmazeutische Prüfungsfragen (IMPP), can lead to different items being taken into consideration in the grading in individual cases and to the application of different pass and grade boundaries in individual cases.This procedure is applied here to examinations containing items with partial credit scoring. A simple numerical rule can be applied which can be used to decide whether a flawed item has be taken into account in the grading procedure in order to achieve a pass or a particular grade for a candidate. This rule – as in the state examinations – must take account of the individual number of scores achieved in the flawed item and, if the relative pass threshold ("automatic adjustment clause") is applied, the average score achieved in the item by all candidates. In addition, however, it also includes the grade boundaries.