THE OPTIONAL ESSAY PROBLEM AND THE HYPOTHESIS OF EQUAL DIFFICULTY

Abstract

ABSTRACTWe examine a special case of examinee choice, the Optional Essay Problem, from the point of view of test equating. The Optional Essay Problem involves equating essay scores when the examinees are required to select an optional essay topic from a list of topics in addition to taking a mandatory test required of all examinees. We derive conditions that must be satisfied if the null hypothesis of ‘equal difficulty’ of the essays is true. (We call this Livingston's Null Hypothesis.) If this hypothesis holds then there is no need to equate the scores on the optional essays. Our conditions take the form of inequalities about unobservable quantities that may be displayed graphically. We illustrate them using a real example from the Advanced Placement Examinations. Then we analyze Livingston's (1988) proposal for adjusting essay scores in the Optional Essay Problem and explain it from the perspective of test equating. We use our explanation to generalize his proposal to two new proposals that are explicit about the assumptions they make concerning the unobserved data. (We argue that every method for adjusting the essay scores in the Optional Essay Problem must make assumptions about unobserved data.) We illustrate the adjustment methods with an example from the Advanced Placement Examinations. Finally, we use the results for adjusting optional essay scores to propose comparable procedures for directly adjusting linear composite scores that include both a mandatory and an optional test score.