Direct Assessment, Direct Validation? An Example From the Assessment of Writing

Abstract

Complex performance assessments are becoming increasingly popular, at least in part because of the directness with which these assessments are accomplished. As Messick (1992) argued, however, the apparent directness of such measures does not exempt them from the need to marshal validity evidence in support of their use. An example is given of one kind of evidence-assessment users' satisfaction with the performances generated by examinees - that may contribute to the validation of performance assessments. The example is from the realm of the direct assessment of writing in the context of graduate school admissions. The data revealed that the correspondence between score level and faculty satisfaction with the papers was strong but not completely linear. Satisfaction and dissatisfaction were not opposite sides of the same coin.