Expert agreement in judging art projects — A myth or reality?

Abstract

In art education, it is customary to accept the expertise of the art teacher as an impartial and objective evaluator of student achievement. But is he or she really impartial and reliable judge of quality? Is the jury method generally used for the assessment of works of art (Schonau, 1994) a valid method for authentic assessment? If so, are all genres and topics of child art equally suitable for assessment? In order to modernize final examinations in the visual arts, basic issues like these had to be addressed as the impressive amount of literature on child art never seems to question the validity of assumptions made by a single evaluator or a body of jurors. Research on child art usually focuses on the description of models for development as evaluated by experts whose training entitles them for judgment (Gardner, 1993; Kindler, 1997). We know of no previous research project that would have questioned the objectivity and reliability of the judgment of these experts. Our experiences as members of art juries disproves the claim of “expert agreement” reached through negotiations. Usually, it is the rhetoric of one of the jurors based on blurred definitions of “beauty” and “creativity”, and not the objective application of previously agreed upon criteria that decides about the merit of a work. This procedure, however, cannot be utilized for examination purposes. But can it be improved? Is there a more reliable way of conducting a jury of works of art?