Validity and Structure, Coherence, and Quality Measures in Writing

Abstract

The purpose of the present study was to investigate convergent and divergent validity of inferences from selected measures of struture, coherence, and quality for compositions. Thirty sixth-graders wrote two stories each. Each story was scored six times, for each of three constructs (structure, coherence, and quality), using two ways of measuring (reader- and text-based). Patterns in correlation matrices were examined for the extent to which they conformed to expectations which would support convergent and divergent validity. The major conclusions of the study were: (a) Overall, the findings strongly supported validity of inferences from the structure measures, called into serious question the validity of inferences from the coherence measures, and weakly supported it for inferences from the quality measures; (b) some evidence was provided for the distinctiveness of the three constructs, structure, coherence, and quality; and (c) some method/halo bias emerged in the reader-based method of assessment.