Effects of availability of diagnostic and non-diagnostic cues on the accuracy of teachers’ judgments of students’ text comprehension

Abstract

AbstractAccurately judging students’ comprehension is a key professional competence for teachers. It is crucial for adapting instruction to students’ needs and thereby promoting student learning. According to the cue-utilization framework, the accuracy of teachers’ judgments depends on how predictive (or diagnostic) the information (or cues) that teachers use to make judgments is of student performance. It is, however, unclear from prior studies if merely providing access to diagnostic cues aids accuracy, or whether this only helps if non-diagnostic cues are unavailable or ignored. Therefore, we investigated, using a within-subjects experimental design, the accuracy of secondary school teachers’ (N = 33) judgments of anonymous students’ text comprehension under four cue availability conditions: 1) non-diagnostic cues only; 2) diagnostic cues only; 3) a mix of diagnostic and non-diagnostic cues; and, 4) after an intervention informing them of the diagnosticity of cues, again a mix of diagnostic and non-diagnostic cues. Access to diagnostic cues enhanced teachers’ judgment accuracy, while access to non-diagnostic cues hindered it. While teachers’ judgment accuracy was not enhanced by the intervention (presumably because it was already relatively high), their diagnostic cue utilization increased, and non-diagnostic cue utilization decreased. In addition, teachers’ calibration increased after the intervention: They knew better when their judgments were (in)accurate. Furthermore, teachers were quite aware that diagnostic cues are diagnostic, but their awareness that non-diagnostic cues (especially students’ interest) are not, could be improved. These results could be useful in designing effective interventions to further foster teachers’ judgment accuracy.