The effect of question preview in listening comprehension tests

Abstract

Previewed comprehension questions in listening tests pose questions of construct validity: how does this test method affect the test results? An examination of the construct of listening comprehension suggests that question preview may affect comprehension positively by focusing the attention or supplying information about the text, or negatively by interfering with subjective comprehension processes, increasing the burden on the attention or imposing shallower processing. Alterna tively it may have no significant effect. The quantitative effect on test results and the qualitative effect on recall were predicted for each possibility. In the study, 78 subjects took listening tests in four different versions, one with questions before, one with questions after, one with questions sandwiched between two hear ings and one with no questions. Subjects also completed questionnaires designed to elicit reactions to each version and, a week later, wrote a free recall of what they had heard. Results from the recall were inconclusive but in the tests the `sandwich' version produced significantly more correct answers. Questionnaire responses indicated that subjects felt the sandwich version facilitated comprehen sion most, but also showed a strong affective attachment to previewed questions. It is concluded that previewed questions seem more helpful than they really are.