Validation of listening comprehension tests: the effect of text and question type

Abstract

The paper reports on a study which investigated the effect of both texts and question types on test taker's scores on listening comprehension tests. Listening comprehension tests were administered to 150 EFL learners in their last year of secondary school. The listening stimuli consisted of three text types, a newsbroadcast, a lecturette, and a consultative dialogue, varying in the degree of oral features they contained. The consultative dialogue was the most orally-oriented version, followed by the lecturette and followed then by the newsbroadcast which constituted the most literate version of the three text types. Test takers listened to two different versions about the two topics and answered identical questions to enable comparison of performance on the different text types. The questions were classified into global and local types, according to the strategies utilized for text processing. A third category, trivial questions type, referred to items relating to trivial factual details, such as numerical figures and names. Results indicated that different types of texts located at different points on the oral/literate continuum resulted in different test scores, so that the more 'listenable' texts were easier. In terms of the ques tion types, the results showed that subjects performed better on items referring to local cues than on items referring to global cues. This was observed across topics as well as across text types and across students' levels. Trivial questions were seen to affect performance differently. Implications of the results for the selection of texts and tasks on listening comprehension tests are drawn to arrive at highly construct valid listening comprehension tests.