The appropriacy of psychometric measurement models for testing second language listening comprehension

Abstract

This article examines the assumptions made by classical and item response (IRT) measurement models, especially the assumption of unidimensionality, and compares these with current theories of listening comprehension, which commonly explain listening in cognitive information-processing terms as a multidimensional interaction between a number of linguistic and non- linguistic skills. On the basis of this, it is hypothesized that second language listening tests will not meet the assumptions made by commonly used, norm- referenced measurement models. This hypothesis is examined by means of the verbal report methodology. Using a structured interview procedure, six L2 listening test-takers are asked to introspect while taking a short-answer listening comprehension test. The resulting protocols are then analysed in order to ascertain the knowledges, skills and abilities which influenced item performance. Results indicate that items typically require a variety of skills for successful performance, and that these usually differ from one test-taker to another. It is argued that this constitutes a violation of the assumption of unidimensionality made by norm-referenced measurement models currently in use.