From Single Words to Passages: Contextual Effects on Predictive Power of Vocabulary Measures for Assessing Reading Performance

Abstract

In the last 15 years or so, language testing practitioners have increasingly favored assessing vocabulary in context. The discrete-point vocabulary measure used in the old version of the Test of English as a Foreign Language (TOEFL) has long been criticized for encouraging test candidates to memorize wordlists out of context although test items based on this format were reported to have provided some useful and reliable prediction of TOEFL reading performance. Language testers believe that vocabulary items embedded in full passages in a reading test will create more positive washback; out of this good intention the vocabulary measurement in the TOEFL Reading subtest was revised to contain only passage-based items. However, apart from Henning's (1991) study, there is little empirical evidence on how a passage-based vocabulary assessment format would compare with the traditional discrete-point vocabulary format in terms of predicting TOEFL reading performance. This article reviews the development of vocabulary item types in the TOEFL and reports on an investigation into the power of the discrete-point and passage-based vocabulary item types in predicting reading scores. Data were collected from more than 200 international students who completed a discrete-point vocabulary measure and a TOEFL Reading Comprehension subtest, which contained 12 questions on vocabulary items embedded in the reading passages. Data were analyzed using correlations, hierarchical multiple regression, and item difficulty and discriminability indexing based on point-biserial correlations. Statistical results indicate that, in assessing reading performance, discrete-point vocabulary items and fully contextualized vocabulary items provide a similar amount of prediction. However, in the context of considering educational impact, this article argues in favor of the continued adoption of the fully contextualized vocabulary item format because it will more likely induce beneficial washback effects than the discrete-point vocabulary item format. The contextualized format also has the advantage of bringing vocabulary testing closer to real-life communicative application of the English language and therefore has more positive implications for the language classroom.