MULTIPLE‐CHOICE CLOZE ITEMS AND THE TEST OF ENGLISH AS A FOREIGN LANGUAGE

Abstract

ABSTRACTThis study examined the relation of TOEFL® performance to a widely used variant of the cloze procedure–the multiple‐choice (MC) cloze method. A main objective was to determine if categories of MC cloze items could be identified that related differentially to the various parts of the TOEFL. MC cloze items were prepared and classified according to whether the involvement of reading comprehension, as defined by sensitivity to long‐range textual constraints, was primary or secondary. For two categories, reading comprehension was primary and knowledge of grammar or vocabulary was secondary, and for two other categories knowledge of grammar or vocabulary was primary and reading comprehension secondary.Examinees taking an operational TOEFL at domestic test centers were given the three basic sections of the test along with a fourth section containing the MC cloze items. Performance was examined for each of nine major language groups.Exploratory and confirmatory factor analyses for the basic TOEFL were performed first, to provide a basis for relating the MC cloze items to the TOEFL structure. These factor analyses suggested that, from a practical standpoint, TOEFL performance can be adequately described by just two factors, which relate to (a) Listening Comprehension, and (b) all other parts of the test–Structure, Written Expression, Vocabulary, and Reading Comprehension.Examination of the MC cloze test showed that the total MC cloze score was relatively reliable and that it was possible to estimate item response theory parameters for the MC cloze items with reasonable accuracy. Thus, the development of the MC cloze items was successful in these respects. However, the correlations among scores for the four MC cloze item categories were approximately as high as their reliabilities, thus providing no strong empirical evidence that the item types within the MC cloze test reflected distinct skills.Correlational analyses related the four MC cloze categories to the five parts of the TOEFL. These analyses revealed a slight tendency for MC cloze items that involved a combination of grammar and reading to relate more highly to the Structure and Written Expression parts of the TOEFL than the other parts, and for MC cloze items that involved a combination of vocabulary and reading to relate more highly to the Vocabulary and Reading Comprehension parts of the TOEFL than the other parts. Although this pattern was relatively consistent across language groups, however, the differences among correlations were not substantial enough to be of practical importance.Multiple regression analyses were performed, using total MC cloze score as the dependent variable and the five TOEFL parts as independent variables. The resulting multiple Rs were mostly in the lower to upper .90s, suggesting that total MC cloze performance can be predicted from TOEFL performance with a relatively high degree of accuracy.In general, the study provided no evidence that distinct skills are measured by the nonlistening parts of the TOEFL or by the four categories of MC cloze items. It would appear that the skills associated with grammar, vocabulary, and reading comprehension are highly interrelated, as assessed by the TOEFL and the MC cloze test.