The relation of multiple-choice cloze items to the Test of English as a Foreign Language

Abstract

Four categories of multiple-choice (MC) cloze items were examined in relation to the TOEFL. The object was to assess the factor structure of the TOEFL and the potential of distinguishing MC cloze items aimed at reading comprehension (defined in terms of textual constraints ranging across clauses) as contrasted with knowledge of grammar (short-range surface syntax and morphology) or vocabulary. Since it is impossible in principle to distinguish such skills absolutely at any given point in a text, a compromise was to identify items whose difficulty seemed to be based primarily on one level of processing and secondarily on another. The pivotal category was reading comprehension. In all, 50 MC cloze items over three texts were used in four subsets: ones for which reading comprehension seemed to be the primary source of difficulty, and (1) grammar secondary or (2) vocabulary secondary (nine and 14 items respec tively) ; and ones for which either (3) grammar or (4) vocabulary was the main source of difficulty and reading comprehension secondary (15 and 12 items). Results were analysed separately for each of nine language groups, with a total of 11,290 subjects in all. Factor analysis of the TOEFL suggested two factors related to (a) the Listening Comprehension section, and (b) the nonlistening subsections. The data did not clearly reveal the expected differential relations between the MC cloze categories and subsections of the TOEFL, though tendencies were apparent and analyses on the whole revealed substantial reliability and validity for the MC cloze items.