Tailored cloze: improved with classical item analysis techniques

Abstract

This study investigates the possibility that the reliability and validity of a cloze procedure can be improved by applying traditional item analysis and selection techniques. Students in a single level (n = 89) at the Guangzhou English Language Centre (People's Republic of China) were chosen for this study because previous experience and research had indicated that cloze tests gene rally produce low reliability and validity coefficients in samples wherein the range of ESL proficiencies is limited. This turned out to be the case when a 399-word every seventh word deletion cloze passage with 50 items was administered to this group. The study was designed such that 250 of the poten tial items in this passage could be piloted and item analyzed. The results of the item analysis were used to select the 'best' items on the basis of item facility and discrimination indices. The resulting 50 item 'tailored cloze' was then read ministered to the same group and the results for the original version of the cloze test were compared to those for the tailored version. These results indicate statistically significant and meaningful improvements in test quality due to the revision. The dispersion of scores, reliability and validity were all substantially improved by the item analysis and selection processes. The article concludes with discussion of the implications of these findings.