An enduring lens for a continuing problem: Close analysis of conceptually scored cloze items

John M O’toole ,Junyu Cheng,Gjyn O’toole

doi:10.1186/s40468-015-0015-3

John M O’toole , Junyu Cheng + Show 1 more

Open Access

https://doi.org/10.1186/s40468-015-0015-3

Copy DOI

Abstract

Abstract This paper provides an example of close analysis of student attempts to fill the gaps that comprise a cloze test. Such analysis may assist diagnosis of the difficulties that groups of students experience with specific text in particular contexts and illustrate the impact of those difficulties. The paper briefly touches on the international, academic role of English, outlines its contemporary role in universities in the People’s Republic of China and then describes a small quantitative study of student difficulty with specialist English that illustrates the impact of those difficulties on text accessibility. One hundred and fourteen Chinese undergraduate students completed a 50 item cloze test which was conceptually scored before close analysis of patterns in clear student error. The impact of the pattern of difficulty that emerged from the Chinese results is illustrated in a number of ways. The rank and proportional difficulty of the identified language features is presented in numerical form and compared with that emerging from previous use of the test in secondary school contexts. The proportional difficulty is then used to guide insertion of words from an unfamiliar language into the base text to provide an indication of the difficulty being experienced by the ‘average’ Chinese undergraduate in reading this passage prepared for a mid-level secondary school audience. The present case provides the opportunity of ‘proof of concept’ for extension of cloze techniques from holistic estimation of readability to identification of specific features of specialist writing styles that may cause difficulty for particular groups of readers. The results of this study suggest that such close analysis of error patterns may provide an illuminative lens on student difficulty with specific language styles within English and that, in this particular case, focus on general academic English may not be sufficient preparation for upper level discipline courses that make use of more specific styles.

Highlights

Cloze testing has formed an enduring, if somewhat controversial, element of the language teaching repertoire for more than 60 years
O’Toole et al Language Testing in Asia (2015) 5:7 to the parts and describes a use of the technique to expose the specific difficulties that a particular group of students experience with specialist text
Close analysis of the clear errors that students make in attempts to fill gaps produced by the cloze procedure has the potential to predict the level of difficulty that students may have with other such text

Summary

Introduction

Cloze testing has formed an enduring, if somewhat controversial, element of the language teaching repertoire for more than 60 years. Much of that controversy has revolved around use of the technique to provide an authentic window on text accessibility or test-taker language competence. O’Toole et al Language Testing in Asia (2015) 5:7 to the parts and describes a use of the technique to expose the specific difficulties that a particular group of students experience with specialist text. It represents an expansion of the conceptual scoring approach to interpretation of ‘clozed’ text. Instructors often view the text they set as unproblematic and they may find the results of such analysis more convincing than general exhortation

Methods

Results

Conclusion