Reconsidering the development of the discourse completion test in interlanguage pragmatics

Abstract

A survey of the field of Interlanguage Pragmatics (ILP) shows that the Discourse Completion Test (DCT)2, also referred to as a ‘discourse completion task’ or a ‘production questionnaire’, has been the most frequently used instrument to evaluate second/foreign language learners’ ability to perform speech acts in a target language, despite the harsh criticism leveled against its low construct validity and its failure to represent the features of authentic discourse. Interestingly, focusing on the statement of objectives of a number of ILP studies using DCTs, one can notice that such studies rarely refer to the DCT as a language test. In addition, an overview of the DCT design process as described in several ILP studies shows that ever since its adaptation for the study of pragmatic ability (Blum-Kulka, 1982), there has been a tendency to use or adapt one of the existing DCT versions used in previous studies based on the argument of comparability of results. While a number of ILP researchers tried to improve the design of the DCT by the inclusion of rejoinders or by enhancing the prompt material (e.g. Billmyer and Varghese, 2000), few attempts have been made to reconsider the DCT development process. McNamara and Roever (2006: 253) urge for the need for “more research on testing of sociopragmatic knowledge and design of discourse completion tests for testing purposes.”The present paper starts with an overview of the literature about DCTs with special reference to the cognitive validity of the instrument and to previous studies dealing with DCT structure and content. Then, with reference to research in the fields of language testing and psychometrics, it shows that, whether used for research or instructional purposes, the DCT shares several qualities with language tests. As such, it is argued that the DCT should be treated as a language test and not as a questionnaire and should, thus, undergo a rigorous developmental process. Based on recent models of language test construction, the paper concludes with an overview of the stages of DCT development.