The gap between communicative ability measurements: General-purpose English speaking tests and linguistic laypersons’ judgments

Abstract

The assessment criteria for general-purpose speaking tests are normally produced from test developers’ intuition or communicative competence models. Therefore, the ideal second language (L2) communication in general-purpose speaking tests reflects language specialists’ perspectives. However, neglect of the views of non- language specialists (i.e., linguistic laypersons) on communication is problematic since these laypersons are interlocutors in many real- world situations. This study (a) investigated whether L2 speakers’ results on general-purpose speaking tests align with linguistic laypersons’ judgments of L2 communicative ability and (b) explored performance features that affect these judgments. Twenty-six post- graduate students of non-linguistic disciplines rated 13 speakers’ communicative ability on general-purpose speaking tests and provided verbal explanations of the performance features affecting their ratings. Their ratings were compared with the speakers’ test results, and the features that determined their ratings were examined. Although these ratings were not completely different from the test results, some speakers’ test results did not align with their ratings. The linguistic laypersons’ judgments were affected not only by features that the general-proficiency tests assessed but by other factors as well. The findings of this study will deepen our understanding of real- world interlocutors’ views on communication and contribute to the development of authentic criteria for general-purpose speaking tests.