Investigating test-takers’ strategy use in task-based L2 pragmatic speaking assessment

Abstract

Abstract This study explicates cognitive validity of task-based L2 pragmatic speaking assessment by examining reported strategy use of test takers at varying performance levels across different task types. Thirty university-level ESL learners completed four pragmatic speaking tasks that differ in the formality of pragmatic actions. Two trained raters scored the task-based pragmatic performances using analytical rating criteria and displayed a satisfactory level of consistency and accuracy in scoring the performances. The test-takers’ retrospective reports were transcribed and analyzed to develop a valid coding scheme that consists of cognitive, metacognitive, and pragmatic strategies. An association between the test-takers’ pragmatic performances scored by the trained raters and their reported strategy use was examined. The higher-ability test takers utilized diverse strategies more frequently, ranging from varied pragmatic strategies to strategies specifically related to managing task demands, compared to the lower-ability test takers. Further, the test takers utilized distinct types of strategies appropriate to handling unique pragmatic task situations and complexities involved in each pragmatic assessment task. These findings explain how the test takers cognitively interacted with the assessment tasks and what strategies potentially led to successful pragmatic performances. The implications of examining pragmatic strategy use were discussed in terms of advancing practices of teaching and assessing L2 pragmatics.