Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis

Abstract

This study investigated the effect of Chatbot-Assisted Dynamic Assessment (CA-DA) on vocabulary learning and provided insights into learner abilities drawn from its implementation. Through the use of mediating chatbots, this study implemented DA to multiple learners simultaneously and provided each learner with human-like interaction. The chatbots were created using Google’s Dialogflow. Fifty-three Korean EFL primary school learners who were confirmed to demonstrate the same range in vocabulary size participated in this study. They were randomly assigned to three groups: CA-DA, Chatbot-Assisted Non-Dynamic Assessment (CA-NDA), and a control. For two treatment sessions, the learners were asked to read texts and identify the meaning of underlined target words. The chatbots provided graduated assistance to learners in the CA-DA group and only target word definitions to learners in the CA-NDA group. The control group did not utilize chatbots. Two posttests (receptive and productive) were administered both immediately after and two weeks after the second treatment session. Interaction records between the chatbots and learners across the two treatment sessions were also collected. Posttest results showed that vocabulary gains in the CA-DA group were significantly higher than in the other groups. Analysis of interactions between the chatbots and learners for the CA-DA sessions provided detailed evidence of learner development. The findings suggest that CA-DA could not only promote vocabulary acquisition but could also offer diagnostic information about individual learners concerning vocabulary learning. This study also demonstrated the potential of chatbot technology to support language learners.