Advancing Language Assessment with AI and ML–Leaning into AI is Inevitable, but Can Theory Keep Up?

Abstract

ABSTRACT Following the burgeoning growth of artificial intelligence (AI) and machine learning (ML) applications in language assessment in recent years, the meteoric rise of ChatGPT and its sweeping applications in almost every sector have left us in awe, scrambling to catch up by developing theories and best practices. This special issue features studies of recent AI and ML advances and thought pieces and attempts to unify our field with a collection of work towards a common set of tools, frameworks, and practices. In this editorial, I briefly review the five studies and four commentaries and discuss the key validity issues around the AI applications covered. To unpack complex validity issues for lay users, I propose accessible questions to ask when evaluating these applications. I stress the importance of developing best practices guiding ethical and responsible use of AI and improving users’ AI literacy skills. In light of users’ increasing access to AI tools in real-world communication, I raise the need for redefining the constructs of language tests to be in sync with what is happening in the real world. These new conceptions of language ability are expected to result in significant changes in task design, scoring, and test interpretation and use.