The effectiveness of artificial intelligence on English language learning achievement

Abstract

Over the past few decades, artificial intelligence (AI) has undergone exponential growth and has been overwhelmingly permeated in the educational field, including English language education. Many individual studies have paid close attention to probing the effect of AI on learning. However, no quantitative meta-analysis has been conducted on the overall effectiveness of AI on English language learning achievement. Hence, to fill the research gap and strengthen the statistical power, this article aims to carry out a meta-analysis for examining the effectiveness of AI on English learning outcomes. A total of 40 empirical studies with 3290 participants across ten countries filtered from five academic electronic databases, yielding 55 effect sizes. Via Comprehensive Meta-Analysis (CMA) software, the results found that AI had a high effect size (g = 0.812) on English language learning achievement, indicating that students who integrated AI significantly outperformed their counterparts who followed traditional pedagogy in English achievement. Additionally, the moderating effect of ten categorical variables (i.e. nation development level, whether undergoing the COVID-19 pandemic, sample size, learning phases, students' majors, sub-fields of English learning, AI applications, intervention duration, research design, research settings) was examined. It found that sample size, learning phases and students’ majors significantly moderated the effectiveness of AI. Confronting the results, potential reasons behind them are discussed, and practical and research implications are proposed.