The effectiveness of artificial intelligence on learning achievement and learning perception: A meta-analysis

Abstract

ABSTRACT Recently, artificial intelligence (AI) technologies have been widely used in the field of education, and artificial intelligence in education (AIEd) has gained increasing attention. However, no quantitative meta-analysis has been conducted on the overall effectiveness of AI on learning achievement and learning perception. To close this research gap, this study conducted a comprehensive meta-analysis of the effects of AI on learning achievement and learning perception. The present meta-analysis synthesized 24 articles with a total of 2908 participants from 2001 to 2020. The findings reveal that AI had a high effect size on learning achievement and a small effect size on learning perception. The effect sizes of 13 moderator variables were analyzed, including sample levels, sample size, learning domains, learning methods, research design, research settings, intervention duration, types of organization for treatment, role of AI, areas of AI application, AI software, AI hardware, and AI technologies. It was found that sample size, sample level, learning domains, types of organization, roles of AI, and hardware significantly moderated the effectiveness of AI. The results and the implications for educators and practitioners are discussed in depth.