Investigating the Effect of an Interactive Educational Chatbot on Reading Comprehension Skills

Abstract

This study aimed to investigate the implementation of an interactive chatbot as a facilitator to assess its effects on reading skills. To this end, 60 Omani intermediate English as a Foreign Language (EFL) learners from a higher education institute were selected as the population and divided into two experimental and control groups with an equal number of learners. Several research-based reading tests have been developed, validated, and piloted to ensure reliability, as measured by Cronbach’s alpha. The test results for the pre-, post-, and delayed post-tests were 0.889, 0.767, and 0.850, which ensured reliability. Although both groups received instructions and practiced reading skills based on the department’s curriculum and delivery plan, an interactive WhatsApp bot was designed to provide additional instructions and exercises for the experimental group for three weeks. A Kolmogorov-Smirnov test was conducted to assess data normality and determine whether parametric- or nonparametric-tests should be used. The results indicated non-normality in all data sets (p < .05) except for the pre-test data of the experimental group (p > .05); Consequently, the Wilcoxon test was employed to analyze results within the groups, while the Mann-Whitney U test was utilized to compare the groups. The interpretation of the test within the control group revealed that learners performed significantly differently in the post- and delayed post-tests (Z = -4.80, -2.95 < .05). Similarly, the results of learners in the experimental group showed that they performed significantly better in the post- and delayed post-tests than in the pre-test (Z = -4.79, -4.80 < .05). Finally, the comparison of both groups based on the Mann-Whitney U test revealed that the experimental group outperformed the control group in the post-test and delayed post-test (U = .00, p < .05). Therefore, it could be concluded that chatbots could be beneficial within the learning process, particularly for improving reading skills. These results are promising for teachers and students who utilize technological devices in their learning and teaching practices.