The use of the Criterion automated assessment system in teaching students of language specialties to write essays in a foreign language: the results of an experimental test

Abstract

Importance. Modern artificial intelligence (AI) tools are able to take on some of a foreign language teacher’s functions, thereby optimizing the learning process. One of these is the teacher’s function of checking and evaluating students’ written papers (essays). There are works in the scientific literature describing step-by-step methods of teaching students to write essays in a foreign language, including individual extracurricular practice of students with an AI tool during the completion of the essay’s draft version. Based on receiving evaluative feedback from the AI tool, students can refine their essays in linguistic, structural and substantive aspects. The Criterion automated assessment system for written papers is one of the modern AI tools that can be used in teaching. However, checking the effectiveness of this system is not the subject of a separate study. The purpose of the study is to clarify the stages of teaching students of language specialties to write essays in a foreign language using evaluative feedback from the Criterion automated assessment system and to test the effectiveness of this technique during experimental training.Materials and Methods. The study participants are 2nd year students of the specialty 45.05.01 “Translation and Translation Studies” of the Lipetsk State Technical University (control group = 25 and experimental group = 25). The students of the control group studied according to the traditional essay teaching methodology, excluding the use of AI tools, and the experimental group participants studied according to an innovative methodology, including individual extracurricular practice of students with the Criterion system in order to receive evaluative feedback for the completion of the essay. The following aspects of control are: a) the lexical side of written speech; b) the grammatical side of written speech; c) the organizational structure of the work; d) the content of the work.Results and Discussion. It is proved that the innovative method of teaching students to write essays, based on the integration of the Criterion automated essay assessment system into the learning process, is more effective than the traditional method in mastering students a) the organizational structure of the essay (t = 2.82 at p = 0.004) and b) the content side of creative work (t = 3.05 at p = 0.002). However, statistical analysis of the results do not confirm the advantage of the innovative technique in the formation of lexical (t = 1.44 at p = 0.08) and grammatical (t = 1.36 at p = 0.09) speech skills of students of a language university.Conclusion. The novelty of the research consists in developing the stages of teaching students to write an essay based on practice with the Criterion automated control system in order to receive evaluative feedback for the completion of the essay. The results obtained can be used in the methodology of teaching secondary school students and university students in teaching writing.