ChatGPT-3.5 as writing assistance in students’ essays

Abstract

ChatGPT-3.5, an AI language model capable of text generation, translation, summarization, and question-answering, has recently been released for public use. Studies have shown it can generate abstracts, research papers, and dissertations, and create quality essays on different topics. This led to ethical issues in using ChatGPT in academic writing, AI authorship, and evaluating students’ essays. However, it is still unknown how ChatGPT performs in students’ environments as a writing assistant tool and if it enhances students’ essay-writing performance. In the present study, we examined students’ essay-writing performances with or without ChatGPT as an essay-writing assistance tool. The average essay grade was C for both control (traditional essay-writing, n = 9) and experimental (ChatGPT-assisted essay-writing, n = 9) groups. None of the predictors affected essay scores: group, writing duration, study module, and GPA. The text unauthenticity was slightly higher in the experimental group, but the similarity among essays was generally low in the overall sample. In the experimental group, the AI classifier recognized more potential AI-generated texts. Our results demonstrate that the ChatGPT group did not perform better in either of the indicators; the students did not deliver higher quality content, did not write faster, nor had a higher degree of authentic text. We anticipate that these results can relieve some concerns about this tool’s usage in academic writing. ChatGPT-assisted writing could depend on the previous knowledge and skills of the user, which might, in certain instances, lead to confusion in inexperienced users and result in poorer essay writing performance.