Do Artificial Intelligence Chatbots Have a Writing Style? An Investigation into the Stylistic Features of ChatGPT-4

Abstract

Even though Turnitin generates AI (Artificial Intelligence) writing detection reports, these AI reports shall not be used for punitive purposes as Turnitin AI reports accuracy is way below the 98% claimed by Turnitin, as revealed in this study. To assist professors, teachers, and content evaluation stakeholders in their strive to identify AI-generated material, this study examines the stylistic features of case study, business correspondence, and academic writing ChatGPT-4 generated responses by exploring sentence length, paragraph structure, word choice, mood, tense, voice, pronouns, keywords density, lexical density, lexical diversity, and reading ease. The study revealed that ChatGPT-4 case study generated responses are produced in paragraphs of 2 to 3 sentences of 16 to 18 words each. The sentences are mainly formed in imperative mood. The use of the second-person pronoun ‘you’ and the second-person possessive determiner ‘your’ is prevalent. Keywords and lexical density are relatively low, lexical diversity is average, and the reading ease is relatively high. The study also found that ChatGPT-4 business correspondence responses are generated in paragraphs of 2 to 3 sentences of 16 to 20 words each. The sentences are mainly generated in declarative mood thru simple present tense in active voice using third-person singular pronouns. Technical words and abbreviations are used without outlining what they stand for. The keywords density, lexical density, and lexical diversity are high and the reading ease is low. The study also revealed that ChatGPT-4 academic writing generated responses are provided in paragraphs of 3 to 4 sentences of 16 to 19 words each. The sentences are mainly generated in declarative mood using active voice, agentless passive in times, with diverse present tenses. Keywords and lexical densities are high and the lexical diversity is low, which makes the reading ease average difficulty, except for the undefined abbreviations. Noticeably, ChatGPT-4 supports the transgender movement by intentionally using the third-person plural pronoun ‘they’ to refer to a singular.