Perception and Evaluation of Text from an Interview with a Real Person and an Interview with an Artificial Intelligence

Abstract

The article presents a study of perception, understanding and evaluation of two interviews, the first one is performed with a real person (from an expert source), and the second is generated by the ChatGPT artificial intelligence (AI). The study covers 84 participants between the ages of 20 and 23, students in “Psychology”, “Bulgarian Philology”, “Slavic Philology”, and “Russian Philology” from the Sofia University “St. Kliment Ohridski”. The research design is of the “between-group” type (in this case, two-group). The first experimental group consisted of 40 subjects evaluates the interview with a real celebrity. The second experimental group consisted of 44 subjects evaluates the interview taken by an artificial intelligence. Both interviews consisted of ten identical questions and ten corresponding answers. Each of the ten answers is evaluated by the participants using a Likert Scale composed of 5-point score scales, each of which is “continuous” with 11 possible answer positions (from “0” to “10”), as follows: [1] inadequate / fully adequate; [2] non-specific (general) / completely specific; [3] non-exhaustive / fully exhaustive; [4] questionable / completely realistic; and [5] unsatisfactory / 10-completely satisfactory. A statistical analysis was made by a two-group test (Two Sample T-Test Unequal Variance). It was established that: consolidated answers (series of maximum ratings) when working with the “artificial” interview; in two of the ten questions asked, a statistically significant difference was observed in favor of the artificial text; in one of the questions, a statistically significant difference is observed in favor of the real interview; no difference in emotional impact and interest in summary assessment of the both pieces of text.