Pragmatic evaluations of automated linguistic creativity

Lorenzo Gatti,Oliviero Stock,Carlo Strapparava,Gözde Özbal

doi:10.1007/s10579-021-09560-6

Abstract

The optimal innovation hypothesis (OIH) offers a good aesthetic and cognitive reference for the kind of linguistic creativity where minimal variations can have a strong effect on the audience and realize an overall intended goal. The same approach can be the basis for creative systems. The question is how to concretely evaluate not only their quality, but also their pragmatic effect. This paper describes the original evaluations of two systems based on the OIH, one automatically yielding witty headlines for incoming news, the other producing song parodies, varying the song lyrics to evoke a given concept. The goal is to bring attention to the importance of evaluating the pragmatic potential of creative systems, in addition to the quality of their output, and to demonstrate how such evaluations can be done.

Highlights

Computational creativity is coming of age as the sub-field of AI research concerned with the development of programs that generate creative output and, by extension, that can show intelligent creative behavior (Colton et al, 2009)
Creative machines should support the achievement of these pragmatic effects, in addition to presenting something aesthetically pleasing
The former are the most well-studied, and they can benefit from decades of research in natural language generation

Summary

Introduction

Computational creativity is coming of age as the sub-field of AI research concerned with the development of programs that generate creative output and, by extension, that can show intelligent creative behavior (Colton et al, 2009). The real issues are the aesthetic quality of the output (Machado & Cardoso, 1998), and— at least for systems concerned with potential applied settings—pragmatics, i.e. the goal the system intends to achieve with the produced linguistic material. Far from proposing a general theory, the OIH is useful as an aesthetic and cognitive reference for the kind of linguistic creativity in which subtle variations can have a strong effect on the audience (as it happens, for example, when the lyrics of a song are repurposed to promote a concept or advertise a product)

Objectives

Methods

Results

Conclusion