Interplay of Visual and Acoustic Cues of Irony Perception: A Case Study of Actor’s Speech

Abstract

AbstractThis paper deals with the interaction of visual and acoustic cues of irony, observed in the speech of Russian professional actors. We selected ironic and non-ironic utterances from modern films and series taking into account narrow and broad context, lexical and semantic markers. Then we extracted the target utterances from the context eliminating any markers of irony. The participants of the perceptual experiments could rely only on the visual and acoustic (prosodic) cues. In the first experiment we suggested to the participants mute video files containing the target ironic and non-ironic utterances. The second experiment was conducted with the audio files only of the same utterances extracted from the films. In the third experiment video and audio were suggested simultaneously, as in the natural situation of film watching, but still without any context or lexical marker. Segment duration, pitch movement, gestures and mimics, as well as their synchrony in the well-recognized target utterances were analyzed. The results of the experiments demonstrated that the visual cues were more important for irony perception than the audio signal. Yet, some video stimuli that had low recognition of irony were better recognized in the experiment with audio. It led us to suppose that actors use in various proportion visual and acoustic cues to express irony in speech. The results of this research can have practical application in both speech recognition and speech generation used in artificial intelligence systems, as well as in the forensic phonetics and second language acquisition.KeywordsVerbal ironyAntiphrasisActors’ SpeechAcoustic cuesGesturesPhoneticsPitch movement