Modality matters: Testing bilingual irony comprehension in the textual, auditory, and audio-visual modality

Abstract

Prior irony research has primarily explored how people comprehend written irony in their first language (L1). Nowadays, however, people often communicate in their second (L2) language, and additionally, irony may be communicated multimodally, i.e., in the textual (reading), auditory (listening), or audio-visual (watching) modality. Hence, communicators need to be able to understand ironic intent in all these modalities, and in both linguistic systems they use (L1, L2). Yet, little is known about how people understand irony when they listen to or watch ironic interactions. Therefore, here we tested irony comprehension in participants' L1: Polish (Study 1) and L2: English (Study 2) in the auditory, textual, and audio-visual modality. We adopted excerpts from a popular TV series, House M.D., and presented them to participants as a reading, listening, or watching task in their L1 or L2. The results showed that irony was faster and easier to process compared to non-irony, irrespectively of the language of operation. Furthermore, irony comprehension was more efficient (i.e., more accurate but slower) when the message was presented in modalities that convey multiple cues signaling irony (i.e., auditory and audio-visual) relative to the textual modality. Finally, participants were faster to recognize irony in the audio-visual and auditory modality relative to the textual modality.