Video, talk and text: How do parties communicate coherently across modalities in live videostreams?

Abstract

This study explores how participants co-accomplish coherence in multi-modal conversation. We observed online watch parties, with an interest in the intertwining of text, talk and video. The data comes from two different communities on the live-streaming platform Twitch.tv. Each community consists of one live-streamer and their viewership. The live-streamers broadcast an audio-visual artifact, while simultaneously communicating with their viewers in real-time. The two were highly interactive, despite one (the streamer) speaking and reading, and the other (the chatters) typing and listening. The crisscross of modalities introduces challenges for the management and intelligibility of conversation. One coherence strategy involved a 4-stage process whereby streamers redirected viewers’ attention, and then initiated a collaborative activity. This 4–stage sequence illustrates a predictable structure that is potentially applicable to other digital & multimodal environments. Methodological challenges of digitally-mediated interaction are addressed, such as transcription of voice-and-text cross-modal conversation.