Aesthetic multimodality of speech and gesture: Towards its functional framework

Abstract

The study develops a functional multimodal approach to speech and gesture behavior to explore aestheticism in more and less staged discourse of cinema and interview. We hypothesize that cinema and interview employ the same communicative functions; however, these functions constitute different frameworks which contribute to the higher aesthetic potential of cinema. This approach allows to study the aesthetic via communicative functions frameworks in multimodal discourse. To establish the function frameworks in cinema and interview, we apply a contrastive functional analysis of speech and gesture in the highly ranked actors’ argumentative and descriptive monologues. With the help of variance and regression analysis, we explore the distribution of pragmatic and discourse-structuring functions (with sub-functions) in speech as contingent on pragmatic, deictic, representational and adaptive functions of gestures. The study confirms that cinematic discourse exploits fewer deictic, representational and adaptive gesture functions, whereas pragmatic gesture functions (especially emphatic ones) appear more frequently and are contingent on several pragmatic and discourse-structuring functions of argumentative and descriptive speech. Interview function frameworks display lower predictability, which shows higher spontaneity of gestures; however, there are specific gestures typical of interview (self-adaptors) which may serve as indicators of pragmatic functions of argumentation. The study also manifests individual variations within the function frameworks. Overall, cinema and interview display variance in replication and regularity of speech and gesture functions, which presumably helps create higher and lower aesthetic effects.