Multimodal perception of prominence in spontaneous speech: A methodological proposal using mixed models and AIC

Abstract

Research on prominence perception has made use of animated agents and controlled speech in experimental settings, but these methodologies have disregarded some aspects of the acoustic and visual correlates of prominence. To overcome these limitations we propose a new methodological approach using spontaneous speech data. For this, we created a small database with extracts from a television talent show and neutralised the prominence-lending properties of the acoustic cues of prominence in the speech signal. In our pilot study twelve naïve listeners marked words for binary prominence (prominent vs. non-prominent) in two modalities, i.e. audio-only and audiovisual, under three conditions involving neutralisation of (a) fundamental frequency, (b) intensity, and (c) both fundamental frequency and intensity. Additionally, the marks of two trained listeners served as control condition. Different generalised linear mixed models were estimated and compared using the Akaike Information Criterion (AIC). The most parsimonious model was then examined using traditional null-hypothesis testing in order to provisionally establish the effects of our independent variables on prominence marking. We argue that spontaneous speech can be successfully applied to the study of the multimodal perception of prominence.