A Kinematic Study of Prosodic Structure in Articulatory and Manual Gestures: Results from a Novel Method of Data Collection.

Martha E. Tyrone,Mark K. Tiede,Jelena Krivokapić

doi:10.5334/labphon.75

Abstract

The primary goal of this work is to examine prosodic structure as expressed concurrently through articulatory and manual gestures. Specifically, we investigated the effects of phrase-level prominence (Experiment 1) and of prosodic boundaries (Experiments 2 and 3) on the kinematic properties of oral constriction and manual gestures. The hypothesis guiding this work is that prosodic structure will be similarly expressed in both modalities. To test this, we have developed a novel method of data collection that simultaneously records speech audio, vocal tract gestures (using electromagnetic articulometry) and manual gestures (using motion capture). This method allows us, for the first time, to investigate kinematic properties of body movement and vocal tract gestures simultaneously, which in turn allows us to examine the relationship between speech and body gestures with great precision. A second goal of the paper is thus to establish the validity of this method. Results from two speakers show that manual and oral gestures lengthen under prominence and at prosodic boundaries, indicating that the effects of prosodic structure extend beyond the vocal tract to include body movement.1

Highlights

The term prosody refers to the suprasegmental structure of the utterance, which encodes prominence and phrasal organization (e.g., Ladd, 2001; Jun, 2005; Byrd & Saltzman, 2003)
To examine whether there might be evidence of lengthening on a portion of the manual gesture, we further examined the duration of the PLATEAUX of the manual gestures
A further possibility is that the manual gesture is coordinated with speech in such a way that the pointing movement precedes the prominence to a large extent, so that the effect of prominence would only occur on the return movement

Summary

Introduction

The term prosody refers to the suprasegmental structure of the utterance, which encodes prominence and phrasal organization (e.g., Ladd, 2001; Jun, 2005; Byrd & Saltzman, 2003). Words can receive focus ( narrow, broad, and contrastive) or be deaccented (see Table 1 for examples), and the phonetic correlates of prominence can vary depending on the focus structure (Breen et al, 2010 for English; Mücke & Grice, 2014 for German).. A recent study of German by Mücke and Grice (2014) was the first to examine the effect of the different degrees of prominence under investigation in our study, namely, deaccented, broad, narrow, and contrastive focus. In their acoustic analysis, they found that deaccented and broad-focused stressed syllables are shorter than narrow-focused stressed syllables, which in turn are shorter than contrastive-focused stressed syllables. In a related study, Cho and Keating (2009) examined the effects of lexical (primary vs. secondary stress) and phrasal prominence (accented vs. unaccented) and found no evidence of cumulative lengthening

Objectives

Methods

Results

Conclusion