Frequency and predictability effects on the duration of content words in conversation

Abstract

This study carefully examines the proposal that word forms in conversation are shorter when they have higher probability, where this includes all factors of their context (Jurafsky et al., Frequency and the Emergence of Linguistic Structure, edited by Bybee and Hopper, 2001, pp. 229–254). Higher word frequencies have long been known to be linked to shorter durations. But it has been hard to confirm a direct relationship between the two. One confound is with predictability or conditional probability, which also affects surface form and is closely related to frequency. Another difficulty is controlling factors related both to duration and contextual probability. Linear regression (and sampling to avoid violating independence assumptions) was used to control for phonological form, speech rate, prosodic prominence, previous uses, and neighboring disfluencies. Both greater word frequencies and greater conditional probabilities given the following word have a strong shortening effect on durations of content words. The effect of other probability measures considered did not reach significance. The study is based on a sample of about 1000 content words from a portion of the Switchboard corpus, phonetically transcribed at ICSI, Berkeley, and coded for intonation by P. Taylor, M. Ostendorf, and S. Shattuck-Hufnagel. [Work supported by NSF.]