Temporal modulations in speech and music

Nai Ding,Aniruddh D Patel,Lin Chen,Henry Butler,Cheng Luo,David Poeppel

doi:10.1016/j.neubiorev.2017.02.011

Abstract

Speech and music have structured rhythms. Here we discuss a major acoustic correlate of spoken and musical rhythms, the slow (0.25–32Hz) temporal modulations in sound intensity and compare the modulation properties of speech and music. We analyze these modulations using over 25h of speech and over 39h of recordings of Western music. We show that the speech modulation spectrum is highly consistent across 9 languages (including languages with typologically different rhythmic characteristics). A different, but similarly consistent modulation spectrum is observed for music, including classical music played by single instruments of different types, symphonic, jazz, and rock. The temporal modulations of speech and music show broad but well-separated peaks around 5 and 2Hz, respectively. These acoustically dominant time scales may be intrinsic features of speech and music, a possibility which should be investigated using more culturally diverse samples in each domain. Distinct modulation timescales for speech and music could facilitate their perceptual analysis and its neural processing.

Highlights

Rhythmic structure is a fundamental feature of both speech and music
We investigate whether single-instrument Western classical music, as a coherent perceptual category, shows a consistent modulation spectrum or whether the modulation spectra depend on the instruments being played
The discourse-level speech recordings were cut into 6-second duration frames and the modulation spectrum averaged over all frames

Summary

Introduction

Rhythmic structure is a fundamental feature of both speech and music. Both domains involve sequences of events (such as syllables, notes, or drum sounds) which have systematic patterns of timing, accent, and grouping [1]. A primary acoustic correlate of perceived rhythm is the slow temporal modulation structure of sound, i.e. how sound intensity fluctuates over time (Fig. 1). Slow temporal modulations are related to the onsets and offsets of notes (or runs of notes in quick succession), which support perceptual phenomena such as beat, meter, and grouping [1, 6,7,8,9,10,11,12]. A number of studies have investigated the neural activation patterns associated with the temporal modulations in the human brain and assessed their relevance to speech and music perception [13,14,15,16,17,18,19,20]

Methods

Results

Conclusion