Auditory perception with slowly-varying amplitude and frequency modulations

Abstract

Amplitude modulation (AM) and frequency modulation (FM) are abundant in natural stimuli, including speech, music, and animal communication sounds. Although amplitude and frequency modulations have been extensively studied physiologically and psychophysically (e.g., Riesz 1928; Grinnell 1963; Suga 1964; Gordon and O'Neill 1998), it is still unclear whether and how the auditory system extracts and uses these cues. For example, there is an ongoing debate on whether amplitude modulation is processed via envelope extraction in the temporal domain (Viemeister 1979) or a second filtering process in the spectral domain (Dau, Kollmeier, and Kohlrausch 1997). It is also unsettled whether frequency modulation is processed independently of amplitude modulation via specialized "FM channels" in the auditory system (Kay and Matthews 1972; Regan and Tansley 1979; Moore and Sek 1996), by a common mechanism (Moore and Sek 1995; Saberi and Hafter 1995). Regardless of the underlying processing mechanisms of amplitude and frequency modulations, both cues have been shown to contribute to speech recognition in quiet laboratory conditions. Remez et al. (1981; 1990) demonstrated that speech could be reliably recognized with three sinusoids that tracked the formant movement, namely frequency modulation. On the other hand, Shannon et al. (1995) demonstrated that speech could also be reliably recognized with primarily temporal envelope cues, namely amplitude modulation. These results have been traditionally taken as an indication of the redundancy of multiple cues in natural speech sounds. Motivated by how to deliver the fine structure cue to cochlear implants, recent studies have implicated possible independent contributions of amplitude and frequency modulations to auditory perception (e.g., Smith, Delgutte, and Oxenham 2002). We have developed a signal processing strategy that extracts slowly-varying amplitude and frequency modulations from the traditionally defined temporal envelope and fine structure cues, i.e., Hilbert transform. This novel strategy also