An audiovisual investigation of the loudness/effort effect for speech and nonspeech events

Abstract

There is some evidence that loudness judgments of speech are more closely related to the degree of vocal effort induced in speech production than to the speech signal's surface acoustic properties, such as intensity [P. Ladefoged and N. P. McKinney, J. Acoust. Soc. Am. 35, 454–460 (1963)]. Other researchers have claimed that speech loudness can be rationalized by considering simply the acoustic complexity of the signal [R. D. Glave and A. C. M. Reitveld, J. Acoust. Soc. Am. 58, 875–879 (1975)]. Since vocal effort can be specified optically as well as acoustically, a study to test the effort‐loudness hypothesis was conducted that used conflicting audiovisual presentations of a speaker producing CVs with different efforts. The prediction was made that if loudness judgments are based on effort perception rather than on simple acoustic parameters, then judgments should be affected by visual as well as auditory information. It is shown that loudness judgments are significantly affected by visual information even when subjects (1) are instructed to base their judgments on only what they hear and (2) cannot detect a discrepancy between the audio and visual components. Moreover, the same results are shown for a nonspeech “clapping” event attesting to the generality of the loudness‐effort effect previously thought to be special to speech