Perceptual organization of speech in one and several modalities: common functions, common resources

Abstract

In order to understand speech, the perceiver meets two challenges: (1) to find a speech signal within ongoing sensory activity, and (2) to project its properties into linguistic phonetic attributes. These functions have customarily been designated as perceptual organization and perceptual analysis. The case of multimodal perceptual organization is revealing to consider because the perceiver finds sensory ingredients spanning modalities. Contemporary accounts offer alternative conceptualizations of these functions based largely on the study of single modalities. A Gestalt derived account hypothesizes that perceptual organization precedes analysis, grouping sensory elements into perceptual streams by a variety of similarity criteria. An account deriving from probabilistic functionalism describes analysis occurring within modalities preceding a stage of organization that binds the derived features. These alternatives and their hybrids appear implausible on empirical and theoretical grounds for accommodating multimodal perceptual organization. Additionally, our studies using sinewave replicas of utterances reveal that the customary models are untenable accounts of unimodal no less than multimodal perceptual organization. A third way, justified by our results, describes auditory perceptual organization of sinewave sentences as a specific instance of the general susceptibility to coherent sensory variation. This account potentially allows a single description of uni- and multimodal perceptual organization.