Aspects of Modality in Audio-Visual Processes

Abstract

The relevance of the sensory modality in which information processing is initiated is not an issue that has received much attention in cognitive psychology. Much in the spirit of information processing approaches to cognition, psychological content is conceived as abstract. Mathematical approaches to information processing have likewise not made much room for conceptualizing the specific contribution of the sensory input modalities. Bimodal speech processing is among the best explored cases, yet it is not clear that investigations of speech processing in another modality than the canonical auditory one, i.e., lipreading, have so far departed from the mainstream information processing frameworks. A more recent theoretical proposal which casts information processing as being a matter of separate modules or of separate content domains leaves room for modality specificity. Against this background two apparently related phenomena are discussed, McGurk effects and ventriloquism. Starting from models that neither postulate modality specific processes nor modality specific representations, the paper asks whether a distinction is needed between content driven and contiguity driven processes.