Chapter 5 - Computational models of multisensory integration

Abstract

Normative Bayesian models of perceptual inference define how observers should combine uncertain information across multiple sensory channels and prior knowledge to obtain the most reliable percept of our environment. In this review, we first introduce forced fusion models that describe how observers integrate sensory signals along with prior knowledge approximately weighted in proportion to their relative reliabilities. Yet, these models describe only the special case of mandatory integration that applies when signals come necessarily from a common source; they cannot model situations where signals can come from common or independent sources. In these more naturalistic situations, observers should integrate signals from common sources but segregate those from independent sources. Recent hierarchical models of Bayesian causal inference solve this so-called causal inference problem by explicitly modeling the world's causal structure (i.e., common or independent sources). To account for observers' uncertainty about the world's causal structure, a final Bayesian causal inference estimate is then obtained by combining the estimates under the assumptions of common or independent sources according to various decision functions (e.g., model averaging). Growing psychophysical and neuroimaging evidence suggests that human observers arbitrate between sensory integration and segregation in line with the principles of Bayesian causal inference.