How Optimal Depth Cue Integration Depends on the Task

Abstract

Bayesian parameter estimation can be used to generate statistically optimal solutions to the problem of cue integration. However, the complexity and dimensionality of these solutions is frequently prohibitive. In this paper, we show how the complexity and performance characteristics of the optimal estimator for a task depend strongly on the detailed formulation of the task, including the choice of representation for the scene variables. In particular, some representations lead to simpler inference algorithms than others. We illustrate the problem of cue integration for the perception of depth from two highly disparate cues, cast shadow position and image size, and show how the complexity and performance of the depth estimators depends on the specific representation (choice) of depth parameter. From the analysis we predict human performance on a simple depth discrimination task from the optimal cue integration in each depth representation. We find that the cue-integration strategy used by human subjects can be described as near-optimal using a particular choice of depth representation.