The role of perspective information in the recovery of 3D structure-from-motion

Abstract

When investigating the recovery of three-dimensional structure-from-motion (SFM), vision scientists often assume that scaled-orthographic projection, which removes effects due to depth variations across the object, is an adequate approximation to full perspective projection. This is so even though SFM judgements can, in principle, be improved by exploiting perspective projection of scenes on to the retina. In an experiment, pairs of rotating hinged planes (open books) were simulated on a computer monitor, under either perspective or orthographic projection, and human observers were asked to indicate which they perceived had the larger dihedral angle. For small displays (4.6×6.0°) discrimination thresholds were found to be similar under the two conditions, but diverged for all larger stimuli. In particular, as stimulus size was increased, performance under orthographic projection declined and by a stimulus size of 32×41° performance was at chance for all subjects. In contrast, thresholds decreased under perspective projection as stimulus size was increased. These results show that human observers can use the information gained from perspective projection to recover SFM and that scaled-orthographic projection becomes an unacceptable approximation even at quite modest stimulus sizes. A model of SFM that incorporates measurement errors on the retinal motions accounts for performance under both projection systems, suggesting that this early noise forms the primary limitation on 3D discrimination performance.