Influence of eye position on stereo matching

Abstract

In animals with binocular depth vision, or stereopsis, the visual fields of the two eyes overlap, shrinking the overall field of view. Eye movements increase the field of view, but they also complicate the first stage of stereopsis: the search for corresponding images on the two retinas. If the eyes were stationary in the head, corresponding images would always lie on retina-fixed bands called epipolar lines. Because the eyes rotate, the epipolar lines move on the retinas. Therefore, the stereoptic system has a choice: it may monitor eye position to keep track of the epipolar lines, or it may give up on tracking epipolar lines and instead search for matches over retina-fixed regions, but in that case the search regions must be 2-D patches, large enough to encompass all possible locations of the epipolar lines in all usual eye positions. We use a new type of random-dot stereogram to show that human stereopsis uses large, retina-fixed search zones. We show that the brain somewhat reduces the size of these search zones by rotating the eyes about their lines of sight in a way that reduces the motion of the epipolar lines. These findings show the link between sensory and motor processes: by considering eye motion we can understand why the brain searches for matching images over 2-D retinal regions rather than along epipolar lines; and by considering retinal correspondence we appreciate why the eyes rotate as they do about their lines of sight.