Depth Estimation Under Motion With Single Pair Rolling Shutter Stereo Images

Abstract

Many methods have been proposed to process stereo matching for rolling shutter image pairs, they treat all pixels from an image pair in an identical way and require additional estimation approaches to estimate motion states of the camera. However, pixels from a rolling shutter image pair naturally have diverse baseline lengths, and motion estimation methods are unstable for one instantaneous image pair input. In this letter, we present a rolling shutter stereo depth estimation pipeline, which can robustly estimate motion states and depth maps by alternating the estimation of the depth maps and refining the motion states from coarse image levels to fine image levels. What is more, we design a novel cost volume building method for rolling shutter image pairs, which adapts depth candidates to the change of baseline lengths for all pixels. We further demonstrate the usability of the proposed method by constructing a new platform, building an outdoor evaluation dataset, and comparing it with baseline methods.