Rolling-Shutter-stereo-aware motion estimation and image correction

Abstract

The vast majority of modern consumer-grade cameras employ the Rolling Shutter (RS) mechanism, which often produces geometrically distorted images due to inter-row delay during imaging, that needs to be properly handled. While several models and solutions have been proposed for the motion estimation problem of the monocular RS camera, the corresponding motion estimation model and algorithm have not been developed for the standard RS stereo setup. This is primarily due to the temporal-dynamic nature of the RS camera, which results in scanline-varying camera poses. This paper fills in this gap by developing two tractable RS stereo models under constant velocity motion and constant acceleration motion respectively, which only need to simply scale the optical flow vector. We further propose two corresponding RS stereo motion estimation algorithms to accurately compute the relative pose of the RS stereo camera between two consecutive frames. As a result of utilizing these RS stereo models associated with the parallel visual rays, our proposed methods offer better disambiguation of the translation and rotation, hence leading to significantly improved performance compared with the Global Shutter (GS) stereo model. Furthermore, we develop an RS stereo image correction method to remove RS distortions and recover high-quality GS images by conducting motion compensation. Experimental results on both simulated data and real RS stereo images show the superiority of our proposed models and methods.