Study of temporal modified-RANSAC based method for the extraction and 3D shape reconstruction of moving objects from dynamic stereo images and for estimating the camera pose

Abstract

This paper proposes a Temporal Modified-RANSAC based method that can discriminate each moving object from the still background in the stereo video sequences acquired by moving stereo cameras, can compute the stereo cameras' egomotion, and can reconstruct the structure of each moving object and the background. We compute optical flows from the depth map and results of tracking feature points. We define 3D flow as a set of connected pixels whose optical flows have a common rotation matrix and translation vector. Our Temporal Modified-RANSAC segments the detected optical flows into flow regions and computes the rotation matrix and translation vector for each flow region. As opposed to the conventional Modified-RANSAC for only two frames, The Temporal Modified- RANSAC can handle temporal images with arbitrary length by performing the Modified-RANSAC to the set of a 3D flow region that classified in the latest frame and new optical flows detected in the current frame iteratively. Finally, the points computed from the depth map in all the frames are registered using each flow region's matrix to the initial positions in the initial frame so that the structures of the moving objects and still background are reconstructed. Experiments using multiple moving objects and real stereo sequences demonstrate promising results of our proposed method.