Effects of texture addition on optical flow performance in images with poor texture

Abstract

This paper investigates the effects of adding texture to images with poorly-textured regions on optical flow performance, namely the accuracy of foreground boundary detection and computation time. Despite significant improvements in optical flow computations, poor texture still remains a challenge to even the most accurate methods. Accordingly, we explored the effects of simple modification of images, rather than the algorithms. To localize and add texture to poorly-textured regions in the background, which induce the propagation of foreground optical flow, we first perform a texture segmentation using Laws' masks and generate a texture map. Next, using a binary frame difference, we constrain the poorly-textured regions to those with negligible motion. Finally, we calculate the optical flow for the modified images with added texture using the best optical flow methods available. It is shown that if the threshold used for binarizing the frame difference is in a specific range determined empirically, variations in the final foreground detection will be insignificant. Employing the texture addition in conjunction with leading optical flow methods on multiple real and animation sequences with different texture distributions revealed considerable advantages, including improvement in the accuracy of foreground boundary preservation, prevention of object merging, and reduction in the computation time. The F-measure and the Boundary Displacement Error metrics were used to evaluate the similarity between detected and ground-truth foreground masks. Furthermore, preventing foreground optical flow propagation and reduction in the computation time are discussed using analysis of optical flow convergence.