Context-adaptive matching for optical flow

Abstract

Modern sparse-to-dense optical flow estimation algorithms usually achieve state-of-art performance. Those algorithms need two steps: matching and interpolation. Matching is often unreliable for very large displacement optical flow due to illumination changes, deformations and occlusion etc. Moreover, conspicuous errors around motion discontinuities still keep serious as most methods consider edge only at interpolation step. The context-adaptive matching (CAM) is proposed for optical flow which is better at large displacement and edge preserving. The CAM is selective in feature extraction, adaptive in flow propagation and search radius adjusting. Selective features are proposed to consider edge preserving in matching step. Except for the usually used SIFT descriptor, the local directional pattern flow (LDPF) is introduced to keep more edge structure, and the oriented fast and rotated brief (ORB) is utilized to select out several most similar candidates. Unlike coarse-to-fine matching, which proposed a propagation step with only neighbors, we propose adaptive propagation to extend the matching candidates in order to improve the possibility of getting right correspondences. Furthermore, guided by prior knowledge and taking advantage of upper layers results, adaptive radius instead of constrained radius are proposed at finer layers. The CAM interpolated by EpicFlow is fast and robust for large displacements especially for fast moving objects and also preserves the edge structure well. Extensive experiments show that our algorithm is on par with the state-of-art optical flow methods on MPI-Sintel, KITTI and Middlebury.