Sparsity in optical flow and trajectories

Abstract

In this article, we apply sparse constraints to improve optical flow and trajectories. We apply sparsity in two ways. First, with two-frame optical flow, we enforce a sparse representation of flow patches using a learned overcomplete dictionary. Second, we apply a low-rank constraint to trajectories via robust coupling. Optical flow is an ill-posed underconstrained inverse problem. Many recent approaches use total variation to constrain the flow solution to satisfy color constancy. In our first results presented, we find that learning a 2D overcomplete dictionary from the total variation result and then enforcing a sparse constraint on the flow improves the result. A new technique using partially overlapping patches accelerates the calculation. This approach is implemented in a coarse-to-fine strategy. Our results show that combining total variation and a sparse constraint from a learned dictionary is more effective than total variation alone. In the second part, we compute optical flow and trajectories from an image sequence. Sparsity in trajectories is measured by matrix rank. We introduce a low-rank constraint of linear complexity using random subsampling of the data. We demonstrate that, by using a robust coupling with the low-rank constraint, our approach outperforms baseline methods on general image sequences.