Multi-scale observation models for motion estimation

Abstract

Multi-scale frameworks based on coarse-to-fine warping strategies are widely used in the state-of-the-art optical flow methods. While they allow the estimation of large motions and usually lead to a faster minimization, they can also create strong dependencies between the successive estimates at different scale levels, yielding sometimes the propagation of undesirable errors from coarse to fine scales without any mean of correction. In this paper, we propose a more flexible framework inspired from fluid mechanics able to partly counter this issue. It relies on filtering equations where the variable of interest (i.e. the velocity field) is decomposed into resolved and unresolved components at each scale. We then derive a new data term that allows to take into account, in the coarse scales, information about smaller scale levels in order to avoid errors propagation during the estimation. Embedded in a simple Lucas-Kanade estimator, our new term is able to greatly improve the results from usual conservation constraints, as shown in the experimental part.