Revisiting Lucas-Kanade and Horn-Schunck

Abstract

This paper revisits classical Lucas-Kanade (LK) and Horn-Schunck (HS) optical flow techniques. The aim is to provide a baseline for other researchers on these two timeless techniques. The formulations presented incorporate modern practices, namely multichannel, multi-resolution with refinement procedure (warping), non-quadratic penalisers and non-linear formulation of the brightness assumption. The experiments conducted demonstrate the performance enhancement that can be assigned to each modern practice. Thereby, the performance of the LK and HS is renewed in order to enable a fair comparison to other state-of-art techniques. The good performance of today's optical flow formulations is possible due to several modern practices, namely multi- resolution (coarse-to-fine) refinement that deals with large displacements, iterative methods based on warping interpolations, multi-channels, robustification of each channel (non- quadratic penalisers), gradient constancy assumption to handle small variation of the brightness value, and photometric colour spaces. Usually, the performance of state-of-art techniques is compared to the classical Lucas-Kanade (LK) and Horn-Schunck (HS) presented by Barron et al. (1994) (2) . We consider these comparisons unfair because the LK and HS do not contemplate the most important modern practices (such as, multi-resolution with refinement, nonlinear formulation of the brightness constancy assumption). Therefore, the full potential of both techniques is not considered. For that reason, this article intends to provide a modern formulation of the LK and HS that incorporates multi-channel, multi-resolution with refinement (iterative procedure based on warping) and nonlinear brightness assumption. In addition, non-quadratic penaliser based on the Charbonnier error function is considered for the neighbourhood weighting of the LK. Thereby, the contributions of this paper include:  Modern and coloured formulation of the LK and HS  Extensive qualitative and quantitative evaluation  Baseline performance analysis for other research The experimental analysis includes comparisons of both non-hierarchical and hierarchical pyramidal architectures. The contribution of the iterative refinement and multi-channel approach is also studied. Finally, the neighbourhood weights of the LK are iteratively defined using the non-quadratic penaliser. When implemented with appropriate and modern practices, classical optical flow formulations such as Lucas- Kanade (9) and Horn-Schunck (10) can achieve respectable optical flow estimations.