Random epipolar constraint loss functions for supervised optical flow estimation

Abstract

The majority of supervised models estimate optical flow through minimizing the numerical difference between the predicted flow and the ground truth, resulting in the loss of positional and geometric characteristics of the calculated flow fields. In addition, these models require a large number of parameters and high computational cost when computing optical flow. To address these issues, this paper presents a novel loss function and a lightweight framework for optical flow estimation. The proposed loss function, called the random epipolar constraint loss function (RECLoss), incorporates epipolar geometry into supervised optimization to transform the numerical difference into geometry constraint. The RECLoss can make the optical flow estimation models more effective and enhance their generalization abilities. Moreover, the design of RECLoss is more interpretable and the estimated optical flow fields from RECLoss have clearly defined mathematical meanings. A lightweight recurrent neural network for optical flow estimation (LRFlow) that balances computational cost and estimation accuracy is also proposed. The LRFlow, containing only 3.0M parameters, consists of a feature extractor, a correlation matching module, and an iterative update unit. The proposed lightweight network achieves state-of-the-art results compared to all other lightweight networks on the challenging MPI-Sintel and KITTI2015 datasets. The effectiveness of RECLoss in improving the accuracy of LRFlow and other state-of-the-art methods such as RAFT and GMA has been validated through extensive experiments. The source code of the project are available at https://github.com/Eryo-iPython/RECLoss.