Optical Flow Estimation in Dense Foggy Scenes With Domain-Adaptive Networks

Abstract

Estimating optical flow in dense foggy scenes is a challenging task. The basic assumptions for computing flow such as brightness and gradient constancy become invalid. To address the problem, we introduce a semi-supervised deep learning method that can learn from real fog images without requiring the corresponding optical flow ground-truths. Our method is a multi-task network, integrating the domain transformation and optical flow networks in one framework. The domain transformation is performed between foggy and clean domains. Under our semi-supervised training strategy, we train our method in a supervised manner with a pair of synthetic fog images, their corresponding clean images and optical flow ground-truths. Subsequently, given a pair of real fog images and a pair of clean images that are not corresponding to each other (unpaired), in the next training batch, we train our network in an unsupervised manner. The supervised and unsupervised training processes are alternated iteratively. Since our method relies on unsupervised learning for real data, we show that it can be used for test-time training. We show in our experiments that performing test-time training improves the results further on our test data. Extensive experiments show that our method outperforms the state-of-the-art methods in estimating optical flow in dense foggy scenes.