Dual Heterogeneous Complementary Networks for Single Image Deraining

Abstract

Single image deraining is an extreme challenge task since it requires to not only recover the spatial detail and high-level contextualized structure of the underlying image but also remove multiple rain layers with various blurring degrees and resolutions. Despite of the great performance advance with the deep learning networks, the dominated researches devote to either constructing deeper and complicated network architecture for recovering reliable detailed texture at the original resolution of the input image or exploiting multi-scale encoder-decode structure for learning semantic context in more larger receptive field while are still far from sufficiency to capture both complementary detailed and semantic contexts. This study proposes a novel dual heterogeneous complementary networks consisting of a main original resolution learning subnet and an auxiliary encoder-decoder subnet for exploring both detailed structure and semantic contexts. Specifically, to capture more plausible intermediate features in dual subnets, we concurrently evaluate the deraining losses of both branches in training phase, and exploit an auxiliary pseudo-label supervised attention module to further guide the feature learning in the main subnet. Moreover, to reconstruct more natural and sharper images, we incorporate multiple losses for network training including an improved MSE, an edge-based loss to recover reliable shape information, and a perceptual loss by evaluating the reconstruction error on the feature map of the learned VGGNet model instead of pixel intensity. Experiments on several benchmark deraining datasets demonstrate great superiority over the state-of-the-arts methods.