Multi-Level Memory Compensation Network for Rain Removal via Divide-and-Conquer Strategy

Abstract

Recently, an increasing number of algorithms have been proposed for rain streak removal. However, most existing methods ignore the discrepancy in removing rain streaks from background contents with different texture richness (frequencies). They adopt a unified scheme to learn the final distribution of rain streaks directly, thus sacrificing the representative accuracy of rain information. To this end, this paper leverages the divide-and-conquer strategy for rain streak removal by decomposing the learning task into several subproblems according to the levels (frequencies) of texture richness in background contents. Particularly, we construct a novel multi-level memory compensation network (MLMCN) for rain streak removal. It achieves a promising solution by individually handling these subproblems under the specific texture richness via several parallel subnetworks. Each subnetwork takes as input a specific sub-sampled image, sampled from the original rain ones via the Gaussian kernel, to individually learn one of the sub-distributions of the rain information. We thus produce a high-quality rain-free image by subtracting the predicted rain information from multiple subnetworks in turn. We experimentally show that our proposed MLMCN outperforms the existing deraining methods in terms of quantitative indicators, visual effects on several benchmark datasets, and the high-level object detection task.