Efficient deep learning of image denoising using patch complexity local divide and deep conquer

Abstract

Image denoising is a fundamental task in computer vision and image processing domain. In recent years, the task has been tackled with deep neural networks by learning the patterns of noises and image patches. However, because of the high diversity of natural image patches and noise distributions, a huge network with a large amount of training data is necessary to obtain a state-of-the-art performance. In this paper, we propose a novel ensemble strategy of exploiting multiple deep neural networks for efficient deep learning of image denoising. We divide the task of image denoising into several local subtasks according to the complexity of clean image patches and conquer each subtask using a network trained on its local space. Then, we combine the local subtasks at test time by applying the set of networks to each noisy patch as a weighted mixture, where the mixture weights are determined by the likelihood of each network for each noisy patch. Our methodology of using locally-learned networks based on patch complexity effectively decreases the diversity of image patches at each single network, and their adaptively-weighted mixture to the input combines the local subtasks efficiently. Extensive experimental results on Berkeley segmentation dataset and standard test images demonstrate that our strategy significantly boosts denoising performance in comparison to using a single network of the same total capacity. Furthermore, our method outperforms previous methods with much smaller training samples and trainable parameters, and so with much reduced time complexity both in training and running.