PDNet: Progressive denoising network via stochastic supervision on reaction-diffusion–advection equation

Abstract

Recent works on residual network and its variants help us to advance our understanding of deep neural networks as differential equations. Specifically, when taking convolution operator as the combinations of differential operators, deep convolutional neural network (DCNN) can be regarded as partial differential equation (PDE). Learn such PDEs have become popular in image restoration especially for image denoising. While current methods for learning the denoising PDEs face two critical issues. On the one hand, the learned PDEs are not guaranteed to behave as progressive denoising as traditional denoising PDEs. On the other hand, the low level feature may diminish with time increases in PDE. To resolve these issues, this paper designs to learn a specialized PDE network (PDNet) which depicts the information propagation in the network as a reaction-diffusion–advection process. Specifically, a reaction-term is introduced to prevent the low-level features from diminishing in the PDE network, and a decreasing residual gain strategy with stochastic supervision is adopted to facilitate the progressive denoising property. The progressive denoising property not only reveals the semantic of the network path but also regularizes the network training. Compared to vanilla DCNNs, the PDNet has clear physical meaning. Meanwhile PDNet simplifies the ResNet in that batch normalization is removed from the residual blocks and there is only one nonlinear unit in a residual block (2 for ResNet). Extensive experimental results on benchmark datasets verified that PDNet is not only efficient but also effective.