Bayesian denoising of structured sources and its implications on learning-based denoising

Abstract

Abstract Denoising a stationary process $(X_{i})_{i \in \mathbb{Z}}$ corrupted by additive white Gaussian noise $(Z_{i})_{i \in \mathbb{Z}}$ is a classic, well-studied and fundamental problem in information theory and statistical signal processing. However, finding theoretically founded computationally efficient denoising methods applicable to general sources is still an open problem. In the Bayesian set-up where the source distribution is known, a minimum mean square error (MMSE) denoiser estimates $X^{n}$ from noisy measurements $Y^{n}$ as $\hat{X}^{n}=\mathrm{E}[X^{n}|Y^{n}]$. However, for general sources, computing $\mathrm{E}[X^{n}|Y^{n}]$ is computationally very challenging, if not infeasible. In this paper, starting from a Bayesian set-up, a novel denoising method, namely, quantized maximum a posteriori (Q-MAP) denoiser is proposed and its asymptotic performance is analysed. Both for memoryless sources, and for structured first-order Markov sources, it is shown that, asymptotically, as $\sigma _{z}^{2} $ (noise variance) converges to zero, ${1\over \sigma _{z}^{2}} \mathrm{E}[(X_{i}-\hat{X}^{\mathrm{QMAP}}_{i})^{2}]$ converges to the information dimension of the source. For the studied memoryless sources, this limit is known to be optimal. A key advantage of the Q-MAP denoiser, unlike an MMSE denoiser, is that it highlights the key properties of the source distribution that are to be used in its denoising. This key property leads to a new learning-based denoising approach that is applicable to generic structured sources. Using ImageNet database for training, initial simulation results exploring the performance of such a learning-based denoiser in image denoising are presented.