Patch Ordering as a Regularization for Inverse Problems in Image Processing

Abstract

Recent work in image processing suggests that operating on (overlapping) patches in an image may lead to state-of-the-art results. This has been demonstrated for a variety of problems including denoising, inpainting, deblurring, and super-resolution. The work reported in [I. Ram, I. Cohen, and M. Elad, IEEE Trans. Image Process., 23 (2014), pp. 2779--2792] and [I. Ram, M. Elad, and I. Cohen, IEEE Trans. Image Process., 22 (2013), pp. 2764--2774] takes an extra step forward by showing that ordering these patches to form an approximate shortest path can be leveraged for better processing. The core idea is to apply a simple filter on the resulting 1D smoothed signal obtained after the patch-permutation. This idea has been also explored in combination with a wavelet pyramid, leading eventually to a sophisticated and highly effective regularizer for inverse problems in imaging. In this work we further study the patch-permutation concept and harness it to propose a new simple yet effective regularization for image restoration problems. Our approach builds on the classic maximum a posteriori probability (MAP), with a penalty function consisting of a regular log-likelihood term and a novel permutation-based regularization term. Using a plain 1D Laplacian, the proposed regularization forces robust smoothness ($L1$) on the permuted pixels. Since the permutation originates from patch ordering, we propose to accumulate the smoothness terms over all of the patches' pixels. Furthermore, we take into account the found distances between adjacent patches in the ordering by weighting the Laplacian outcome. We demonstrate the proposed scheme on the following diverse set of problems: (i) severe Poisson image denoising, (ii) Gaussian image denoising, (iii) image deblurring, and (iv) single image super-resolution. In all of these cases, we use recent methods that handle these problems as initialization to our scheme. This is followed by an L-BFGS optimization of the above-described penalty function, leading to state-of-the-art results, especially for highly ill-posed cases.