Advanced inpainting-based macroblock prediction with regularized structure propagation in video compression

Abstract

In this paper, we propose an optimized inpainting-based macroblock (MB) prediction mode (IP-mode) in the state-of-the-art H.264/AVC video compression engine, and investigate a natural extension of structured sparsity over the ordered Belief Propagation (BP) inference in inpainting-based prediction. The IP-mode is regularized by a global spatio-temporal consistency between the predicted content and the co-located known texture, and could be adopted in both Intra and Inter frames without redundant assistant information. It is solved by an optimization problem under Markov Random Field (MRF), and the structured sparsity of the predicted macroblock region is inferred by tensor voting projected from the decoded regions to tune the priority of message scheduling in BP with a more convergent manner. Rate-distortion optimization is maintained to select the optimal mode among the inpainting-based prediction (IP-), the intra-, and inter-modes. Compared to the existing prediction modes in H.264/AVC, the proposed inpainting-based prediction scheme is validated to achieve a better R-D performance for homogeneous visual patterns and behave a more robust error resilience capability with an intrinsic probabilistic inference.