Sparse spatio-temporal representation with adaptive regularized dictionaries for super-resolution based video coding

Abstract

We propose a sparse representation learning with adaptive regularized dictionaries and develop a low bit-rate video coding scheme. In a reversed-complexity manner, it select a subset of key frames to encode at original resolution, while the rest are down-sampled and super-resolution reconstructed by a sparse super-resolution estimations using key frames as training set. Since primitive patches are of low dimensionality and can be well learned from the primitive patches across different images, video frame is divided into three layers: a primitive layer, a non-primitive coarse layer, and a non-primitive smooth layer. The non-primitive layer is constructed as volumes to keep consistent along the motion trajectory, which enables sparse representations over a learned 3-D spatio-temporal dictionary. Correspondingly, the target is formulated as an optimization problem by constructing a sparse representation of low-resolution frame patches or volumes over adaptive regularized dictionaries: a set of 2-D sub dictionary pairs trained from 2-D primitive patches and a 3-D dictionary trained from non-primitive volumes. In reconstruction, the lost high-frequency information of the down-sampled frames can be synthesized from the sparse spatio-temporal representation over the adaptive regularized dictionaries. Experimental results validate the compression efficiency of the proposed scheme versus the H.264/AVC in terms of both objective and subjective comparison.