Real-time foveation techniques for low bit rate video coding

Abstract

Lossy video compression methods often rely on modeling the abilities and limitations of the intended receiver, the human visual system (HVS), to achieve the highest possible compression with as little effect on perceived quality as possible. Foveation, which is non-uniform resolution perception of the visual stimulus by the HVS due to the non-uniform density of photoreceptor cells in the eye, has been demonstrated to be useful for reducing bit rates beyond the abilities of uniform resolution video coders. In this work, we present real-time foveation techniques for low bit rate video coding. First, we develop an approximate model for foveation. Then, we demonstrate that foveation, as described by this model, can be incorporated into standard motion compensation and discrete cosine transform (DCT)-based video coding techniques for low bit rate video coding, such as the H.263 or MPEG-4 video coding standards, without incurring prohibitive complexity overhead. We demonstrate that foveation in the DCT domain can actually result in computational speed-ups. The techniques presented can be implemented using the baseline modes in the video coding standards and do not require any modification to, or post-processing at, the decoder.