A framework for error protection of region of interest coded images and videos

Abstract

In this paper, we propose a framework for unequal error protection (UEP) of image and video streaming over a wireless channel. Our framework of allocating the parity symbols associated with error control coding to the image or video codestream takes advantage of the different levels of importance that particular spatial regions of visual multimedia content has to human observers. As such, it provides stronger protection against transmission impairments to those parts of an image or video stream that correspond to the regions of interest (ROIs) while weaker protection is applied to the background (BG). For this purpose, an image or video stream represented by a sequence of packets is split into smaller cells in such a way that certain cells contain the parts of a codestream that represent ROIs and the last cell carries solely BG information. The available parity budget obtained from the given code rate is then distributed among these cells based on their contribution to the overall perceptual quality of a reconstructed image or video. A dynamic programming approach is utilized to facilitate optimal allocation of parity to ROIs and BG for ROI based UEP. The performance of the proposed ROI based UEP scheme is analyzed and compared with both the optimal UEP without ROI processing and equal error protection (EEP) in terms of an objective perceptual quality metric, the structural similarity (SSIM) index. Performance results validate the effectiveness of our framework and the superior performance of the proposed UEP scheme compared to EEP. The performance of the proposed UEP scheme matches well with that of the optimal UEP without ROI processing, especially, for multiple spatial description image and video coding while computational complexity can be kept much lower.