Error Concealment Techniques Research Articles

An integrated temporal error concealment for H.264/AVC based on spatial evaluation criteria

Owing to error-prone transmission networks, the compressed video bit stream is prone to packet loss in the transmission channel. This loss causes serious distortion and the distortion will propagate to successive frames, especially in highly compressed video coding standard. Therefore, it is very important to efficiently enhance the restored result. In this paper, an integrated temporal error concealment technique for H.264/AVC is proposed. The technique could effectively restore the corrupted data by adaptively integrating error concealment approaches with the adaptive weight-based switching algorithm. The integrated mechanism is based on spatial evaluation criteria, judged by boundary distortion estimation and texture intensity. Experimental results show that the technique could effectively enhance the performance of error concealment.

Algorithm and Architecture Design of Image Inpainting Engine for Video Error Concealment Applications

Error concealment techniques can improve subjective video quality in a decoder when the video bitstream is corrupted during transmission. In this paper, to achieve perceptually pleasant results, an image inpainting technique in which structure information generated from edge information is adopted as the spatial error concealment method. In addition, a modified boundary matching algorithm for temporal error concealment is proposed for temporal frames. To maintain low hardware costs as regards the error concealment engine, the processing iteration number of each macroblock is limited to four based on the proposed inpainting algorithm. Block-based pipeline scheduling is also proposed to reduce the number of processing cycles and the on-chip memory size. Moreover, a cache-based data reuse scheme is developed to reduce the processing cycles and external bandwidth. Moreover, the two concealment modes share the same computational core to reduce hardware costs. A prototype chip is implemented by using the UMC 90 nm process. The total gate count is approximately 121 k at 200 MHz. The maximum processing capability can support 244.8 k macroblocks per second or 1920 × 1080 4:2:0 30 Hz video. The core size is 1.30 × 1.30 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> . The average power dissipation is 131.4 mW at 200 MHz. Compared to other error concealment methods, the proposed design can achieve better perceptual quality at an acceptable additional hardware cost.

Confident Recovery for Lost Motion Vectors in Video Transmission

Abstract Error concealment techniques are very efficient for recover the corrupted or lost information of compressed video sequences caused by video communication over error-prone networks. A new error concealment algorithm is proposed to give a more precise estimation of the lost motion vectors. The confident degree method is used to define a filter of neighbor blocks’ motion vectors. The estimation will be executed according to the output of the filter. Simulation results show that the proposed method can recover the higher quality image in different video sequences, comparing to the existing traditional concealment algorithms.

Spatial error concealment with sequence-aligned texture modeling and adaptive directional recovery

A spatial error concealment technique based on the sequence-aligned texture modeling and the adaptive directional recovery is proposed in this work. The sequence alignment technique captures the local variation and the global trend of image textures with surrounding uncorrupted pixels, and provides the best texture model under a given cost function. With the derived texture model, geometric interpolation is used to recover lost pixels adaptively based on pixel locations. There are four candidate modes of pixel sequences to recover lost pixels, and one of them is selected for the concealment purpose. The selection criterion is based on the texture pattern modes of surrounding uncorrupted blocks. The pixel sequences used for error concealment can be obtained from the computation of the decoder or the side information from the encoder. Extensive experimental results are given to demonstrate that our error concealment technique outperforms several benchmark methods in both objective and subjective tests.

An effective GM/LM-based video error concealment

Error concealment (EC) techniques are often utilized at decoder side to improve reconstructed videos, in case of some information is lost during transmission on a wireless or band-width limited communication channel. In this paper, a global motion/local motion (GM/LM)-based error concealment method is proposed. First, the correct-macro-blocks (CMBs) are classified into global motion compensated MBs and local motion compensated MBs adaptively. Then, an erroneous MB (EMB) is classified into one of the three types: global motion MB (GMB), local motion MB (LMB), and global/local overlapping MB (GLMB) according to the MB type information in its neighbors. For the EMB with its type, GMB is recovered using the global motion vector (GMV). The EMB with its type LMB is recovered using the average motion vector information. And for the MB with its type GLMB, a recursive boundary matching strategy is utilized to search an optimal recovering motion vector. Experimental results show the effectiveness of the proposed GM/LM-based error concealment method.

Spatial error concealment technique for losslessly compressed images using data hiding in error-prone channels

Error concealment techniques are significant due to the growing interest in imagery transmission over error-prone channels. This paper presents a spatial error concealment technique for losslessly compressed images using least significant bit (LSB)-based data hiding to reconstruct a close approximation after the loss of image blocks during image transmission. Before transmission, block description information (BDI) is generated by applying quantization following discrete wavelet transform. This is then embedded into the LSB plane of the original image itself at the encoder. At the decoder, this BDI is used to conceal blocks that may have been dropped during the transmission. Although the original image is modified slightly by the message embedding process, no perceptible artifacts are introduced and the visual quality is sufficient for analysis and diagnosis. In comparisons with previous methods at various loss rates, the proposed technique is shown to be promising due to its good performance in the case of a loss of isolated and continuous blocks.

An efficient spatio-temporal boundary matching algorithm for video error concealment

The highly error-prone channel and limited computational power of terminal devices necessitates the implementation of robust yet simple error concealment. Error concealment techniques commonly make use of the surrounding correctly received image data or motion information for concealment. In this paper, we propose an efficient spatio-temporal boundary matching algorithm (ESTBMA) which exploits both spatial and temporal information to reconstruct the lost motion vectors (MV) and also introduces a new side smoothness measurement. The motion vector corresponding to the minimum of the distortion function is used as the estimation of motion vector of the lost block. Compared to the classical average motion vector (AVMV), median motion video (MMV) and boundary matching algorithm (BMA), simulation results show that the proposed algorithm can recover the higher quality image on both subjective visual evaluation and objective numerical metrics.

High‐Performance Region‐of‐Interest Image Error Concealment with Hiding Technique

Recently region‐of‐interest (ROI) based image coding is a popular topic. Since ROI area contains much more important information for an image, it must be prevented from error decoding while suffering from channel lost or unexpected attack. This paper presents an efficient error concealment method to recover ROI information with a hiding technique. Based on the progressive transformation, the low‐frequency components of ROI are encoded to disperse its information into the high‐frequency bank of original image. The capability of protection is carried out with extracting the ROI coefficients from the damaged image without increasing extra information. Simulation results show that the proposed method can efficiently reconstruct the ROI image when ROI bit‐stream occurs errors, and the measurement of PSNR result outperforms the conventional error concealment techniques by 2 to 5 dB.

Data Embedding in JPEG Bitstream by Code Mapping

We propose an algorithm to embed data directly in the bitstream of JPEG imagery. The motivation for this approach is that images are seldom available in uncompressed form. Algorithms that operate in spatial domain, or even in coefficient domain, require full (or at best) partial decompression. Our approach exploits the fact that only a fraction of JPEG code space is actually used by available encoders. Data embedding is performed by mapping a used variable length code (VLC) to an unused VLC. However, standard viewers unaware of the change will not properly display the image. We address this problem by a novel error concealment technique. Concealment works by remapping run/size values of marked VLCs so that standard viewers do not lose synchronization and displays the image with minimum loss of quality. It is possible for the embedded image to be visually identical to the original even though the two files are bitwise different. The algorithm is fast and transparent and embedding is reversible and file-size preserving. Under certain circumstances, file size may actually decrease despite carrying a payload.

No-Reference Video Quality Monitoring for H.264/AVC Coded Video

When video is transmitted over a packet-switched network, the sequence reconstructed at the receiver side might suffer from impairments introduced by packet losses, which can only be partially healed by the action of error concealment techniques. In this context we propose NORM (NO-Reference video quality Monitoring), an algorithm to assess the quality degradation of H.264/AVC video affected by channel errors. NORM works at the receiver side where both the original and the uncorrupted video content is unavailable. We explicitly account for distortion introduced by spatial and temporal error concealment together with the effect of temporal motion-compensation. NORM provides an estimate of the mean square error distortion at the macroblock level, showing good linear correlation (correlation coefficient greater than 0.80) with the distortion computed in full-reference mode. In addition, the estimate at the macroblock level can be successfully exploited by forward quality monitoring systems that compute quality objective metrics to predict mean opinion score (MOS) values. As a proof of concept, we feed the output of NORM to a reduced-reference quality monitoring system that computes an estimate of the structural similarity metric (SSIM) score, which is known to be well correlated with perceptual quality.

Error Resilient Coding and Error Concealment in Scalable Video Coding

Scalable video coding (SVC), which is the scalable extension of the H.264/AVC standard, was developed by the Joint Video Team (JVT) of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group). SVC is designed to provide adaptation capability for heterogeneous network structures and different receiving devices with the help of temporal, spatial, and quality scalabilities. It is challenging to achieve graceful quality degradation in an error-prone environment, since channel errors can drastically deteriorate the quality of the video. Error resilient coding and error concealment techniques have been introduced into SVC to reduce the quality degradation impact of transmission errors. Some of the techniques are inherited from or applicable also to H.264/AVC, while some of them take advantage of the SVC coding structure and coding tools. In this paper, the error resilient coding and error concealment tools in SVC are first reviewed. Then, several important tools such as loss-aware rate-distortion optimized macroblock mode decision algorithm and error concealment methods in SVC are discussed and experimental results are provided to show the benefits from them. The results demonstrate that PSNR gains can be achieved for the conventional inter prediction (IPPP) coding structure or the hierarchical bi-predictive (B) picture coding structure with large group of pictures size, for all the tested sequences and under various combinations of packet loss rates, compared with the basic joint scalable video model (JSVM) design applying no error resilient tools at the encoder and only picture copy error concealment method at the decoder.

Error Resilient Coding and Error Concealment in Scalable Video Coding

Scalable video coding (SVC), which is the scalable extension of the H.264/AVC standard, was developed by the Joint Video Team (JVT) of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group). SVC is designed to provide adaptation capability for heterogeneous network structures and different receiving devices with the help of temporal, spatial, and quality scalabilities. It is challenging to achieve graceful quality degradation in an error-prone environment, since channel errors can drastically deteriorate the quality of the video. Error resilient coding and error concealment techniques have been introduced into SVC to reduce the quality degradation impact of transmission errors. Some of the techniques are inherited from or applicable also to H.264/AVC, while some of them take advantage of the SVC coding structure and coding tools. In this paper, the error resilient coding and error concealment tools in SVC are first reviewed. Then, several important tools such as loss-aware rate-distortion optimized macroblock mode decision algorithm and error concealment methods in SVC are discussed and experimental results are provided to show the benefits from them. The results demonstrate that PSNR gains can be achieved for the conventional inter prediction (IPPP) coding structure or the hierarchical bi-predictive (B) picture coding structure with large group of pictures size, for all the tested sequences and under various combinations of packet loss rates, compared with the basic joint scalable video model (JSVM) design applying no error resilient tools at the encoder and only picture copy error concealment method at the decoder.

Recovering Connected Error Region Based on Adaptive Error Concealment Order Determination

Parts of compressed video streams may be lost or corrupted when being transmitted over bandwidth limited networks and wireless communication networks with error-prone channels. Error concealment (EC) techniques are often adopted at the decoder side to improve the quality of the reconstructed video. Under the conditions of a high rate of data packets that arrives at the decoder corrupted, it is likely that the incorrectly decoded macro-blocks (MBs) are concentrated in a connected region, where important spatial reference information is lost. The conventional EC methods usually carry out the block concealment following a lexicographic scan (from top to bottom and from left to right of the image), which would make the methods ineffective for the case that the corrupted blocks are grouped in a connected region. In this paper, a temporal error concealment method, adaptive error concealment order determination (AECOD), is proposed to recover connected corrupted regions. The processing order of an MB in a connected corrupted region is adaptively determined by analyzing the external boundary patterns of the MBs in its neighborhood. The performances, on several video sequences, of the proposed EC scheme have been compared with those obtained by using other error concealment methods reported in the literature. Experimental results show that the AECOD algorithm can improve the recovery performance with respect to the other considered EC methods.

Adaptive Pixel Interpolation for Spatial Error Concealment

Error concealment techniques are widely used as the efficient ways to recover the lost information at the decoder. This paper proposes an adaptive pixel interpolation technique for spatial error concealment in the block based coding system. For a specific missing pixel in a corrupted block, we use the different prediction values derived from neighbor regions of the block to interpolate the recovery value of the missing pixel by adaptively setting different weights to these prediction values. More importantly, it enables the recovery of the consecutive blocks loss, which is the most often data loss pattern in practice. Experimental results show the proposed technique gains more accurate recovery of the missing pixels than the existing schemes.

A robust scheme for distributed speech recognition over loss-prone packet channels

In this paper, we propose a whole recovery scheme designed to improve robustness against packet losses in distributed speech recognition systems. This scheme integrates two sender-driven techniques, namely, media-specific forward error correction (FEC) and frame interleaving, along with a receiver-based error concealment (EC) technique, the weighted Viterbi algorithm (WVA). Although these techniques have been already tested separately, providing a significant increase of performance in clean acoustic environments, in this paper they are jointly applied and their performance in adverse acoustic conditions is evaluated. In particular, a noisy speech database and the ETSI Advanced Front-end are used, while the dynamic features, which play an important role in adverse acoustic environments, and their confidences for the WVA algorithm are examined. In order to solve the issue of mixing two sender-driven techniques (both causing a delay) whose direct composition causes an increase of the global latency, we propose a double stream scheme which limits the latency to the maximum delay of both techniques. As a result, with very few overhead bits and a very limited delay, the integrated scheme achieves a significant improvement in the performance of a DSR system over a degraded transmission channel, both in clean and noisy acoustic conditions.

A Spatially Enhanced Error Concealment Technique and its Potential Alternative Application to Reduce H.264 Stream Sizes

With more and more video content being transmitted digitally and with user expectations continually rising, errorconcealment is becoming an increasingly important part ofstreaming media. Often overlooked in the past, even nowmanufacturers are often only doing the bare minimum necessary in order to avoid complexity. This paper first presents a combination of simple techniques that when combined produce an extremely effective concealment method that maintains spatially correlated edges throughout any lost data; this in turn gives an increase in both mathematical and visual performance when compared against the commonly used bilinear concealment technique. Secondly this paper looks at an alternative use of thebilinear passive error concealment algorithm that is often used by H.264 decoders. Occasionally a concealed macroblock is mathematically closer to the original than an encoded and decoded one, by removing these from the stream at the encoder and thus forcing the decoder to conceal the missing data, a significant reduction in the bit stream size (up to 5%) can be achieved with almost no loss in quality.

Error concealment aware streaming video system over packet-based mobile networks

A compressed video bitstream is extremely sensitive to errors when it is transmitted via networks with bursty packet loss such as 3G and mobile WiMAX. Error concealment techniques have been used to improve the recovered picture quality in cases in which parts of the coded picture are not available at the decoder for reconstruction. However, most conventional error concealment methods show low image reconstruction performance due to unavailable motion information about image blocks during burst packet loss, and require high energy consumption to find the best error concealment method in the receiver. The proposed error concealment aware error resilient streaming video system has an intelligent error concealment selector. The method recommends the best error concealment method per macroblock, by additionally transmitting its error concealment selection codes to the decoder. The streaming client conceals the corrupted MB via the chosen error concealment method, with low energy consumption, even when the frame loss is bursty. The experimental results show that the system is highly effective, in terms of the video quality and energy consumption, at the cost of a slight overhead for transmitted bit streams.

Frequency-Domain Stochastic Error Concealment for Wireless Audio Applications

Real time digital audio delivery over Wireless Local Area Networks (WLANs) represents an attractive, flexible and cost effective framework for realizing high-quality, multichannel home audio applications. However, the unreliable nature of WLANs IP link frequently imposes significant playback quality degradation, due to delay or permanent loss of a number of transmitted digital audio packets. In this paper, a novel packet error concealment technique is presented, based on the spectral reconstruction of the statistical equivalent of a previously successfully received audio data packet. It is shown that the proposed data reconstruction scheme outperforms previously published error concealment strategies, in both terms of objective and perceptual criteria.

Low-complexity video error concealment for mobile applications using OBMA

Low-complexity error concealment techniques for mobile applications are studied extensively in this paper. The boundary matching algorithm (BMA) is an attractive choice for mobile video concealment due to its low complexity. Here, we examine a variant of BMA called the outer boundary matching algorithm (OBMA). Although BMA and OBMA are similar in their design principle, it is empirically observed that OBMA outperforms BMA by a significant margin (typically, 0.5 dB or higher) while maintaining the same level of complexity. In this work, we attempt to explain the superior performance of OBMA, and conclude that OBMA provides an excellent tradeoff between the complexity and the quality of concealed video for a wide range of test video sequences and error conditions.

Robust multiple description image coding over wireless networks based on wavelet tree coding, error resilient entropy coding, and error concealment

In this paper, we propose an integrated robust multiple description coding (MDC) scheme for compressed still images over error-prone channels with both packet loss and random bit errors. Multiple descriptions are first generated independently by using multiple description scalar quantizer (MDSQ). To achieve an excellent error resilient capability for each description, multiple bitstreams are then generated based on wavelet trees along the spatial orientations. The spatial orientation trees in the wavelet domain are individually encoded using SPIHT. Error propagation is thus limited within each bitstreams. However, synchronization words are usually needed to avoid error propagation across multiple independent bitstreams. In order to maintain high compression efficiency robust synchronization, we adopt error resilient entropy coding (EREC) to re-organize these variable-length bitstreams into fixed-length data slots before multiplexing and transmission. Therefore, the synchronization of the start of each bitstream can be automatically obtained at the receiver. Finally, to alleviate the devastating image degradation resulted from errors in the beginning of the bitstreams, we propose an error concealment technique to both constrain the EREC/MDC decoding and post-process the decoded wavelet coefficients. With analysis and experimental results, we have demonstrated how each of these components has contributed to the overall robustness of the proposed transmission of compressed images over wireless channels.