Adaptive Error Concealment Research Articles

Motion vector recovery for video error concealment based on the plane fitting

In a wireless transmission, the compressed video stream can result in lower quality of image reconstruction with the motion vector mismatch. A motion vector recovery for video error concealment based on the plane fitting algorithm is proposed. According to the characteristics of the H.264 video standard, the adjacent macro blocks and the multi reference frame have the correlation. The motion vector which surrounds the lost macro-block in each divided block is projected by one plane which adopts one point to present the MV changing tendency in small regions. This algorithm reconstructs the motion vector of the damaged macro-block by the fitting plane algorithm based on the improved boundary matching algorithm, and then the optimal motion vector is selected to reconstruct the damaged images. This algorithm not only avoids the block effects that caused by the original algorithm, but also in different probability of RTP packet losing, the peak signal to noise ratio of the algorithm has improved 0.3 ~ 2.5 dB compared to the adaptive intra-frame error concealment algorithm .

Adaptive error concealment for temporal–spatial multiple description video coding

With the development of 3G/4G and WiFi networks, there has been a growing demand for video delivery over wireless channels. This increase of video traffic creates a significant challenge for efficient, reliable and adaptable video coding techniques. Multiple description coding (MDC) is an error resilient video coding method that has been proposed for streaming applications when retransmission is unacceptable. In this paper, we present error concealment approaches for two different four-description MDC architectures. In our first architecture, four descriptions are generated by temporal–spatial subsampling and encoded independently. To improve the error robustness, several adaptive error concealment methods are described. One of them is a frame-level method based on error tracking. It takes into account the distortion due to error concealment and error propagation. The other methods are macroblock-level methods based on foreground–background mapping and distortion mapping respectively. Our second MDC partition architecture aims at better coding efficiency and improved error robustness. Here spatial partitioning is done after prediction so that each of the two descriptions shares the same prediction loop. Experimental results are presented to demonstrate the efficacy of our proposed methods.

영역 구분을 통한 다시점 영상의 깊이맵 손상 복구 기법

깊이맵은 색채 영상과 달리 색상 정보가 아닌 깊이 정보가 픽셀 값을 이루고 있어서 색차가 크더라도 비슷한 거리에 있으면 유사한 픽셀 값으로 나타난다. 또한 색채 영상은 미세한 영역에서 단계적으로 값이 변하면서 대상의 윤곽을 나타내지만 깊이맵은 픽셀 값이 계단식으로 변하여 픽셀 값이 변하는 지점에서 급변하는 경향이 있다. 이러한 깊이맵의 특성상 경계 영역과 그 외의 영역으로 크게 영상의 영역을 구분 할 수 있고, 전송 시 나타나는 오류에 대해 각 영역에 효과적으로 적용 될 수 있는 오류 은닉 방법이 필요하다. 본 연구에서는 깊이맵의 경계 방향에 따라 영역을 구분하고 각 영역의 값이 변화하는 방향성에 맞추어 적응적으로 오류가 은닉 될 수 있는 방법을 적용하여 깊이맵을 복구한다. 복구된 깊이맵은 다시점 영상과 함께 합성하여 새로운 중간 시점 영상을 만들고 이를 객관적 화질평가 방법을 통해 평가한다. The pixel value of depth image is depth value so that different objects which are placed on nearby position have similar pixel value. Moreover, the pixels of depth image have distinct pixel values compared to adjacent pixels while those of color image has very similar values. Accordingly distorted depth image of multiview video plus depth (MVD) needs proper error concealment methods considering the characteristics of depth image when transmission errors are happened. In this paper, classifying regions of depth image to consider edge directions and then applying adaptive error concealment methods to each region are proposed. Recovered depth images utilize with multiview video data to synthesize intermediate-view point video. The synthesized view is evaluated by objective quality metrics to demonstrate proposed method performance.

An Adaptive Error Concealment Method for Depth Map in 3D Video Coding

Stereoscopic images and video are synthesized from virtual views, which are produced from a real view and its depth map. However, this process involves a large amount of data which not only increases the transmission load, but also must be stored within the display devices which therefore must possess huge storage capacities. To solve these problems, the Joint Video Team has developed the 3D video coding (3DVC) standard to reduce the redundant data for multiview video and its depth map through the correlation between the real view and its corresponding depth map. When depth data in the 3DVC bitstream are lost during transmission, it causes the decoded depth map to be incomplete. Then the quality of the decoded depth map is reduced and the quality of the stereoscopic video will also be decreased. To increase the reconstructed quality of the damaged macro-block, this paper proposes an adaptive error concealment selection to determine the most suitable error concealment method based on fuzzy reasoning. The temporal and spatial correlations of the damaged macro-block which are defined by the motion vectors of neighboring blocks and their depth information are used for the adaptive selection. From the simulation result, the proposed algorithm can reduce the concealing time by at least 0.15–2.86 frame/sec and can improve quality by about 0.13–0.17 dB. The proposed algorithm is suitable for the 3DVC video transmission.

An Adaptive Error Concealment Method Based on Fuzzy Reasoning for Multi-View Video Coding

This study proposes an adaptive error concealment selection process in order to improve the quality of reconstructed images which have low computation complexity for multi-view video coding. In this study, the proposed algorithm uses the motion vector variances and texture histogram bins in the blocks neighboring the damaged block in order to define two correlations; the degree of motion and the degree of texture similarity. To increase the accuracy of the suitable EC for the damaged macro-block, these two correlations are used to select a suitable error concealment method for reconstructing the damaged macro-block through fuzzy reasoning. The motion degree indicates the motion complexity for the damaged macro-block, and the texture similarity degree indicates the spatial continuity between the damaged macro-block and its neighboring block. The proposed adaptive error concealment is selected to reduce concealing time on homogenous damaged blocks and to improve the reconstructed image quality on non-homogenous damaged blocks. The experiment results show that the proposed algorithm can reduce the concealing time by at least 1.16 s and can improve the peak signal-to-noise (PSNR) by 0.13 dB–2.44 dB and 0.11 dB–3.52 dB in the main views and auxiliary views, respectively. This proposed algorithm is suitable for the multi-view video transmission on the consumer electronic fields such as 3D mobile devices, 3D television.

Adaptive Error Concealment Technology Based on H.264 for Digital Satellite Set-Top Boxes

In order to improve the decoding performance of digital satellite set-top boxes, this paper proposed a directional boundary matching algorithm on temporal domain for H.264, it considers using the spatial smoothing of video images and the movement consistency in neighbor macro-blocks to restore the damaged macro-block, with pixels matching in three directions on the boundary. On this basis, a adaptive error concealment algorithm of multi-mode is proposed, which avoids the judgment of the partition mode of damaged macro-block. Experimental results show that when in the same case of packet loss, compared with the traditional concealment algorithms, the algorithm proposed in this paper has better performance in the improvement of PSNR and visual effect.

Adaptive error concealment algorithm based on residual distribution for whole frame losses in H.264

The H.264 communication over network may cause a whole frame loss.To solve the problem,an error-concealment algorithm based on residual distribution was proposed for whole frame packet loss in H.264.Firstly,the residual information of reference frame was analyzed.Then,according to the result,the motion vector copy algorithm was used in the region where the image was smooth or the image object's motion was rigid.For the other regions,the motion vector of each pixel was re-estimated,and then optical flow algorithm was introduced.Experimental results show that the algorithm outperforms the traditional one on both visual quality and Peak Signal-to-Noise Ratio(PSNR).

An Adaptive Spatio-Temporal Error Concealment Scheme Based on H.264/AVC

An Adaptive Spatio-Temporal Error Concealment (ASTEC) scheme based on H.264/AVC is proposed in the paper to improve the quality of the rebuilt frame in H.264/AVC decoder side. Combining the spatial error concealment method with the temporal one, a new scheme is proposed to conceal an erroneous macro block according to its encoded mode. The temporal error concealment scheme adopts the weighted Boundary Matching Algorithm to recover the erroneous macro block. The spatial error concealment scheme recovers the erroneous macro block by exploiting the directional interpolation which subdivides the macro block according to the number and the direction of edges going through it. Experiments show that the proposed concealment scheme can restrain the diffusion of erroneous macro block and improve the quality of the rebuilt frame subjectively as well as objectively.

Recovering Connected Error Region Based on Adaptive Error Concealment Order Determination

Parts of compressed video streams may be lost or corrupted when being transmitted over bandwidth limited networks and wireless communication networks with error-prone channels. Error concealment (EC) techniques are often adopted at the decoder side to improve the quality of the reconstructed video. Under the conditions of a high rate of data packets that arrives at the decoder corrupted, it is likely that the incorrectly decoded macro-blocks (MBs) are concentrated in a connected region, where important spatial reference information is lost. The conventional EC methods usually carry out the block concealment following a lexicographic scan (from top to bottom and from left to right of the image), which would make the methods ineffective for the case that the corrupted blocks are grouped in a connected region. In this paper, a temporal error concealment method, adaptive error concealment order determination (AECOD), is proposed to recover connected corrupted regions. The processing order of an MB in a connected corrupted region is adaptively determined by analyzing the external boundary patterns of the MBs in its neighborhood. The performances, on several video sequences, of the proposed EC scheme have been compared with those obtained by using other error concealment methods reported in the literature. Experimental results show that the AECOD algorithm can improve the recovery performance with respect to the other considered EC methods.

Passive Error Concealment for Wavelet-Coded I-Frames With an Inhomogeneous Gauss–Markov Random Field Model

In video communication over lossy packet networks (e.g., the Internet), packet loss errors can severely damage the transmitted video. The damaged video can largely be repaired with passive error concealment, where neighboring information is used to estimate missing information. We address the problem of passive error concealment for wavelet coded data with dispersive packetization. The reported techniques of this kind have many problems and usually fail in the reconstruction of high-frequency content. This paper presents a novel locally adaptive error concealment method for subband coded I-frames based on an inhomogeneous Gaussian Markov random field model. We estimate the parameters of this model from a local context of each lost coefficient, and we interpolate the lost coefficients accordingly. The results demonstrate a significant improvement over the reported related methods both in terms of objective performance measures and visually. The biggest improvement of the proposed method compared to the state-of-the-art in the field is the correct reconstruction of high-frequency information such as textures and edges.

Adaptive temporal error concealment scheme for H.264/AVC video decoder

Resilience to channel noise is indispensable for video communication based consumer applications. Temporal error concealment is a kind of decoder-based technique to compensate for transmission errors and is popular in most consumer electronics. In this paper, an adaptive temporal error concealment scheme is proposed based on H.264 video standard to improve error resilience ability for video consumer applications. Mode switching mechanism is devised to flexibly choose appropriate temporal concealment mode. The search range for candidate motion vectors could be adaptively determined by the range of spatially and temporally neighboring motion vectors. Weighted outer boundary distortion function could dynamically select the optimal replacing motion vector for local area. Experiment and comparison with recent error concealment schemes shows that our proposed algorithm could effectively improve received video quality with reduced requirement for decoding time.

An error resilient video coding and transmission solution over error-prone channels

With the rapid development of network and multimedia technology, the error control in video coding and video transmission over error-prone channels has become increasingly important. The DCT based predictive coding and VLC based entropy coding greatly increase the coding efficiency, but make the compressed video stream very sensitive to transmission errors. Therefore, this paper proposes a reliable error resilient video coding and video transmission framework. In order to improve the robustness to packet loss errors, an error resilient video coding algorithm named Z-FMO is proposed. Some channel may bring random bit errors, an adaptive error concealment algorithm based on macroblock boundary gradient namely ECMBG is proposed aiming at such problem. As the indispensable part of a video transmission system, we implement an adaptive video transmission control algorithm JCBAF. Experimental results show that the proposed framework performs well both in R-D performance and subjective quality.

Standard Deviation and Intra Prediction Mode Based Adaptive Spatial Error Concealment (SEC) in H.264/AVC

Transmission of compressed video over error prone channels may result in packet losses or errors, which can significantly degrade the image quality. Therefore an error concealment scheme is applied at the video receiver side to mask the damaged video. Considering there are 3 types of MBs (Macro Blocks) in natural video frame, i.e., Textural MB, Edged MB, and Smooth MB, this paper proposes an adaptive spatial error concealment which can choose 3 different methods for these 3 different MBs. For criteria of choosing appropriate method, 2 factors are taken into consideration. Firstly, standard deviation of our proposed edge statistical model is exploited. Secondly, some new features of latest video compression standard H.264/AVC, i.e., intra prediction mode is also considered for criterion formulation. Compared with previous works, which are only based on deterministic measurement, proposed method achieves the best image recovery. Subjective and objective image quality evaluations in experiments confirmed this.

Locally Adaptive Passive Error Concealment for Wavelet Coded Images

This letter presents a novel locally adaptive error concealment method for subband coded images. For each lost low-frequency coefficient, we estimate the optimal interpolation weights from its neighborhood. The calculation of the interpolation weights is optimized in the mean squared error sense, and it takes into account the errors that would arise by horizontally and vertically interpolating the available neighbors of the lost coefficient. Compared to methods of similar complexity, the proposed scheme estimates the lost coefficients more accurately: on average, the PSNR is increased by up to 4.5 dB. The reconstructed images also look better, and our method is fast and of low complexity.

Robust Digital Terrestrial TV Broadcasting System with Error-Resilient Schemes

In this paper, error-resilient schemes are proposed to support robust video transmission for digital terrestrial TV broadcasting (DTTB). In particular, a temporal error concealment incorporated with a low-complexity block-matching is developed, achieving an effective reception of predictive pictures in harsh terrestrial environment. Special algorithms are also designed for isolated I-pictures. Moreover, combined with an intra/inter case prediction, an adaptive error concealment scheme is further contrived to fit for different error conditions. Extensive simulations have been conducted under various DTTB channel conditions, even with a very high packet error rate, to verify the effectiveness of the proposed schemes.

A real-time IMT-2000 audio transmission system

For real-time audio streaming services over next-generation mobile phone systems, such as live concert broadcasting and radio relaying, we have developed a realtime IMT-2000 audio transmission system. This system consists of a real-time encoder and a portable real-time decoder, which are both based on fixed-point DSP chips. TWINVQ technology is used for audio compression, and a scheme to protect audio data against bit-errors was designed based on error sensitivity observations. Self-synchronization and adaptive error concealment are carried out at the decoder. We confirmed that the system can transmit high-quality stereo audio samples at 32 kHz with a total bit rate of 64 kbit/s over a simulated IMT-2000 channel. We also checked that this system can connect to a real IMT-2000 experimental network, and found that it is easily applicable to various audio streaming services.

Real-time streaming video with adaptive bandwidth control and DCT-based error concealment

This work deals with dropped streaming video packets from the viewpoint of minimizing future dropped packets, as well as restoring data lost in packets already dropped, specifically geared toward real-time transmission of H.263+ bitstream. Minimization of future dropped packets is achieved by an adaptive least mean squares bandwidth (BW) controller predicting the available bandwidth supported by the Internet with relatively little loss. Restoration of lost video data is performed by a combination of error concealment (EC) techniques, including interpolation of border pixels in DCT domain. All algorithms presented are computationally inexpensive and thus suited for real-time video applications.

Variable-bit-rate HDTV codec for ATM networks

A variable-bit-rate HDTV codec based on a motion-adaptive DCT and multimode quantisation is developed for ATM environments in B-ISDN. Adaptive 2-layered coding and error concealment are implemented to keep picture quality high by compensating for ATM cell loss. The codec hardware restricts the peak bit rate to 65 Mbit/s for HDTV conference application and to 130Mbit/s for secondary distribution. Experimental transport shows that the ATM cell loss compensation scheme has good resilience at a cell error rate as high as 10−4.