Multi-hypothesis Motion-compensated Prediction Research Articles

Error Resilient Video Coding Using B Pictures in H.264

Since the quality of compressed video is vulnerable to errors, video transmission over unreliable Internet is very challenging today. Multi-hypothesis motion-compensated prediction (MHMCP) has been shown to have error resilience capability for video transmission, where each macroblock is predicted by a linear combination of multiple signals (hypotheses). B picture prediction is a special case of MHMCP. In H.264/AVC, the prediction of B pictures is generalized such that both of the two predictions can be selected from the past pictures or from the subsequent pictures. The multiple reference picture framework in H.264/AVC also allows previously decoded B pictures to be used as references for B picture coding. In this paper, we will discuss the error resilience characteristics of the generalized B pictures in H.264/AVC. Three prediction patterns of B pictures are analyzed in terms of their error-suppressing abilities. Both theoretical models (picture level error propagation) and simulation results are given for the comparison.

H.264 error resilience coding based on multi-hypothesis motion-compensated prediction

In this paper, we propose efficient schemes for enhancing the error robustness of multi-hypothesis motion-compensate predictive (MHMCP) coder without sacrificing the coding efficiency significantly. The proposed schemes utilize the concept of reference picture interleaving and data partitioning to make the MHMCP-coded video more resilient to channel errors, especially for burst channel error. Besides, we also propose a scheme of integrating adaptive intra-refresh into the proposed MHMCP coder to further improve the error recovery speed. Extensive simulation results show that the proposed methods can effectively and quickly mitigate the error propagation and the penalty on coding efficiency for clean channels due to the inserted error resilience features is rather minor.

Analysis of multihypothesis motion compensated prediction (MHMCP) for robust visual communication

A multihypothesis motion compensated prediction (MHMCP) scheme, which predicts a block from a weighted superposition of more than one reference blocks, is proposed and analyzed for error resilient visual communication in this research. By combining these reference blocks effectively, MHMCP can enhance the error resilient capability of compressed video as well as achieve a coding gain. In particular, we investigate the error propagation effect in the MHMCP coder and analyze the rate-distortion performance in terms of the hypothesis number and hypothesis coefficients. It is shown that MHMCP suppresses the short-term effect of error propagation more effectively than the intra-refreshing scheme.

Rate-constrained multihypothesis prediction for motion-compensated video compression

This paper investigates linearly combined motion-compensated signals for video compression. In particular, we discuss multiple motion-compensated signals that are jointly estimated for efficient prediction and video coding. First, we extend the wide-sense stationary theory of motion-compensated prediction (MCP) for the case of jointly estimated prediction signals. Our theory suggests that the gain by multihypothesis MCP is limited and that two jointly estimated hypotheses provide a major portion of this achievable gain. In addition, the analysis reveals a property of the displacement error of jointly estimated hypotheses. Second, we present a complete multihypothesis codec which is based on the ITU-T Recommendation H.263 with multiframe capability. Multiframe motion compensation chooses one prediction signal from a set of reference frames, whereas multihypothesis prediction chooses more than one for the linear combination. With our scheme, the time delay associated with B-frames is avoided by choosing more than one prediction signal from previously decoded pictures. Experimental results show that, multihypothesis prediction improves significantly coding efficiency by utilizing variable block size and multiframe motion compensation. We show that variable block size and multihypothesis prediction provide gains for different scenarios and that multiframe motion compensation enhances the multihypothesis gain. For example, the presented multihypothesis codec with ten reference frames improves coding efficiency by up to 2.7 dB when compared to the reference codec with one reference frame for the set of investigated test sequences.

Efficiency analysis of multihypothesis motion-compensated prediction for video coding

Overlapped block motion compensation or B-frames are examples of multihypothesis motion compensation where several motion-compensated signals are superimposed to reduce the bit-rate of a video codec. This paper extends the wide-sense stationary theory of motion-compensated prediction (MCP) for hybrid video codecs to multihypothesis motion compensation. The power spectrum of the prediction error is related to the displacement error probability density functions (pdfs) of an arbitrary number of hypotheses in a closed-form expression. We then study the influence of motion compensation accuracy on the efficiency of multihypothesis motion compensation as well as the influence of the residual noise level and the gain from optimal combination of N hypotheses. For the noise-free limiting case, doubling the number of (equally good) hypotheses can yield a gain of up to 1/2 bits/sample, while doubling the accuracy of motion compensation can additionally reduce the bit-rate by up to 1 bit/sample independent of N. For realistic noise levels, it is shown that the introduction of B-frames or overlapped block motion compensation can provide larger gains than doubling motion compensation accuracy. Spatial filtering of the motion-compensated candidate signals becomes less important if more hypotheses are combined. The critical accuracy beyond which the gain due to more accurate motion compensation is small moves to larger displacement error variances with increasing noise and increasing number of hypotheses N. Hence, sub-pel accurate motion compensation becomes less important with multihypothesis MCP. The theoretical insights are confirmed by experimental results for overlapped block motion compensation, B-frames, and multiframe motion-compensated prediction with up to eight hypotheses from ten previous frames.