Joint Video Team Research Articles

Advances in the scalable amendment of H.264/AVC

To support clients with diverse capabilities, ISO/IEC MPEG and ITU-T form a joint video team (JVT) to develop a scalable video coding (SVC) technology that uses single bitstream to provide multiple spatial, temporal, and quality (SNR) resolutions, thus satisfying low-complexity and low-delay constraints. It is an amendment of the emerging standard H.264/AVC and it provides an H.264/AVC-compatible base layer and a fully scalable enhancement layer, which can be truncated and extracted on-the-fly to obtain a preferred spatio-temporal and quality resolution. An overview of the adopted key technologies in the SVC and a comparison in coding efficiency with H.264/AVC are presented

Rate-distortion analysis and streaming of SP and SI frames

The new SP and SI picture types, introduced in the latest video coding standard H.264, allow drift-free bitstream switching and can also be used for error-resilience and random access. In this paper, we propose a model for the rate-distortion performance of SP and SI pictures and compare it to experimental results, obtained with our implementation of an SP/SI encoder, made publicly available and recently adopted by the Joint Video Team. The model predicts how the relative sizes of SP and SI slices can be traded off. We analyze, both theoretically and experimentally, how this can be used to minimize the transmitted bit rate when SP frames are used for video streaming with packet losses and derive optimal settings for our encoder. We investigate the benefits of SI and SP frames for error resilience as compared with periodic I frame insertion. Empirical rate-distortion curves predict rate-distortion gains may be obtained. Experiments carried out over a simulated throughput-limited network confirm this to be the case when the end-to-end delay is limited. We analyze the influence of loss rate and delay on the congestion-rate-distortion performance of streaming with SI and SP frames. Our results identify scenarios for which SI and SP frames provide an attractive alternative to streaming with I frames.

On Lagrange multiplier and quantizer adjustment for H.264 frame-layer video rate control

H.264/AVC encoder employs a complex mode-decision technique based on rate-distortion optimization. It calculates rate-distortion cost (RDcost) for all possible modes to choose the best one having the minimum RDcost. This paper presents a frame-layer rate control for H.264/AVC that computes the Lagrange multiplier (/spl lambda//sub MODE/) for mode decision by using a quantization parameter (QP) which may be different from that used for encoding. At the same time, we also compare actual bits produced by previous macroblocks (MBs) with the total bits allocated to these MBs to further modify /spl lambda//sub MODE/. The objective of these measures aims to produce bits as close to the frame target bits for rate control as possible. This is very important in the case of low-bit-rate tight buffer applications. In order to obtain an accurate QP for a frame, we employ a complexity-based bit-allocation scheme and a QP adjustment method. Simulation results comparing with the H.264 Joint Video Team (JVT) rate control method show that the H.264 encoder, using the proposed algorithm, achieves a visual quality improvement of about 0.56 dB, performs better for buffer overflow and underflow, and achieves a smaller PSNR deviation.

RTP payload format for H.264/SVC scalable video coding

The scalable extension of H.264/AVC, known as scalable video coding or SVC, is currently the main focus of the Joint Video Team’s work. In its present working draft, the higher level syntax of SVC follows the design principles of H.264/AVC. Self-contained network abstraction layer units (NAL units) form natural entities for packetization. The SVC specification is by no means finalized yet, but nevertheless the work towards an optimized RTP payload format has already started. RFC 3984, the RTP payload specification for H.264/AVC has been taken as a starting point, but it became quickly clear that the scalable features of SVC require adaptation in at least the areas of capability/operation point signaling and documentation of the extended NAL unit header. This paper first gives an overview of the history of scalable video coding, and then reviews the video coding layer (VCL) and NAL of the latest SVC draft specification. Finally, it discusses different aspects of the draft SVC RTP payload format, including the design criteria, use cases, signaling and payload structure.

A Software-Hardware Co-Implementation of MPEG-4 Advanced Video Coding (AVC) Decoder with Block Level Pipelining

We present a baseline MPEG-4 Advanced Video Coding (AVC) decoder based on the methodology of joint optimization of software and hardware. The software is first optimized with algorithm improvements for frame buffer management, boundary padding, content-aware inverse transform and context-based entropy decoding. The overall decoding throughput is further enhanced by pipelining the software and the dedicated hardware at macroblock level. The decoder is partitioned into the software and hardware modules according to the target frame rate and complexity profiles. The hardware acceleration modules include motion compensation, inverse transform and loop filtering. By comparing the optimized decoder with the committee reference decoder of Joint Video Team (JVT), the experimental results show improvement on the decoding throughput by 7 to 8 times. On an ARM966 board, the optimized software without hardware acceleration can achieve a decoding rate up to 5.9 frames per second (fps) for QCIF video source. The overall throughput is improved by another 27% to 7.4 fps on the average and up to 11.5 fps for slow motion video sequences. Finally, we provide a theoretical analysis of the ideal performance of the proposed decoder.

Performance and Complexity Co-evaluation of the Advanced Video Coding Standard for Cost-Effective Multimedia Communications

The advanced video codec (AVC) standard, recently defined by a joint video team (JVT) of ITU-T and ISO/IEC, is introduced in this paper together with its performance and complexity co-evaluation. While the basic framework is similar to the motion-compensated hybrid scheme of previous video coding standards, additional tools improve the compression efficiency at the expense of an increased implementation cost. As a first step to bridge the gap between the algorithmic design of a complex multimedia system and its cost-effective realization, a high-level co-evaluation approach is proposed and applied to a real-life AVC design. An exhaustive analysis of the codec compression efficiency versus complexity (memory and computational costs) project space is carried out at the early algorithmic design phase. If all new coding features are used, the improved AVC compression efficiency (up to 50% compared to current video coding technology) comes with a complexity increase of a factor 2 for the decoder and larger than one order of magnitude for the encoder. This represents a challenge for resource-constrained multimedia systems such as wireless devices or high-volume consumer electronics. The analysis also highlights important properties of the AVC framework allowing for complexity reduction at the high system level: when combining the new coding features, the implementation complexity accumulates, while the global compression efficiency becomes saturated. Thus, a proper use of the AVC tools maintains the same performance as the most complex configuration while considerably reducing complexity. The reported results provide inputs to assist the profile definition in the standard, highlight the AVC bottlenecks, and select optimal trade-offs between algorithmic performance and complexity.

Rate control using linear rate- ρ model for H.264

The paper models relations of rate– ρ and QP– ρ, where ρ is defined as the percentage of zero coefficients in a frame. These two models are used for a frame-level rate-control of H.264. A linear approximation scheme is adopted to model the rate– ρ relations for the rate-control. The proposed frame-level rate-control method exploits rate-distortion optimization (RDO) to estimate macroblock modes and bitrates for the initial quantization parameter (QP). An intra-rate model is also designed to determine an initial QP for the intra-frame. In experimental results, peak signal-to-noise ratio, bitrate estimation error, rate-control accuracy in the scene changes, and computational complexity of the proposed method are analyzed for various video sequences. According to the experimental analysis, the proposed method outperforms the existing rate-control method (Joint Video Team of ISO/IEC and ITV-T the Fifth Meeting, JVT-DO69, Geneva, Switzerland, 9–17 October 2002).

Advanced video coding

Advanced video coding techniques will allow broadcasters to improve the efficiency offered by MPEG-2 digital TV systems by up to 75%. The opportunity for significant improvements in MPEG-2 performance is now very limited. MPEG-2 solutions are likely to become more dense, lower power and inexpensive, but they are not going to produce much lower bit rates. It is fortunate that, also during the last ten years, Moore's Law - which states that semiconductor/computer processing power increases by a factor of two every eighteen months - has also progressed relentlessly. More processing power at a lower price has enabled a whole new generation of compression technology. Both set-top boxes and headend equipment are now capable of much more complex calculations at commercially acceptable prices than were possible at the time of MPEG-2 standardisation. The two most significant algorithms/systems proposed currently are Windows Media 9 Series, from Microsoft, and MPEG-4 part 10, an open standards algorithm from the ITU/MPEG Joint Video Team. Both these systems draw heavily on the experience of MPEG-2, but add extra tools to improve the coding efficiency - particularly at low bit rates.

Fast loop filtering using separable characteristics of the integer 2-D discrete cosine transform

When an image is highly compressed, video coding using the discrete cosine transform (DCT) and quantization produces noticeable image degradations, such as blocking artifacts and ringing noise. In order to reduce the degradations, a loop filtering algorithm is proposed, which consists of simple deblocking filtering and a fast decision rule for ringing and blocking conditions. The proposed method utilizes a four-point integer DCT for detecting blocking and ringing conditions. It also adopts an early termination policy during detection of the blocking and ringing conditions in the DCT domain. This policy speeds up computation of the filtering. The proposed method is compared with the loop filtering of the Joint Video Team (JVT) codec. As an experimental result, the computation time of the proposed method is approximately 20% faster than that of the JVT loop filtering, while the PSNR is almost the same.