Video Coding Experts Group Research Articles

Introduction to the Special Issue on HEVC Extensions and Efficient HEVC Implementations

High Efficiency Video Coding (HEVC) is the most recent standard in the series of major video coding standards jointly produced by the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). HEVC was first approved in 2013 in the ITU-T as Recommendation H.265 and in ISO/IEC as International Standard 23008-2, and it offers an unprecedented degree of compression capability for a very wide variety of applications. In the three years since its initial completion, it has been extended in several important ways to further broaden its scope. This special issue on HEVC features two sections: 1) HEVC extensions and 2) efficient HEVC implementations.

High Efficiency Video Coding

In recent years,with the development of video coding technology and the increasing popularity of high definition(HD)video content,the ISO/IEC Moving Picture Experts Group(MPEG)and the ITU-T Video Coding Experts Group(VCEG)formed the joint collaborative team on video coding(JCV-VC)in 2010which aims to develop the next-generation video coding standard for HD video application,called High Efficiency Video Coding(HEVC).So far the test model HM has been used for performance evaluation and algorithm test.The final international standard of HEVC has been completed and published in the end of 2012based on the working plan of JCV-VC.In this paper,the key technologies about HEVC will be surveyed,specifically including the quad-tree structure of coding unit,prediction unit and transform unit,the advanced motion vector prediction and merging technology,angular intra prediction,DCT-based fractional pixel interpolation filter and context adaptive arithmetic coding etc.Finally the coding performance of some coding tools and complexity of HEVC are analyzed in detail.

The H.264 Video Coding Standard

Since 1990, the Video Coding Expert Group (VCEG) and the Moving Pictures Expert Group (MPEG) have focused their research in video coding techniques for different applications. The International Telecommunication Union (ITU-T) and the MPEG groups joined in a single group to form the Joint Video Team (JVT) and worked together on a new video standard called H.264, H.264/AVC Advanced Video Coding, or MPEG-4 part 10, which achieves more coding efficiency with simpler syntax specifications than the previous standards, more integration with all network protocols, and multiplex architectures. The H.264 is the state-ofthe-art codec and covers a wide range of applications with excellent results such as videoconferencing, video streaming, and video transmission over fixed and wireless networks with different transport protocols among others.

Efficient residual data coding in CABAC for HEVC lossless video compression

After the development of the next generation video coding standard, referred to as high efficiency video coding (HEVC), the joint collaborative team of the ITU-T video coding experts group and the ISO/IEC moving picture experts group has now also standardized a lossless extension of such a standard. HEVC was originally designed for lossy video compression, thus, not ideal for lossless video compression. In this paper, we propose an efficient residual data coding method for HEVC lossless video compression. Based on the fact that there are statistical differences of residual data between lossy and lossless coding, we improved the HEVC lossless coding using sample-based angular prediction (SAP), modified level binarization, and binarization table selection with the weighted sum of previously encoded level values. Experimental results show that the proposed method provides high compression ratio up to 11.32 and reduces decoding complexity.

High efficiency video coding: the next frontier in video compression [Standards in a Nutshell

High Efficiency Video Coding (HEVC) is a new video compression standard developed jointly by ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Moving Pictures Expert Group (MPEG) through their Joint Collaborative Team on Video Coding (JCT-VC). The first version of HEVC will be finalized by the JCT-VC in January 2013. The HEVC project was launched to achieve major savings-e.g., reduction by about half for 1,280 × 720 high-definition (HD) and higher-resolution progressives can video-for equivalent visual quality relative to the bit rate needed by the widely used H.264/ MPEG-4 Advanced Video Coding (AVC) standard. For high resolution video where such additional compression is most urgently required, implementations of the current draft standard are already meeting or exceeding the targeted goal. We review the architecture and building blocks of HEVC, which were carefully selected with regard to compression capability versus complexity and to enable parallelism for the signal processing operations. Given the benefits that HEVC provides, it is likely to become the new primary reference for video compression.

HEVC: The New Gold Standard for Video Compression: How Does HEVC Compare with H.264/AVC?

Digital video has become ubiquitous in our everyday lives; everywhere we look, there are devices that can display, capture, and transmit video. The recent advances in technology have made it possible to capture and display video material with ultrahigh definition (UHD) resolution. Now is the time when the current Internet and broadcasting networks do not even have sufficient capacity to transmit large amounts of HD content-Let alone UHD. The need for an improved transmission system is more pronounced in the mobile sector because of the introduction of lightweight HD resolutions (such as 720 pixel) for mobile applications. The limitations of current technologies prompted the International Standards Organization/International Electrotechnical Commission Moving Picture Experts Group (MPEG) and International Telecommunication Union-Telecommunication Standardization Sector Video Coding Experts Group (VCEG) to establish the Joint Collaborative Team on Video Coding (JCT-VC), with the objective to develop a new high-performance video coding standard.

Game-based cross-layer channel allocation with SVC-encoded multimedia streams in cognitive radio networks

The available unlicensed spectrum is increasingly being used by new wireless technologies, but past measurements show that the licensed spectrum is extremely underutilized. To address this issue, the IEEE 802.22 Working Group is developing a novel wireless air interface standard based on cognitive radios (CRs), i.e. IEEE 802.22 wireless regional area networks (WRANs). Moreover, over the last decade wireless multimedia applications have developed rapidly, raising significant concerns about the quality of service (QoS) of multimedia stream transmissions. In particular, the Joint Video Team (JVT) and ITU-T Video Coding Experts Group (VCEG) jointly proposed Scalable Video Coding (SVC) as the next-generation multimedia compression standard. However, the current IEEE 802.22 WRAN draft does not specify QoS mechanisms for SVC-encoded multimedia stream transmission in CR networks. To resolve this problem, we developed a cross-layer channel allocation algorithm (CLCAA) and a novel media access control (MAC) protocol to work with the algorithm. The CLCAA adapts to the characteristics of multimedia traffic and variations of wireless channels by determining the weighting of source–destination pair, which is determined by the deadlines of SVC-encoded multimedia streams, the queuing delay and channel conditions. The CLCAA then allocates transmission opportunities to source–destination pairs based on their weightings and game theory. We also conducted extensive simulations to demonstrate the efficiency of the CLCAA scheme. The simulation results show that the CLCAA scheme not only guarantees QoS for multimedia traffic but also achieves fairness across different streams. Copyright © 2012 John Wiley & Sons, Ltd.

A Single-Pass-Based Localized Adaptive Interpolation Filter for Video Coding

Recently, an efficient coding tool named adaptive interpolation filtering (AIF) has been proposed to hybrid video coding scheme. By introducing Wiener filter into the fractional-pixel interpolation procedure, AIF can reduce the inter-prediction error and improve coding efficiency significantly. However, the training-based Wiener filter mechanism brings AIF an inherent multi-pass encoding structure, which imposes big burdens on the encoder in terms of huge computational complexity and memory access. In this paper, we propose a single-pass-based localized adaptive interpolation filtering (SPL-AIF) algorithm for video coding, which can reduce the complexity of AIF dramatically without sacrifice of its outstanding coding performance. The proposed SPL-AIF algorithm is based on the observation that there is a high correlation among optimal interpolation filters of consecutive frames, and different regions in a frame often possess different statistical characteristics. Accordingly, the proposed algorithm can be designed including two major parts. First, a competitive filter set which includes the optimal interpolation filters of several previous frames as well as the fixed H.264/AVC interpolation filters is built up for the coding of the current frame. Then a rate-distortion optimization criterion is used to select the best one at macroblock (MB) level. In order to reduce overhead, a predictive coding method is used to compress the filter signaling flag for each MB. Experimental results show that, by using the proposed algorithm, the encoding complexity can be reduced significantly while the average coding gain in Bjontegaard distortion bit-rate reduction can be improved about 1% compared with the multi-pass AIF. The proposed method has been adopted into the Video Coding Expert Group Key Technology Area software.

Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard

Significant improvements in video compression capability have been demonstrated with the introduction of the H.264/MPEG-4 advanced video coding (AVC) standard. Since developing this standard, the Joint Video Team of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG) has also standardized an extension of that technology that is referred to as multiview video coding (MVC). MVC provides a compact representation for multiple views of a video scene, such as multiple synchronized video cameras. Stereo-paired video for 3-D viewing is an important special case of MVC. The standard enables inter-view prediction to improve compression capability, as well as supporting ordinary temporal and spatial prediction. It also supports backward compatibility with existing legacy systems by structuring the MVC bitstream to include a compatible “base view.” Each other view is encoded at the same picture resolution as the base view. In recognition of its high-quality encoding capability and support for backward compatibility, the stereo high profile of the MVC extension was selected by the Blu-Ray Disc Association as the coding format for 3-D video with high-definition resolution. This paper provides an overview of the algorithmic design used for extending H.264/MPEG-4 AVC towards MVC. The basic approach of MVC for enabling inter-view prediction and view scalability in the context of H.264/MPEG-4 AVC is reviewed. Related supplemental enhancement information (SEI) metadata is also described. Various “frame compatible” approaches for support of stereo-view video as an alternative to MVC are also discussed. A summary of the coding performance achieved by MVC for both stereo- and multiview video is also provided. Future directions and challenges related to 3-D video are also briefly discussed.

High Performance, Low Complexity Video Coding and the Emerging HEVC Standard

This paper describes a low complexity video codec with high coding efficiency. It was proposed to the high efficiency video coding (HEVC) standardization effort of moving picture experts group and video coding experts group, and has been partially adopted into the initial HEVC test model under consideration design. The proposal utilizes a quadtree-based coding structure with support for macroblocks of size 64 × 64, 32 × 32, and 16 × 16 pixels. Entropy coding is performed using a low complexity variable length coding scheme with improved context adaptation compared to the context adaptive variable length coding design in H.264/AVC. The proposal's interpolation and deblocking filter designs improve coding efficiency, yet have low complexity. Finally, intra-picture coding methods have been improved to provide better subjective quality than H.264/AVC. The subjective quality of the proposed codec has been evaluated extensively within the HEVC project, with results indicating that similar visual quality to H.264/AVC High Profile anchors is achieved, measured by mean opinion score, using significantly fewer bits. Coding efficiency improvements are achieved with lower complexity than the H.264/AVC Baseline Profile, particularly suiting the proposal for high resolution, high quality applications in resource-constrained environments.

Analysis and Modeling of H.264 Unconstrained VBR Video Traffic

In future communication networks, video is expected to represent a large portion of the total traffic, given thatespecially variable bit rate (VBR) coded video streams, are becoming increasingly popular. Consequently, traffic modeling and characterization of such video services is essential for the efficient traffic control and resource management. Besides, providing an insight of video coding mechanisms, traffic models can be used as a tool for the allocation of network resources, the design of efficient networks for streaming services and the reassurance of specific QoS characteristics to the end users. The new H.264/AVC standard, proposed by the ITU-T Video Coding Expert Group (VCEG) and ISO/IEC Moving Pictures Expert Group (MPEG), is expected to dominate in upcoming multimedia services, due to the fact that it outperforms in many fields the previous encoded standards. This article presents both a frame and a layer (i.e. I, P and B frames) level analysis of H.264 encoded sources. Analysis of the data suggests that the video traffic can be considered as a stationary stochastic process with an autocorrelation function of exponentially fast decay and a marginal frame size distribution of approximately Gamma form. Finally, based on the statistical analysis, an efficient model of H.264 video traffic is proposed.

Error Resilient Coding and Error Concealment in Scalable Video Coding

Scalable video coding (SVC), which is the scalable extension of the H.264/AVC standard, was developed by the Joint Video Team (JVT) of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group). SVC is designed to provide adaptation capability for heterogeneous network structures and different receiving devices with the help of temporal, spatial, and quality scalabilities. It is challenging to achieve graceful quality degradation in an error-prone environment, since channel errors can drastically deteriorate the quality of the video. Error resilient coding and error concealment techniques have been introduced into SVC to reduce the quality degradation impact of transmission errors. Some of the techniques are inherited from or applicable also to H.264/AVC, while some of them take advantage of the SVC coding structure and coding tools. In this paper, the error resilient coding and error concealment tools in SVC are first reviewed. Then, several important tools such as loss-aware rate-distortion optimized macroblock mode decision algorithm and error concealment methods in SVC are discussed and experimental results are provided to show the benefits from them. The results demonstrate that PSNR gains can be achieved for the conventional inter prediction (IPPP) coding structure or the hierarchical bi-predictive (B) picture coding structure with large group of pictures size, for all the tested sequences and under various combinations of packet loss rates, compared with the basic joint scalable video model (JSVM) design applying no error resilient tools at the encoder and only picture copy error concealment method at the decoder.

Error Resilient Coding and Error Concealment in Scalable Video Coding

Scalable video coding (SVC), which is the scalable extension of the H.264/AVC standard, was developed by the Joint Video Team (JVT) of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group). SVC is designed to provide adaptation capability for heterogeneous network structures and different receiving devices with the help of temporal, spatial, and quality scalabilities. It is challenging to achieve graceful quality degradation in an error-prone environment, since channel errors can drastically deteriorate the quality of the video. Error resilient coding and error concealment techniques have been introduced into SVC to reduce the quality degradation impact of transmission errors. Some of the techniques are inherited from or applicable also to H.264/AVC, while some of them take advantage of the SVC coding structure and coding tools. In this paper, the error resilient coding and error concealment tools in SVC are first reviewed. Then, several important tools such as loss-aware rate-distortion optimized macroblock mode decision algorithm and error concealment methods in SVC are discussed and experimental results are provided to show the benefits from them. The results demonstrate that PSNR gains can be achieved for the conventional inter prediction (IPPP) coding structure or the hierarchical bi-predictive (B) picture coding structure with large group of pictures size, for all the tested sequences and under various combinations of packet loss rates, compared with the basic joint scalable video model (JSVM) design applying no error resilient tools at the encoder and only picture copy error concealment method at the decoder.

인접 화소 성분 분석을 이용한 H.264/AVC에서의 Intra 예측 정보 생략

Moving Picture Experts Group (MPEG)과 Video Coding Experts Group (VCEG)은 기존의 MPEG-4와 H.263 표준안보다 우수한 비디오 압축 기술을 개발하였다. 이 새로운 표준안은 H.264/AVC (Advanced Video Coding) 이라 하며, MPEG-4 Part 10과 ITU-T Recommendation H.264로 공동 발표 되었다. H.264/AVC에는 9가지의 모드를 가진 <TEX>$Intra4{\times}4$</TEX> 예측 부호화가 존재하는데, 이는 영상의 공간적인 상관성을 이용한 압축 부호화 방식이다. 본 논문에서는 압축 효율을 높이기 위하여 인접 화소들의 유사성을 이용한 ABS (Adaptive Bit Skip) 모드를 제안함으로써, 적응적으로 intra mode bit를 생략하여 부호화 효율을 높이는 방법을 제안하였다. 실험 결과, 제안한 방법은 기존의 방법과 비교하여 R-D 곡선 상에서 평균 약 0.2 dB의 PSNR (Peak signal-to-noise ratio) 향상이 있었고 평균 약 3.6%의 비트율을 절약하였다. The Moving Picture Experts Group (MPEG) and Video Coding Experts Group (VCEG) have developed a new standard that promises to outperform the earlier MPEG-4 and H.263 standards. The new standard is called H.264/AVC (Advanced Video Coding) and is published jointly as MPEG-4 Part 10 and ITU-T Recommendation H.264. In particular, the H.264/AVC intra prediction coding provides nine directional prediction modes for every <TEX>$4{\times}4$</TEX> block in order to reduce spatial redundancies. In this paper, an ABS (Adaptive Bit Skip) mode is proposed. In order to achieve coding efficiency, the proposed method can remove the mode bits to represent the prediction mode by using the similarity of adjacent pixels. Experimental results show that the proposed method achieves the PSNR gain of about 0.2 dB in R-D curve and reduces the bit rates about 3.6% compared with H.264/AVC.

Evaluation of the H.264 Scalable Video Coding in Error Prone IP Networks

The Joint Video Team, composed by the ISO/IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG), has standardized a scalable extension of the H.264/AVC video coding standard called scalable video coding (SVC). H.264/SVC provides scalable video streams which are composed by a base layer and one or more enhancement layers. Enhancement layers may improve the temporal, the spatial or the signal-to-noise ratio resolutions of the content represented by the lower layers. One of the applications of this standard is related to video transmission in both wired and wireless communication systems, and it is therefore important to analyze in which way packet losses contribute to the degradation of quality, and which mechanisms could be used to improve that quality. This paper provides an analysis and evaluation of H.264/SVC in error prone environments, quantifying the degradation caused by packet losses in the decoded video. It also proposes and analyzes the consequences of QoS-based discarding of packets through different marking solutions.

스케일러블 비디오 부호화의 개요 및 성능 분석

근래의 무선망 및 인터넷의 초고속화에 따라 풍부한 멀티미디어에 대한 사용자의 요구가 증대되고 있으며, 이동 통신의 활성화 및 DMB(Digital Media Broadcasting)와 같은 통방융합 서비스의 등장으로 여러 네트워크 망을 통해 다양한 성능을 가진 단말로의 멀티미디어 서비스가 이루어지고 있다. 이러한 환경 변화에 따라 비트율을 줄이기 위한 비디오 압축 부호화 기술뿐만 아니라 시변의 전송 조건, 전송망 특성, 다양한 단말 성능, 사용자 선호도 등에 대한 비디오의 적응 기술에 관심이 모아졌으며, 그 결과 ISO/IEC MPEG(Moving Picture Experts Group) 및 ITU-T VCEG(Video Coding Experts Group)의 JVT(Joint Video Team)에서는 비디오 부호화의 스케일러빌러티(scalbility)를 제공하기 위한 SVC(Scalable Video Coding) 표준화를 진행하였다. 본 논문에서는 공간/시간/화질적 스케일러빌러티를 지원하기 위한 SVC의 대표적 표준 기술들에 대해 설명하고, 기존의 단일 계층(single layer) 부호화 방식과 비교하여 SVC의 부가 비트 요구량 및 부호화 효율에 대해 성능 평가를 하였다. Seamless streaming of multimedia content via heterogeneous networks to viewers using a variety of devices has been a desire for many multimedia services, for which the multimedia contents should be adapted to usage environments such as network characteristics, terminal capabilities, and user preferences. Scalability in video coding is one of attractive features to meet dynamically changing requirements of heterogeneous networks. Currently a new scalable video coding (SVC) is standardizing in the Joint Video Team (JVT) of the ISO/IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG), which will be released as Extension 3 of H.264/MPEG-4 AVC. In this paper, we introduce new technologies of SVC and evaluate performance of it especially regarding on overhead bit-rate and coding efficiency to support spatial, temporal, and quality scalability.

Interlaced Coding in SVC

The scalable extension of AVC scalable video coding (SVC) is a current standardization project of the joint video team (JVT) of the ITU-T video coding experts group (VCEG) and the ISO/IEC moving picture experts group (MPEG). SVC has been initially designed for progressive video. However, even if progressive material is becoming the favorite format for production, broadcasting and consumers equipments, interlaced material is still widely used in the video world and will not disappear in the next few years. The specification of SVC should be finalized in 2007. Thanks to its AVC-based design and its AVC base-layer compatibility, migration from AVC to SVC equipments can be considered in a near future. In this context, interlaced support becomes an important requirement for SVC. This paper presents the main concepts for supporting interlaced coding in SVC. After a brief description of the basics of SVC, the generalizations of AVC interlaced tools are first described. Then main issues related to interlaced video scalable encoding are identified and the new mechanisms introduced in the SVC specification for raising these issues are presented. The paper also discusses related applications side and identifies several use cases illustrating the interest of interlaced support in SVC.

Dynamic and interoperable adaptation of SVC for QoS-enabled streaming

Seamless streaming of multimedia content that ensures Quality of Service (QoS) over heterogeneous networks has been a desire for many multimedia services, for which the multimedia contents should be adapted to usage environments such as network characteristics, terminal capabilities, and user preferences. Scalability in video coding is a good feature to meet the requirement of heterogeneous networks. Currently, the Joint Video Team (JVT) of the ISO/IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG) is standardizing a new scalable video coding standardization, called as SVC, which will become an extension to H.264/MPEG-4 AVC. In this paper, we propose a dynamic adaptation scheme of SVC bit-stream using the MPEG-21 Digital Item Adaptation (DIA) tool. MPEG-21 DIA framework provides systematic solutions in choosing the optimal adaptation operation to given conditions and supports interoperable video adaptation. The experiment results showed that the proposed adaptation scheme provides QoS-enabled delivery and consumption of SVC with time-varying constraints of network, terminal, and user preference, in a robust and efficient way. In addition, session mobility, in which a session is movable from one type of device to another type of device, was performed using switching spatial layer of SVC stream'.

The H.264/MPEG4 advanced video coding standard and its applications

H.264/MPEG4-AVC is the latest video coding standard of the ITU-T video coding experts group (VCEG) and the ISO/IEC moving picture experts group (MPEG). H.264/MPEG4-AVC has recently become the most widely accepted video coding standard since the deployment of MPEG2 at the dawn of digital television, and it may soon overtake MPEG2 in common use. It covers all common video applications ranging from mobile services and videoconferencing to IPTV, HDTV, and HD video storage. This article discusses the technology behind the new H.264/MPEG4-AVC standard, focusing on the main distinct features of its core coding technology and its first set of extensions, known as the fidelity range extensions (FRExt). In addition, this article also discusses the current status of adoption and deployment of the new standard in various application areas

Flexible macroblock ordering in H.264/AVC

H.264/AVC is a new standard for digital video compression jointly developed by ITU-T’s Video Coding Experts Group (VCEG) and ISO/IEC’s Moving Picture Experts Group (MPEG). Besides the numerous tools for efficient video coding, the H.264/AVC specification defines some new error resilience tools. One of them is flexible macroblock ordering (FMO) which is the main focus of this paper. An in-depth overview is given of the internals of FMO. Experiments are presented that demonstrate the benefits of FMO as an error resilience tool in case of packet loss over IP networks. The flexibility of FMO comes with a certain overhead or cost. A quantitative assessment of this cost is presented for a number of scenarios. FMO can, besides for pure error resilience, also be used for other purposes. This is also addressed in this paper.