Joint Video Team Research Articles

MVP2P: Layer-dependency-aware live MVC video streaming over peer-to-peer networks

Multiview video supports observing a scene from different viewpoints. The Joint Video Team (JVT) developed H.264/MVC to enhance the compression efficiency for multiview video, however, MVC encoded multiview video (MVC video) still requires high bitrates for transmission. This paper investigates live MVC video streaming over Peer-to-Peer (P2P) networks. The goal is to minimize the server bandwidth costs whist ensuring high streaming quality to peers. MVC employs intra-view and inter-view prediction structures, which leads to a complicated layer dependency relationship. As the peers’ outbound bandwidth is shared while supplying all the MVC video layers, the bandwidth allocation to one MVC layer affects the available outbound bandwidth of the other layers. To optimize the utilization of the peers’ outbound bandwidth for providing video layers, a maximum flow based model is proposed which considers the MVC video layer dependency and the layer supplying relationship between peers. Based on the model, a layer dependency aware live MVC video streaming method over a BitTorrent-like P2P network is proposed, named MVP2P. The key components of MVP2P include a chunk scheduling strategy and a peer selection strategy for receiving peers, and a bandwidth scheduling algorithm for supplying peers. To evaluate the efficiency of the proposed solution, MVP2P is compared with existing methods considering the constraints of peer bandwidth, peer numbers, view switching rates, and peer churns. The test results show that MVP2P significantly outperforms the existing methods.

Fast coding unit selection and motion estimation algorithm based on early detection of zero block quantified transform coefficients for high‐efficiency video coding standard

High-efficiency video coding (HEVC) is the newest video coding standard developed by the joint video team, consisting of ITU-T video coding experts group and ISO/IEC Moving Picture Experts Group. The HEVC standard has aggregated an exhaustive algorithm for mode decision based on a recursive quad-tree structured coding tree block. Moreover, several specific features have been incorporated into the motion estimation (ME) process to improve its coding efficiency. However, they resulted in very high computational complexity. To accelerate the encoding process, fast mode decision algorithms for the partitioning module and also for the ME module were proposed in this study. These algorithms are based on early zero block detection technique. To improve the efficiency of these algorithms, an overall algorithm which combines the two techniques has been implemented. The performance of the proposed algorithm was checked through a comparative analysis in terms of encoding time and compression rate. Compared to HEVC test model 10.0, the authors’ proposed algorithms bring a great reduction of the HEVC complexity encoder with a saving time, which can reach 25% in average for different tested videos and a slight coding loss in terms of image quality and compression rate.

An Adaptive Error Concealment Method for Depth Map in 3D Video Coding

Stereoscopic images and video are synthesized from virtual views, which are produced from a real view and its depth map. However, this process involves a large amount of data which not only increases the transmission load, but also must be stored within the display devices which therefore must possess huge storage capacities. To solve these problems, the Joint Video Team has developed the 3D video coding (3DVC) standard to reduce the redundant data for multiview video and its depth map through the correlation between the real view and its corresponding depth map. When depth data in the 3DVC bitstream are lost during transmission, it causes the decoded depth map to be incomplete. Then the quality of the decoded depth map is reduced and the quality of the stereoscopic video will also be decreased. To increase the reconstructed quality of the damaged macro-block, this paper proposes an adaptive error concealment selection to determine the most suitable error concealment method based on fuzzy reasoning. The temporal and spatial correlations of the damaged macro-block which are defined by the motion vectors of neighboring blocks and their depth information are used for the adaptive selection. From the simulation result, the proposed algorithm can reduce the concealing time by at least 0.15–2.86 frame/sec and can improve quality by about 0.13–0.17 dB. The proposed algorithm is suitable for the 3DVC video transmission.

Temporal and Inter-View Consistent Error Concealment Technique for Multiview Plus Depth Video

Multiview plus depth (MVD) is an emerging video format with many applications, including 3-D television and free viewpoint television. During the broadcast of a compressed MVD video, transmission errors may cause the loss of whole frames, resulting in significant degradation of video quality. Error concealment techniques have been widely used to deal with transmission errors in video communication. However, the existing solutions do not address the requirement that the reconstructed frames should be consistent with neighboring frames, i.e., corresponding pixels should have consistent color information. We propose a new consistency model for error concealment of MVD video that allows one to maintain a high level of consistency between frames of the same view (temporal consistency) and those of neighboring views (inter-view consistency). We then propose an algorithm that uses our model to implement concealment in a consistent way. Simulations with the reference software for the multiview video coding project of the joint video team of the ISO/IEC MPEG and ITU-T VCEG show that our method outperforms benchmark techniques, including a baseline approach based on the boundary matching algorithm, with respect to both reconstruction quality and view consistency.

IJARCCE - Computer and Communication Engineering

This paper presents an error detection and data recovery (EDDR) design, based on the residue-and-quotient (RQ) code, to embed into motion estimation (ME) for video coding testing applications. An error in processing elements (PEs), i.e. key components of a ME, can be detected and recovered effectively by using the EDDR design. The proposed EDDR design for ME testing can detect errors and recover data with an acceptable area overhead and timing penalty. The functional verification and synthesis can be done by Xilinx ISE. That is when compare to the existing design the implemented design area and timing will be reduced.. The new Joint Video Team (JVT) video coding standard has garnered increased attention recently. Generally, motion estimation computing array (MECA) performs up to 50% of computations in the entire video coding system, and is typically considered the computationally most important part of video coding systems. Thus, integrating the MECA into a system-on-chip (SOC) design has become increasingly important for video coding applications. Although advances in VLSI technology allow integration of a large number of processing elements (PEs) in an MECA into an SOC, this increases the logic- per-pin ratio, thereby significantly decreasing the efficiency of chip logic testing. For a commercial chip, a video coding system must introduce design for testability (DFT), especially in an MECA. The objective of DFT is to increase the ease with which a device can be tested to guarantee high system reliability. Many DFT approaches have been developed. These approaches can be divided into three categories: ad hoc (problem oriented), structured, and built-in self-test (BIST). Among these techniques, BIST has an obvious advantage in that expensive test equipment is not needed and tests are low cost.

Performance analysis of inter-layer prediction module for H.264/SVC

With the introduction of diverse variety of display resolutions and transmission channel capacities, the Joint Video Team (JVT) has developed a new scalable video coding standard (H264/SVC) as an extension of H.264/AVC. The SVC can compress numerous video sequences of various resolutions as a single bit-stream. The spatial scalability is mainly based on coding tools such as inter-layer predictions that offer a better coding efficiency. Consequently, for the enhancement layer, a thorough mode decision process based on the base layer mode predictions is performed on the Joint Scalable Video Model software implementation. In order to highlight the inter-layer's coding efficiency, several analyses were made using numerous HD sequences with different motion characteristics. In fact, in this paper we present the efficiency of different inter-layer prediction modes for spatial scalability. As a first step, analysis results show the significant role of inter-layer prediction process for coding efficiency. As a second step, role of each inter-layer prediction type is investigated. Simulations results show the important role of inter-layer intra prediction and inter-layer residual prediction when targeting a better coding efficiency.

Multiuser Successive Maximum Ratio Transmission (MS-MRT) for Video Quality Maximization in Unicast and Broadcast MIMO OFDMA-Based 4G Wireless Networks

In this paper, we propose a novel low-complexity beamforming algorithm for multiuser successive maximum ratio transmission (MS-MRT) toward space-division multiplexing to achieve video quality maximization in the downlink of a multiuser (MU) multiple-input-multiple-output (MIMO) orthogonal frequency-division multiple-access (OFDMA)-based fourth-generation (4G) wireless network. We compare the performance of this algorithm in terms of system throughput and video quality with the two popular precoding techniques, namely, block diagonalization (BD) and successive optimization (SO). We also compare its performance with the generalized versions of these two algorithms termed coordinated transmit-receive processing (CTR)-BD and CTR-SO algorithms, respectively. Further, we propose an extension of the MS-MRT scheme to broadcast scenarios, termed broadcast successive maximum ratio transmission (BS-MRT), which computes the optimal broadcast beamforming vector to maximize the video quality at each of the broadcast group members while maintaining orthogonality to the previously scheduled user groups. We also demonstrate that the proposed MS-MRT and BS-MRT schemes can be naturally adapted to the context of various MU OFDMA scheduling algorithms such as proportional fairness (PF) and round robin (RR), to name a few, to maximize video quality through optimal scheduling in the presence of a large number of users. We discuss implementations of the proposed MS-MRT scheme in association with these scheduling algorithms for MU unicast and broadcast video transmission scenarios. The simulation results presented in this work rely on the rate models derived using the Joint Scalable Video Model (JSVM) software developed by the Joint Video Team (JVT) and are, thus, readily applicable in practice. The presented results clearly demonstrate the ability of the proposed algorithm to maximize the video quality in comparison with the other competing MU MIMO precoding techniques. Further, employing the proportional fair scheduling algorithm in conjunction with the MS-MRT results in an overall enhancement in the received video quality compared with the RR and max-rate algorithms.

The H.264 Video Coding Standard

Since 1990, the Video Coding Expert Group (VCEG) and the Moving Pictures Expert Group (MPEG) have focused their research in video coding techniques for different applications. The International Telecommunication Union (ITU-T) and the MPEG groups joined in a single group to form the Joint Video Team (JVT) and worked together on a new video standard called H.264, H.264/AVC Advanced Video Coding, or MPEG-4 part 10, which achieves more coding efficiency with simpler syntax specifications than the previous standards, more integration with all network protocols, and multiplex architectures. The H.264 is the state-ofthe-art codec and covers a wide range of applications with excellent results such as videoconferencing, video streaming, and video transmission over fixed and wireless networks with different transport protocols among others.

A Whole Rate-Distortion Optimize Pronounce Based on Wireless Sensor Network

Based on the Joint Video Team (JVT) of the ITU-T Video Coding Experts Group VC EG and the IS O/IEC Moving Picture Experts Group MPEG, an RD optimal Macro Block mode decision scheme for Internet error channel streaming is introduced. The scheme employs the luminance Rate Distortion (RD) optimal mode decision scheme so as to take the effects of video encoding distortion and the channel error propagation to get higher error robustness for error transmission. Based on the Wireless Sensor Network, this paper analyzes the data distortion problem when transmitting H.264 coded video stream over error-prone channel. And the authors also have discussed a widely accepted technique that introduces more intra-coded information on macro block basis. Additionally, this paper introduces a simple loss and multiplication factor estimation method, the rate-distortion optimized assessing strategy over the whole situation.

A Novel Method for Joint Video by Using Meca

The new Joint Video Team (JVT) video coding standard has garnered increased attention recently. Generally, motion estimation computing array (MECA) performs up to 50% of computations in the entire video coding system, and is typically considered the computationally most important part of video coding systems. For a commercial chip, a video coding system must introduce design for testability (DFT), especially in an MECA. The objective of DFT is to increase the ease with which a device can be tested to guarantee high system reliability. Among these techniques, BIST has an obvious advantage in that expensive test equipment is not needed and tests are low cost. Moreover, BIST can generate test simulations and analyze test responses without outside support, making tests and diagnoses of digital systems quick and effective. However, as the circuit complexity and density increases, the BIST approach must detect the presence of faults and specify their locations for subsequent repair. The extended techniques of BIST are built-in self-diagnosis and built-in self-re- pair (BISR). This work develops a built-in self-detection/correction (BISDC) architecture for motion estimation computing arrays (MECAs). Based on the error detection/correction concepts of biresidue codes, any single error in each processing element in an MECA can be effectively detected and corrected online using the proposed BISD and built-in self-correction circuits. Performance analysis and evaluation demonstrate that the proposed BISDC architecture performs well in error detection and correction with minor area overhead. Keywords: Data recovery, error detection, motion estimation, reliability, residue-and-quotient (RQ) code

Game-based cross-layer channel allocation with SVC-encoded multimedia streams in cognitive radio networks

The available unlicensed spectrum is increasingly being used by new wireless technologies, but past measurements show that the licensed spectrum is extremely underutilized. To address this issue, the IEEE 802.22 Working Group is developing a novel wireless air interface standard based on cognitive radios (CRs), i.e. IEEE 802.22 wireless regional area networks (WRANs). Moreover, over the last decade wireless multimedia applications have developed rapidly, raising significant concerns about the quality of service (QoS) of multimedia stream transmissions. In particular, the Joint Video Team (JVT) and ITU-T Video Coding Experts Group (VCEG) jointly proposed Scalable Video Coding (SVC) as the next-generation multimedia compression standard. However, the current IEEE 802.22 WRAN draft does not specify QoS mechanisms for SVC-encoded multimedia stream transmission in CR networks. To resolve this problem, we developed a cross-layer channel allocation algorithm (CLCAA) and a novel media access control (MAC) protocol to work with the algorithm. The CLCAA adapts to the characteristics of multimedia traffic and variations of wireless channels by determining the weighting of source–destination pair, which is determined by the deadlines of SVC-encoded multimedia streams, the queuing delay and channel conditions. The CLCAA then allocates transmission opportunities to source–destination pairs based on their weightings and game theory. We also conducted extensive simulations to demonstrate the efficiency of the CLCAA scheme. The simulation results show that the CLCAA scheme not only guarantees QoS for multimedia traffic but also achieves fairness across different streams. Copyright © 2012 John Wiley & Sons, Ltd.

H.264/AVC rate control with enhanced rate-quantisation model and bit allocation

Rate control algorithms aim to achieve perceivable video quality for real-time video communication applications under some real-world constraints, such as bandwidth, delay, buffer sizes etc. This study presents an adaptive and efficient rate control framework for low-delay H.264/AVC video communication. At the basic unit level, to realise quantisation decision improvement, a novel rate–qantisation (R–Q) model is proposed for P frame. This P frame R–Q model is established through the ρ domain rate control theory to compute quantisation according to target bit assignment. In addition, an I frame R–Q model is presented to measure frame level coding activity for quantisation decision without performing computationally intensive intra-prediction, which helps alleviating the frame skipping problem. When compared with Joint Video Team (JVT)-G012, a rate control scheme adopted by JVT H.264/AVC reference software JM12.3, the proposed rate control framework shows good performance in terms of improved average luminance peak signal noise ratio (PSNR), reduced number of skipped frames and smaller mismatch between target and actual bit rates.

Fast H.264 Encoding Based on Statistical Learning

H.264/AVC, the latest video coding standard of the Joint Video Team, greatly outperforms previous standards in terms of coding bitrate and video quality, because it adopts several new techniques. However, the computational complexity is also considerably increased due to these new components. In this paper, we propose fast algorithms based on statistical learning to reduce the computational cost involved in three main components in H.264 encoder, i.e., intermode decision, multi-reference motion estimation (ME), and intra-mode prediction. First, representative features are extracted to build the learning models. Then, an offline pre-classification approach is used to determine the best results from the extracted features, thus a significant amount of computation is reduced based on the classification strategy. The proposed statistical learning-based approach is applied to the aforementioned three main components in H.264 encoder to speed up the computation. Experimental results show that the ME time of the proposed system is significantly sped up with 12 times faster than the conventional fast ME algorithm of H.264, and the total encoding time of the proposed encoder is greatly reduced with about four times faster than the fast encoder EPZS in the H.264 reference code with negligible video quality degradation.

Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard

Significant improvements in video compression capability have been demonstrated with the introduction of the H.264/MPEG-4 advanced video coding (AVC) standard. Since developing this standard, the Joint Video Team of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG) has also standardized an extension of that technology that is referred to as multiview video coding (MVC). MVC provides a compact representation for multiple views of a video scene, such as multiple synchronized video cameras. Stereo-paired video for 3-D viewing is an important special case of MVC. The standard enables inter-view prediction to improve compression capability, as well as supporting ordinary temporal and spatial prediction. It also supports backward compatibility with existing legacy systems by structuring the MVC bitstream to include a compatible “base view.” Each other view is encoded at the same picture resolution as the base view. In recognition of its high-quality encoding capability and support for backward compatibility, the stereo high profile of the MVC extension was selected by the Blu-Ray Disc Association as the coding format for 3-D video with high-definition resolution. This paper provides an overview of the algorithmic design used for extending H.264/MPEG-4 AVC towards MVC. The basic approach of MVC for enabling inter-view prediction and view scalability in the context of H.264/MPEG-4 AVC is reviewed. Related supplemental enhancement information (SEI) metadata is also described. Various “frame compatible” approaches for support of stereo-view video as an alternative to MVC are also discussed. A summary of the coding performance achieved by MVC for both stereo- and multiview video is also provided. Future directions and challenges related to 3-D video are also briefly discussed.

Efficient SKIP Mode Detection for Coarse Grain Quality Scalable Video Coding

Scalable video coding (SVC) was recently standardized by the Joint Video Team as an extension of H.264. In SVC, a computationally expensive exhaustive mode decision is employed to select the best coding mode for each macroblock (MB), which achieves a high coding efficiency. In order to reduce computational complexity, we propose an efficient SKIP mode detection approach for coarse grain quality SVC. It makes use of the coding information of spatial neighboring MBs and the co-located MB in base layer to predict the SKIP mode MB and early terminate its mode decision procedure. Experimental results show that the proposed early SKIP mode decision approach can achieve the average computational saving about 54% with almost no loss of rate distortion (RD) performance in the enhancement layer.

A Fast Multi-reference Frame Selection Algorithm for Multiview Video Coding

Multiview video coding (MVC) plays an important role in three-dimensional video applications. Joint Video Team developed a joint multiview video model (JMVM) with multi-reference frame technology. Motion and disparity estimation are employed in multi-reference frame technology to provide better rate distortion performance. However, the process of searching blocks with variable sizes for motion and disparity estimation in multi-reference frames significantly increases the computational complexity. After analyzing the statistical features of multi-reference frames in MVC, this paper proposes a fast multi-reference frame selection algorithm based on a dynamic threshold technology correlated with the types of frames. All views of prediction structure in JMVM are categorized into three types, that is, the basic view without inter-view reference relation, the first layer views that only refer anchor frame from upper view, the second layer views that need to refer the basic view and the first layer views. Furthermore, all frames in MVC prediction structure are divided into three categories, the anchor frame without any reference, non-anchor frame in the base view and the first layer view, and the non-anchor frame in other views. Dynamic threshold technique is given to terminate the process of searching multi-reference frames early. The proposed algorithm also reduces the useless candidate reference frames by filtering out those reference frames that are less likely to contain the best matched results and achieves significant speed-up. Experimental results show that the proposed algorithm can achieve 50.13% ~ 72.19% reduction of encoding time in comparison with JMVM7.0, while the proposed algorithm hardly influences the rate distortion performance of MVC.

Context Adaptive Lagrange Multiplier (CALM) for Rate-Distortion Optimal Motion Estimation in Video Coding

In this paper, we propose an efficient and practical algorithm to dynamically adapt the Lagrange multipliers for each macroblock based on the context of the neighboring or upper layer blocks to improve rate-distortion performance. Our method improves the accuracy for the detection of true motion vectors as well as the most efficient encoding modes for luma, which are used for deriving the motion vectors, and modes for chroma. Simulation results for H.264/advanced video coding video demonstrate that our method reduces bit rate significantly and achieves peak signal-to-noise ratio gain over those of the joint model (JM) software for all sequences tested, with negligible extra computational cost. The improvement is particularly significant for high motion high-resolution videos. This paper describes our work that led to our Joint Video Team adopted contribution (included in software JM 12.0 onward), collectively known as context adaptive Lagrange multiplier (CALM).

Frame-layer bit allocation for multi-view video coding based on frame complexity estimation

Current multi-view video coding (MVC) reference model in joint video team (JVT) does not provide efficient rate control schemes. This paper presents a rate control algorithm for MVC by improving the quadratic rate-distortion (R-D) model. We reasonably allocate bit-rate among views based on the correlation analysis. The proposed algorithm consists of three levels to control the rate bits more accurately, of which the frame layer allocates bits according to the frame complexity and the temporal activity. Extensive experiments show that the proposed algorithm can control the bit rate efficiently.

Interactive region of interest scalability for wavelet based scalable video coder

The current wavelet-based scalable video coder supports quality, spatial and temporal scalabilities required for recent multimedia applications. One of the key scalability required by the Joint Video Team (JVT) is Interactive Region of Interest (IROI) scalability. This paper proposes a technique to interactively extract ROI from scalable sub bit-streams. At the encoder, after sub-band decomposition the 3D wavelet tree is divided into N sub tree, each of these has spatial relevance in the original video, which can be decoded interactively. A generalized model for optimal value of N is proposed, that satisfies minimum requirement of the JVT about IROI support at reduced cost of the encoder. The ROI request from decoder along with resolution and bandwidth are mapped to that in the sub-band domain and respective sub-bit-streams are identified. The extractor assigns more bandwidth to ROI and remaining to non-ROI regions. Hence the decoder reconstructs the video with highest quality at the ROI. The proposed IROI functionality enhances the scalability of video coding by additional dimension.

Bit-Depth Scalable Coding Using a Perfect Picture and Adaptive Neighboring Filter

Bit-depth scalability is a new research field in the on-going scalable extension of the H.264/AVC (SVC) video coding standard. The key point is to accurately predict the enhancement layer, whose bit depth is 10 or more, from the 8 bit base layer. An improved inter-layer prediction scheme for bit-depth scalability was developed that ensures compatibility with the standard and improves the encoding efficiency. The scheme uses a “perfect” 8 bit picture with an adaptive neighbor filter whose coefficients are optimized by minimizing the block distortion between the 8 bit reconstructed picture and the “perfect” picture to achieve a more precise 10 bit prediction based on the filtered picture. Double arithmetic precision is used to further improve the encoding efficiency. Experimental results show that the scheme outperforms the recent joint video team (JVT) proposal in the joint scalable video model (JSVM).