NAL Units Research Articles

Bitstream-Oriented Protection for the H.264/Scalable Video Coding (SVC)

The newly standardized H.264/SVC enable multimedia suppliers to provide video bitstream with temporal, spatial and quality scalabilities to meet various needs of end users. Its bitstream scalability characteristic is also especially suitable for multimedia applications under merged heterogeneous networks with different network protocols, capacities and throughputs. Although the network environment, to some extent, provides security protection against the illegal users, it is not enough to protect against the authorized users from digesting contents beyond its authorization. Hence the issue of content protection for the H.264/SVC has aroused researchers’ interests in recent years. In this paper, we present an efficient bitstream-oriented protection scheme for the H.264/SVC in a compression friendly and format compliant manner. The encryption is implemented on the Network Abstraction Layer (NAL) unit level. To improve the computational efficiency as well as to provide sufficient security, selective cryptographic algorithms with different computation cost are employed for different content level of SVC according to its significance. The I slice NAL units from the base layer of SVC bitstream are encrypted with the symmetric AES algorithm in Cipher Block Chaining mode. And the other NAL units are protected by simplified XOR cipher. Furthermore, a robust key management (key generation and distribution) mechanism is also discussed in this paper. The security analysis and simulation results further verify that the proposed methods can effectively protect the H.264/SVC bitstream at low computational cost.

Transport analysis and quality evaluation of MVC video streaming

This paper explores the performance of the emerging H.264/MVC standard for stereo video under various packetization schemes, encoding parameters and network conditions. An experimentation test-bed platform has been developed to support the multi session transmission approach for various video packetization options under various number of NAL units per frame. The paper presents measurements in terms of overhead as well as 3D video quality for sequences with different characteristics in terms of spatial resolution and motion. Extensive test-bed experiments indicate that the fragmentation of frames in more than one NAL units improves the perceived video quality measured objectively in terms of PSNR, VQM, SSIM for both base and non-base view as well as, subjectively in terms of 3D MOS.

A Multiresolution Channel Decomposition for H.264/AVC Unequal Error Protection

The most commonly used transmission channel in nowadays provides the same level of protection for all the information symbols. As the level of protection should be adequate to the importance of the information set, it is justified to use UEP channels in order to protect information of variable importance. Multiresolution channel decomposition has emerged as a strong concept and when combined with H.264/AVC layered multiresolution source it leads to outstanding results especially for mobile TV applications. Our approach is a double multiresolution scheme with embedded constellation modulations on its baseband channels followed by OFDM time/frequency multiresolution passband modulation. The aim is to protect The NAL units carrying the most valuable information by the coarse constellations into coarse sub-channels, and the NAL units that contain residual data by fined constellations and transposed into the fined OFDM sub-channels. In the multiresolution protection coding, our approach is a multiresolution decomposition of the core convolutional constituent of the PCCC where the NAL units carrying the most valuable information are coded by the rugged coefficient of the multiresolution code and the NAL units that contains residual data are coded by refined less secure coding coefficients.

Embedding Protection Inside H.264/AVC and SVC Streams

A backward compatible error-protection mechanism embedded into the H.264 (AVC or SVC) syntax is described. It consists of the addition into the H.264 bitstream of supplementary network abstraction layer (NAL) units that contain forward error-correction (FEC) data generated by a block error-correction code. The proposed mechanism allows to leave the original information bits and NAL units intact and does not rely on any side information or extra signalling coming from lower layers, ensuring backward compatibility with the standard syntax. Simulation results obtained with Reed-Solomon and Low-density parity check error-correcting codes show significant improvements for both erroneous and lossy transmission channel configurations.

A Practical RTP Packetization Scheme for SVC Video Transport over IP Networks

Scalable video coding (SVC) has been standardized as an extension of the H.264/AVC standard. This paper proposes a practical real-time transport protocol (RTP) packetization scheme to transport SVC video over IP networks. In combined scalability of SVC, a coded picture of a base or scalable enhancement layer is produced as one or more video layers consisting of network abstraction layer (NAL) units. The SVC NAL unit header contains a (DID, TID, QID) field to identify the association of each SVC NAL unit with its scalable enhancement layer without parsing the payload part of the SVC NAL unit. In this paper, we utilize the (DID, TID, QID) information to derive hierarchical spatio-temporal relationship of the SVC NAL units. Based on the derivation using the (DID, TID, QID) field, we propose a practical RTP packetization scheme for generating single RTP sessions in unicast and multicast transport of SVC video. The experimental results indicate that the proposed packetization scheme can be efficiently applied to transport SVC video over IP networks with little induced delay, jitter, and computational load.

Group-of-pictures-based unequal error protection for scalable video coding extension of H.264/AVC

We address the problem of unequal error protection (UEP) for SNR enhancement layer network abstraction layer (NAL) units of scalable video coding extension of H.264/AVC standard over wireless packet-erasure channel. We develop a UEP scheme by jointly selecting SNR NAL units and allocating unequal amounts of protection to selected NAL units for every group of pictures in the sequence. A simple heuristic algorithm is proposed to quickly derive the protection pattern. Experimental results demonstrate the proposed UEP scheme provides significant error resilience.

Optimized Bit Extraction Using Distortion Modeling in the Scalable Extension of H.264/AVC

The newly adopted scalable extension of H.264/AVC video coding standard (SVC) demonstrates significant improvements in coding efficiency in addition to an increased degree of supported scalability relative to the scalable profiles of prior video coding standards. Due to the complicated hierarchical prediction structure of the SVC and the concept of key pictures, content-aware rate adaptation of SVC bit streams to intermediate bit rates is a nontrivial task. The concept of quality layers has been introduced in the design of the SVC to allow for fast content-aware prioritized rate adaptation. However, existing quality layer assignment methods are suboptimal and do not consider all network abstraction layer (NAL) units from different layers for the optimization. In this paper, we first propose a technique to accurately and efficiently estimate the quality degradation resulting from discarding an arbitrary number of NAL units from multiple layers of a bitstream by properly taking drift into account. Then, we utilize this distortion estimation technique to assign quality layers to NAL units for a more efficient extraction. Experimental results show that a significant gain can be achieved by the proposed scheme.

Robust Video Transmission Over Wireless LANs

Home wireless networks are mainly used for data transmission; however, they are now being used in video delivery applications, such as video on demand or wireless internet protocol (IP) television. Off-the-shelf technologies are inappropriate for the delivery of real-time video. In this paper, a packetization method is presented for robust H.264 video transmission over the IEEE 802.11 wireless local area network (WLAN) configured as a wireless home network. To overcome the poor throughput efficiency of the IEEE 802.11 Medium Access Control (MAC), an aggregation scheme with a recovery mechanism is deployed and evaluated via simulation. The scheme maps several IP packets (each containing a single H.264 video packet called a Network Abstraction Layer (NAL) unit) into a single larger MAC frame. Video robustness is enhanced by using small NAL units and by retrieving possible error-free IP packets from the received MAC frame. The required modifications to the legacy MAC are described. Results in terms of throughput efficiency and peak-signal-to-noise ratio (PSNR) are presented for the case of broadcast and real-time transmission applications. Compared to the legacy case, an 80% improvement in throughput efficiency is achieved for a similar PSNR video performance. For fixed physical layer resources, our system provides a 2.5-dB gain in video performance over the legacy case for a similar throughput efficiency. The proposed solution provides considerable robustness enhancement for video transmission over IEEE 802.11-based WLANs.

Polyphase Downsampling Based Multiple Description Coding Applied to H.264 Video Coding

This paper presents a video coding method that improves error resilient functionality of H.264 with good coding efficiency. The method is based on PD (polyphase downsampling) multiple description coding. The only changes to H.264 are inserting PD before the DCT process and having new data partitioning NAL units. A coded slice is sent on 3 data partitioning NAL units. A header NAL unit contains motion vectors and block modes. Each of the other two NAL units contains a description generated by PD multiple description coding. The experimental results on all 9 of the test sequences of JVT SVC show that the proposed method gives 0.5 to 5 dB enhancement over the existing H.264 FMO checker board mode with motion vector based error-concealment.

RTP payload format for H.264/SVC scalable video coding

The scalable extension of H.264/AVC, known as scalable video coding or SVC, is currently the main focus of the Joint Video Team’s work. In its present working draft, the higher level syntax of SVC follows the design principles of H.264/AVC. Self-contained network abstraction layer units (NAL units) form natural entities for packetization. The SVC specification is by no means finalized yet, but nevertheless the work towards an optimized RTP payload format has already started. RFC 3984, the RTP payload specification for H.264/AVC has been taken as a starting point, but it became quickly clear that the scalable features of SVC require adaptation in at least the areas of capability/operation point signaling and documentation of the extended NAL unit header. This paper first gives an overview of the history of scalable video coding, and then reviews the video coding layer (VCL) and NAL of the latest SVC draft specification. Finally, it discusses different aspects of the draft SVC RTP payload format, including the design criteria, use cases, signaling and payload structure.

SVC bitstream adaptation in MPEG-21 multimedia framework

Scalable video coding (SVC) is the most promising video format for applications of collaborative communication. MPEG-21 standard has newly emerged to enable the interoperability of multimedia delivery in heterogeneous environments. In this paper we study adaptation of SVC bitstream in the context of MPEG-21 multimedia framework. For interfacing SVC bitstream with MPEG-21 based adaptation system, we propose three SVC specific adaptation operators. Based on our previous work with multidimensional video adaptation, we present an effective approach, using MPEG-21 DIA AdaptationQoS description tool, to model QoS control for SVC adaptation. Then we show how the operator values could be computed from that representation. For the actual adaptation at bitstream level, we propose a procedure to remove the unnecessary NAL units from an SVC bitstream. The result of this study enables QoS management of SVC streaming in an efficient and standardized manner.

RTP-based broadcast streaming of high definition H.264/AVC video: An error robustness evaluation

In this work, we present an evaluation of the performance and error robustness of RTP-based broadcast streaming of high-quality high-definition (HD) H.264/AVC video. Using a fully controlled IP test bed (Hillestad et al., 2005), we broadcast high-definition video over RTP/UDP, and use an IP network emulator to introduce a varying amount of randomly distributed packet loss. A high-performance network interface monitoring card is used to capture the video packets into a trace file. Purpose-built software parses the trace file, analyzes the RTP stream and assembles the correctly received NAL units into an H.264/AVC Annex B byte stream file, which is subsequently decoded by JVT JM 10.1 reference software. The proposed measurement setup is a novel, practical and intuitive approach to perform error resilience testing of real-world H.264/AVC broadcast applications. Through a series of experiments, we evaluate some of the error resilience features of the H.264/AVC standard, and see how they perform at packet loss rates from 0.01% to 5%. The results confirmed that an appropriate slice partitioning scheme is essential to have a graceful degradation in received quality in the case of packet loss. While flexible macroblock ordering reduces the compression efficiency about 1 dB for our test material, reconstructed video quality is improved for loss rates above 0.25%.

Architectural Unit Testing

A formal testing methodology is outlined in this paper, that proves applicable to validation of architectural units in object-oriented models, and its use is illustrated in the context of the design of a robot teleoperation architecture. Automated generation of test cases to validate the functionality of the robot trajectory generation unit showcases the key features of this methodology. A disciplined use of UML state diagrams, to model the unit's dynamics consistently with its static properties as modeled by class diagrams, enables one to provide such models with Input/Output Labelled Transition Systems (IOLTS) semantics, whence a rich machinery of testing theories and tools based on those theories become readily available. Our case study tells that, besides black-box testing of nal implementation units, white-box analysis of architectural units may greatly benefit from the exibility of parameterized I/O-conformance relations. Test purposes turn out to be a useful methodological link between functional requirements, which they are drawn from, and conformance relations, which they help one to instantiate, thereby delimiting test selection to purposeful tests. Contingent aspects of our methodology include: a mechanical translation of state diagrams in Basic LOTOS, a non-mechanical, use-case driven synthesis of test purposes, expressed in the same language, and the use of the TGV tool for automated test case generation. Other choices in these respects are well possible, without a ecting the characteristic traits of the proposed methodology, that are rather to be found in: 1) the combination of object-oriented architectural modeling with IOLTS semantics; 2) the aim at maximizing the potential for test generation from UML models, in a broad view of testing which applies throughout the development process; 3) the speci c proposal to consider internal actions as testable actions, in view of a better coordination between testing (discovery of faults) and debugging (discovery of internal sources of faults).