Video Bitstream Research Articles

Lossless fragile watermarking algorithm in compressed domain for multiview video coding

The hierarchical B picture (HBP) prediction structure is a typical coding scheme used for multiview video coding (MVC). This paper proposes a fragile watermarking algorithm for HBP-based multiview video coding. B_DIRECT_16 × 16 and B_SKIP are two types of macroblocks used for H.264/AVC coding, and are respectively generated in the DIRECT and SKIP prediction modes. Modification of the syntactic elements enables the watermark to be embedded while the B_SKIP macroblock is converted into the B_DIRECT_16 × 16 macroblock. To authenticate the integrity of the multiview video content, the watermark is generated from the content-based features of the cross-view of the multiview video bitstream. Experimental implementation revealed that the structural similarity index (SSIM) of the video quality was not affected by the proposed watermarking scheme. Compared to current algorithms, the proposed algorithm has a higher embedding capacity, produces less bitrate increase, and is less complex. It satisfies the transparency, bitrate stability, fragility, and real-time performance requirements of multiview video authentication.

Coding unit complexity-based predictions of coding unit depth and prediction unit mode for efficient HEVC-to-SHVC transcoding with quality scalability

To support good video quality of experiences in heterogeneous environments, transcoding an existed HEVC (high efficiency video coding) video bitstream to a SHVC (scalability extension of HEVC) bitstream with quality scalability is highly required. A straightforward way is to first fully decode the input HEVC bitstream and then fully re-encode it with the SHVC encoder, which requires a tremendous computational complexity. To solve the problem, in this paper, a coding unit complexity (CUC)-based prediction method for predictions of CU (coding unit) depth and PU (prediction unit) mode for efficient HEVC-to-SHVC transcoding with quality scalability is proposed to significantly reduce the transcoding complexity. The proposed method contains two prediction techniques, including (i) early termination and (ii) adaptive confidence interval, and predicts the CU depth and PU mode relying on the decoded information from the input HEVC bitstream. Experimental results have shown that the proposed method significantly outperforms the traditional HEVC-to-SHVC method by 74.14% on average in reductions of encoding time for SHVC enhancement layer.

A holistic modeling for QoE estimation in live video streaming applications over LTE Advanced technologies with Full and Non Reference approaches

Current mobile networks are providing high speed access to Internet at a rate of Gigabits per second. In this scenario, traditional services over wired networks are an alternative, in particular those based on live video streaming. But in the transition, different issues should be considered due to the rapid changing network conditions and the limited resources of the mobile devices. These issues should be taken into account to keep a good Quality of Experience (QoE) of the video in terms of a high Mean Opinion Score (MOS), a subjective video quality. Our goal is to estimate and predict this subjective metric in a holistic manner. Thus, we have analyzed and measured different variables related to Quality of Service, bit stream and basic video quality metrics of live video streaming over LTE Advanced (4G) mobile networks, as well as server side. With these variables, we apply a statistical method, called Factor Analysis based on the correlation of the variables, which allows us to estimate MOS of the delivered video in a robust and reliable way, both in Full Reference and Non Reference approaches. The real MOS has been evaluated through surveys. Finally, we compare the accuracy of the estimated MOS for the different approaches and compare their performance with well known publicly available video quality algorithms.

An Efficient Motion Detection and Tracking Scheme for Encrypted Surveillance Videos

Performing detection on surveillance videos contributes significantly to the goals of safety and security. However, performing detection on unprotected surveillance video may reveal the privacy of innocent people in the video. Therefore, striking a proper balance between maintaining personal privacy while enhancing the feasibility of detection is an important issue. One promising solution to this problem is to encrypt the surveillance videos and perform detection on the encrypted videos. Most existing encrypted signal processing methods focus on still images or small data volumes; however, because videos are typically much larger, investigating how to process encrypted videos is a significant challenge. In this article, we propose an efficient motion detection and tracking scheme for encrypted H.264/AVC video bitstreams, which does not require the previous decryption on the encrypted video. The main idea is to first estimate motion information from the bitstream structure and codeword length and, then, propose a region update (RU) algorithm to deal with the loss and error drifting of motion caused by the video encryption. The RU algorithm is designed based on the prior knowledge that the object motion in the video is continuous in space and time. Compared to the existing scheme, which is based on video encryption that occurs at the pixel level, the proposed scheme has the advantages of requiring only a small storage of the encrypted video and has a low computational cost for both encryption and detection. Experimental results show that our scheme performs better regarding detection accuracy and execution speed. Moreover, the proposed scheme can work with more than one format-compliant video encryption method, provided that the positions of the macroblocks can be extracted from the encrypted video bitstream. Due to the coupling of video stream encryption and detection algorithms, our scheme can be directly connected to the video stream output (e.g., surveillance cameras) without requiring any camera modifications.

LIGHTWEIGHT ENCRYPTION FOR HIGH EFFICIENCY VIDEO CODING (HEVC)

Video data are being compressed and distributed using one of several coding standards, among which the most recent and popular is the High Efficiency Video Coding (HEVC) standard. The threatening growth of security attacks, on the other hand, has brought security and privacy concerns to the attention of governments and people as well. In the absence of a reliable security system, shared multimedia data used on the public networks such as the internet will continue to be exposed to different types of attacks, making end-to-end encryption for video data a necessity to protect their sensitive information. Therefore, providing a reliable video security technique that complies to and fulfills the requirements of HEVC is pertinent. In this paper, a fast selective encryption approach is developed to provide protection for video bitstreams of HEVC, which can be used in real-time video applications, with low computational overhead and maintaining the standard’s video bit rate. This approach employs the popular Advance Encryption Standard (AES) algorithm to encrypt selected elements in the horizontal intra prediction modes. Experimental evaluations confirm the provision of adequate security level of video information, with no bitrate increase, no increase in computational delay and no additional impact on the compression performance when compared to non-secure techniques, while also achieving a satisfactory trade-off between the encryption reliability, flexibility, and computational complexity. The security level of this method was found to be strongly secure against plaintext and brute force attacks.

Bitstream-Oriented Protection for the H.264/Scalable Video Coding (SVC)

The newly standardized H.264/SVC enable multimedia suppliers to provide video bitstream with temporal, spatial and quality scalabilities to meet various needs of end users. Its bitstream scalability characteristic is also especially suitable for multimedia applications under merged heterogeneous networks with different network protocols, capacities and throughputs. Although the network environment, to some extent, provides security protection against the illegal users, it is not enough to protect against the authorized users from digesting contents beyond its authorization. Hence the issue of content protection for the H.264/SVC has aroused researchers’ interests in recent years. In this paper, we present an efficient bitstream-oriented protection scheme for the H.264/SVC in a compression friendly and format compliant manner. The encryption is implemented on the Network Abstraction Layer (NAL) unit level. To improve the computational efficiency as well as to provide sufficient security, selective cryptographic algorithms with different computation cost are employed for different content level of SVC according to its significance. The I slice NAL units from the base layer of SVC bitstream are encrypted with the symmetric AES algorithm in Cipher Block Chaining mode. And the other NAL units are protected by simplified XOR cipher. Furthermore, a robust key management (key generation and distribution) mechanism is also discussed in this paper. The security analysis and simulation results further verify that the proposed methods can effectively protect the H.264/SVC bitstream at low computational cost.

Data Partitioning Technique for Improved Video Prioritization

A compressed video bitstream can be partitioned according to the coding priority of the data, allowing prioritized wireless communication or selective dropping in a congested channel. Known as data partitioning in the H.264/Advanced Video Coding (AVC) codec, this paper introduces a further sub-partition of one of the H.264/AVC codec’s three data-partitions. Results show a 5 dB improvement in Peak Signal-to-Noise Ratio (PSNR) through this innovation. In particular, the data partition containing intra-coded residuals is sub-divided into data from: those macroblocks (MBs) naturally intra-coded, and those MBs forcibly inserted for non-periodic intra-refresh. Interactive user-to-user video streaming can benefit, as then HTTP adaptive streaming is inappropriate and the High Efficiency Video Coding (HEVC) codec is too energy demanding.

An approach to embed image in video as watermark using a mobile device

Evolution in communication media and Internet technology has made information access easier and faster. At the same time there has been a major concern towards the security of various multimedia objects. Multimedia objects are files that contain images, video clips etc. These are the most popular shared objects over the internet through various transferrable devices. One of the way to secure the content is digital watermarking. Digital watermarking is the method to embed the secret data within text, image or video for ownership verification. In this work related to bitstream video watermarking technique, a study of spatial and temporal domain watermarking is carried out. An experiment is conducted to analyze the execution time of insertion and extraction process of bitstream watermarking on videos. Bit error count and power consumed by mobile device are computed for analysis of the bitstream video watermarking technique. As a proof of concept, the embedding of image on video is implemented using a computer and then the same logic is implemented for execution on a mobile phone. Experimental results demonstrate that, the method is feasible to be implemented using a computer and mobile phone. Image that is inserted on the video is extracted with an acceptable PSNR value. Execution time and power consumed in a mobile phone are within the acceptable limits. This ascertains the application’s feasibility for a mobile phone.

Analysis of YouTube’s traffic adaptation to dynamic environments

The popular Internet service, YouTube, has adopted by default the HyperText Markup Language version 5 (HTML5). With this adoption, YouTube has moved to Dynamic Adaptive Streaming over HTTP (DASH) as Adaptive BitRate (ABR) video streaming technology. Furthermore, rate adaptation in DASH is solely receiver-driven. This issue motivates this work to make a deep analysis of YouTube’s particular DASH implementation. Firstly, this article provides a state of the art about DASH and adaptive streaming technology, and also YouTube traffic characterization related work. Secondly, this paper describes a new methodology and test-bed for YouTube’s DASH implementation traffic characterization and performance measurement. This methodology and test-bed do not make use of proxies and, moreover, they are able to cope with YouTube traffic redirections. Finally, a set of experimental results are provided, involving a dataset of 310 YouTube’s videos. The depicted results show a YouTube’s traffic pattern characterization and a discussion about allowed download bandwidth, YouTube’s consumed bitrate and quality of the video. Moreover, the obtained results are cross-validated with the analysis of HTTP requests performed by YouTube’s video player. The outcomes of this article are applicable in the field of Quality of Service (QoS) and Quality of Experience (QoE) management. This is valuable information for Internet Service Providers (ISPs), because QoS management based on assured download bandwidth can be used in order to provide a target end-user’s QoE when YouTube service is being consumed.

Application-Guided Power-Efficient Fault Tolerance for H.264 Context Adaptive Variable Length Coding

This paper presents a fault-tolerance technique for H.264's Context-Adaptive Variable Length Coding (CAVLC) on unreliable computing hardware. The application-specific knowledge is leveraged at both algorithm and architecture levels to protect the CAVLC process (especially context adaptation and coding tables) in a reliable yet power-efficient manner. Specifically, the statistical analysis of coding syntax and video content properties are exploited for: (1) selective redundancy of coefficient/header data of video bitstreams; (2) partitioning the coding tables into various sub-tables to reduce the power overhead of fault tolerance; and (3) run-time power management of memory parts storing the sub-tables and their parity computations. Experimental results demonstrate that leveraging application-specific knowledge reduces area and performance overhead by 2x compared to a double-parity table protection technique. For functional verification and area comparison, the complete H.264 CAVLC architecture is prototyped on a Xilinx Virtex-5 FPGA (though not limited to it).

Efficient temporal and interlayer parameter prediction for weighted prediction in scalable high efficiency video coding

Weighted prediction (WP) is an efficient video coding tool that was introduced since the establishment of the H.264/AVC video coding standard, for compensating the temporal illumination change in motion estimation and compensation. WP parameters, including a multiplicative weight and an additive offset for each reference frame, are required to be estimated and transmitted to the decoder by slice header. These parameters cause extra bits in the coded video bitstream. High efficiency video coding (HEVC) provides WP parameter prediction to reduce the overhead. Therefore, WP parameter prediction is crucial to research works or applications, which are related to WP. Prior art has been suggested to further improve the WP parameter prediction by implicit prediction of image characteristics and derivation of parameters. By exploiting both temporal and interlayer redundancies, we propose three WP parameter prediction algorithms, enhanced implicit WP parameter, enhanced direct WP parameter derivation, and interlayer WP parameter, to further improve the coding efficiency of HEVC. Results show that our proposed algorithms can achieve up to 5.83% and 5.23% bitrate reduction compared to the conventional scalable HEVC in the base layer for SNR scalability and 2× spatial scalability, respectively.

Customizing short-length LT codes with evolution strategies for video streaming protection

Designing short-length Luby Transform (SLLT) codes to best protect video streaming and multicasting over lossy communication remains largely an empirical exercise. In this paper, we present a systematic approach to customize the decoding performance of these codes so that the protected video bitstreams may have the best playback quality over a wide range of channel loss rates. Our approach begins with the proposal of a new SLLT decoding performance model based on three parameters: decoding overhead, symbol decoding failure rate and tail probability of symbol decoding failure rate. We then formulate the design of SLLT codes as a multi-objective optimization problem, specify the design objectives in terms of goal program, and search for the most suitable codes using an augmented weighted Tchebycheff method implemented with the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Two design examples are provided to illustrate the effectiveness of our approach: (1) an SLLT post-code of a short-length raptor code that provides erasure protection to H.264 AVC bitstreams, and (2) an SLLT post-code of a rateless UEP code that supports graceful degradation of H.264 SVC playback quality. Empirical results demonstrate that the proposed method is capable of producing SLLT codes with customized decoding performance, whereas, the customized codes enable the playback pictures to attain significantly higher PSNR values at different stages of the decoding process than the pictures recovered under the protection of conventionally optimized codes.

Compressed-domain visual saliency models: a comparative study

Computational modeling of visual saliency has become an important research problem in recent years, with applications in video quality estimation, video compression, object tracking, retargeting, summarization, and so on. While most visual saliency models for dynamic scenes operate on raw video, several models have been developed for use with compressed-domain information such as motion vectors and transform coefficients. This paper presents a comparative study of eleven such models as well as two high-performing pixel-domain saliency models on two eye-tracking datasets using several comparison metrics. The results indicate that highly accurate saliency estimation is possible based only on a partially decoded video bitstream. The strategies that have shown success in compressed-domain saliency modeling are highlighted, and certain challenges are identified as potential avenues for further improvement.

Video processing for panoramic streaming using HEVC and its scalable extensions

Panoramic streaming is a particular way of video streaming where an arbitrary Region-of-Interest (RoI) is transmitted from a high-spatial resolution video, i.e. a video covering a very “wide-angle” (much larger than the human field-of-view – e.g. 360°). Some transport schemes for panoramic video delivery have been proposed and demonstrated within the past decade, which allow users to navigate interactively within the high-resolution videos. With the recent advances of head mounted displays, consumers may soon have immersive and sufficiently convenient end devices at reach, which could lead to an increasing demand for panoramic video experiences. The solution proposed within this paper is built upon tile-based panoramic streaming, where users receive a set of tiles that match their RoI, and consists in a low-complexity compressed domain video processing technique for using H.265/HEVC and its scalable extensions (H.265/SHVC and H.265/MV-HEVC). The proposed technique generates a single video bitstream out of the selected tiles so that a single hardware decoder can be used. It overcomes the scalability issue of previous solutions not using tiles and the battery consumption issue inherent of tile-based panorama streaming, where multiple parallel software decoders are used. In addition, the described technique is capable of reducing peak streaming bitrate during changes of the RoI, which is crucial for allowing a truly immersive and low latency video experience. Besides, it makes it possible to use Open GOP structures without incurring any playback interruption at switching events, which provides a better compression efficiency compared to closed GOP structures.

Joint Motion Compensation and MC-Reused Spatial Error Concealment on Hardware Architecture Design

Error concealment (EC) can recover visual quality, while transmission error occurs on video bitstream. Generally, EC is an extra function integrated into the video decoder and also consumes an additional hardware resource. It’s hardware architecture is only activated, when transmission error occurs. Otherwise, it is totally idle, causing a poor hardware utilization. To overcome this problem, the MC-Reused SEC algorithm is proposed. It makes the arithmetic operations of EC compliant with those of MC. This indicates arithmetic operations can be shared with each other. Both MC and EC can be processed on an identical hardware architecture instead of individual ones. Consequently, the limitation on hardware utilization for joint MC and EC can be eliminated. This work has proposed a hardware architecture that can be fully-shared between MC and EC with a 100 % hardware utilization. This hardware architecture can demonstrate a higher hardware efficiency, which considers power, gate count, and throughput. Moreover, the hardware-sharing can also increase the cost-efficiency especially for more advanced process technologies. This work is realized with the technology of TSMC 0.18um CMOS process. The throughputs of MC and EC are 1.0 Gpixels/sec and 320 Mpixels/sec, respectively. Experiment results reveal that this work demonstrates a competitive performance on hardware-efficiency.

Peer-to-peer live video streaming with rateless codes for massively multiplayer online games

We present a multi-level multi-overlay hybrid peer-to-peer live video system that enables players of Massively Multiplayer Online Games to simultaneously stream the video of their game and watch the game videos of other players. Each live video bitstream is encoded with rateless codes and multiple trees are used to transmit the encoded symbols. Trees are constructed dynamically with the aim to minimize the transmission rate at the source while maximizing the number of served peers and guaranteeing on-time delivery and reliability. ns-2 simulations and real measurements on the Internet show competitive performance in terms of start-up delay, playback lag, rejection rate, used bandwidth, continuity index, and video quality.

Two-dimensional reversible data hiding-based approach for intra-frame error concealment in H.264/AVC

Highly compressed video bit-stream is extremely sensitive to transmission error. A novel two-dimensional reversible data hiding-based approach for intra-frame error concealment is proposed, which aims at improving video quality at decoder when video bit-stream incur transmission errors. The scheme involves embedding the motion vector (MV) of a macroblock (MB) into other MB within the same intra-frame, and extracting the embedded MV from the received video frame for reconstruction of the corrupted MB. Based on the distribution of the motion vector data and the characteristic of histogram shifting, a specific two-dimensional reversible data hiding mechanism is designed. Consequently, the distortion of the marked video can be controlled at a low level. Experimental results show that the damaged macroblocks can be recovered with a higher quality using the reversible data hiding methodology, as compared to the non-reversible data hiding method. Furthermore, experimental results demonstrate that the proposed algorithm outperforms some state-of-the-art reversible data hiding error concealment schemes in improving the perceptual quality.

Media-Friendly and TCP-Friendly Rate Control Protocols for Multimedia Streaming

This paper describes a design framework for TCP-friendly and media-friendly rate control algorithms for multimedia streaming applications. The idea of this framework is to start from a TCP-friendly Rate Control (TFRC) transmission rate and then alter this transmission rate so that it tracks the media characteristics of the stream (e.g., bitrate) or other application characteristics like the client buffer fill level. In this way, the media-friendly property of the algorithm is achieved. We give three rules that guide how the TFRC throughput should track the evolution of the stream’s media characteristics and remain TCP-friendly in the long term. We also present, as proof of concept, four simple media-friendly and TCP-friendly congestion control algorithms built using the aforementioned framework. These congestion control algorithms are better suited for multimedia streaming applications than traditional TCP congestion control or smooth congestion control algorithms like TFRC. We have performed evaluations of two of the four proposed media-friendly and TCP-friendly congestion control algorithms under various network conditions and validated that they represent viable transport solutions, better than TFRC, for variable bitrate video streams. More specifically, our two media-friendly and TCP-friendly congestion control algorithms maintained a TCP-friendly throughput in the long term in all experiments and avoided an empty buffer at the client side in situations when TFRC could not achieve this.

ALAVIT: An adaptive layered video transmission for wireless OFDM channels

It is well-known that the layered transmission for video bit-streams generated by a layered coding can gracefully accommodate the receivers' heterogeneity in wireless networks. In this paper, we propose a new layered video transmission scheme employing the subcarrier allocation based on the wireless channel characteristics of orthogonal frequency division multiplexing (OFDM) physical (PHY) layer, leading to an adaptive layered video transmission (ALAVIT). In our scheme, scalable video coding (SVC) is exploited to generate the layered video bit-streams; and the resulting base layer (BL) and enhancement layer (EL) bits are modulated differently to obtain their individual wireless symbols. According to the estimated channel characteristics described by the subcarrier channel gains of a connected wireless receiver, subcarriers with good channel quality are allocated to BL symbols for a better protection of these more important bits. In this unicast case, our ALAVIT scheme is demonstrated to provide a much better performance as compared to the state-of-the-art PHY layer techniques such as s-mod and MixCast. This framework is then generalized from unicast to multicast where the problem is formulated into an optimization involving the average bit-error-rate (BER) over all connected receivers. While the complexity of solving this problem remains extremely high for the current OFDM case of 64 or even more subcarriers, we demonstrate by using a case study of 8 subcarriers that the optimal solution indeed offers a better performance than the heuristic solutions.

Inter-frame distortion drift analysis for reversible data hiding in encrypted H.264/AVC video bitstreams

Due to the privacy-preserving requirement for cloud data management, it is necessary to perform data hiding in the encrypted domain. This paper proposes an effective scheme for reversible data hiding in encrypted H.264/AVC video bitstreams. In the encryption phase, the codewords of intra prediction modes, motion vector differences and partial residual coefficients are encrypted without video bit rate increment for preserving the confidentiality of video content. In the data hiding phase, we first present a theoretical analysis of the picture distortion caused by data embedding and the subsequent inter-frame distortion drift. Based on the analysis, we estimate the embedding distortions caused by modifying different residual coefficients and embed data into residual coefficients with different priorities for decreasing the inter-frame distortion drift. The data embedding is implemented by the histogram shifting technique. If the receiver decrypts the encrypted video bitstream and extracts the embedded data, the original video bitstream can be perfectly recovered. In addition, if the receiver decrypts the encrypted video bitstream without extracting the embedded data, the bitstream can still be decoded to obtain the reconstructed video with good quality.