Spatial Scalable Video Coding Research Articles

Fast watermarking scheme for real-time spatial scalable video coding

Recent advancements in high resolution scalable video coding have significantly increased the computational complexities of watermarking solutions for real-time video encoding systems. This paper proposes a human visual system based watermarking algorithm for spatial scalable coding based on the H.264/SVC standard. The proposed algorithm extracts textural feature from a set of 7 high energy quantized coefficients in 4×4 luma INTRA-predicted blocks of all slices and embeds watermark into the highly textured block which has at least one non-zero coefficient in 6 selected locations. The same embedding process is performed for all layers of the video to improve robustness against common video processing attacks. Experiments were conducted by embedding up to 8192 watermark bits into a four-layer spatial scalable coded video. Results suggest that the proposed scheme produces watermarked video with an average visual quality degradation of ∼0.36dB at the expense of 2.18% bitrate overhead. In addition, the proposed watermarking scheme achieves an average detection rate of 0.98, 0.97 and 0.71 against re-encoding, recompression and Gaussian filtering attacks, respectively, using the base layer, and above 0.97, 0.94 and 0.66, respectively when using any of the enhancement layer.

Real-time high-resolution downsampling algorithm on many-core processor for spatially scalable video coding

The progression toward spatially scalable video coding (SVC) solutions for ubiquitous endpoint systems introduces challenges to sustain real-time frame rates in downsampling high-resolution videos into multiple layers. In addressing these challenges, we put forward a hardware accelerated downsampling algorithm on a parallel computing platform. First, we investigate the principal architecture of a serial downsampling algorithm in the Joint-Scalable-Video-Model reference software to identify the performance limitations for spatially SVC. Then, a parallel multicore-based downsampling algorithm is studied as a benchmark. Experimental results for this algorithm using an 8-core processor exhibit performance speedup of 5.25× against the serial algorithm in downsampling a quantum extended graphics array at 1536p video resolution into three lower resolution layers (i.e., Full-HD at 1080p, HD at 720p, and Quarter-HD at 540p). However, the achieved speedup here does not translate into the minimum required frame rate of 15 frames per second (fps) for real-time video processing. To improve the speedup, a many-core based downsampling algorithm using the compute unified device architecture parallel computing platform is proposed. The proposed algorithm increases the performance speedup to 26.14× against the serial algorithm. Crucially, the proposed algorithm exceeds the target frame rate of 15 fps, which in turn is advantageous to the overall performance of the video encoding process.

Generalized Nash Bargaining Solution to Rate Control Optimization for Spatial Scalable Video Coding.

Rate control (RC) optimization is indispensable for scalable video coding (SVC) with respect to bitstream storage and video streaming usage. From the perspective of centralized resource allocation optimization, the inner-layer bit allocation problem is similar to the bargaining problem. Therefore, bargaining game theory can be employed to improve the RC performance for spatial SVC. In this paper, we propose a bargaining game based one-pass RC scheme for spatial H.264/SVC. In each spatial layer (SL), the encoding constraints, such as bit rates, buffer size are jointly modeled as resources in the inner-layer bit allocation bargaining game. The modified rate-distortion (R-D) model incorporated with the inter-layer coding information is investigated. Then the generalized Nash bargaining solution (NBS) is employed to achieve an optimal bit allocation solution. The bandwidth is allocated to the frames from the generalized NBS adaptively based on their own bargaining powers. Experimental results demonstrate that the proposed rate control algorithm achieves appealing image quality improvement and buffer smoothness. The average mismatch of our proposed algorithm is within the range of 0:19%2:63%.

An estimation-theoretic framework for spatially scalable video coding.

This paper focuses on prediction optimality in spatially scalable video coding. It draws inspiration from an estimation-theoretic prediction framework for quality (SNR) scalability earlier developed by our group, which achieved optimality by fully accounting for relevant information from the current base layer (e.g., quantization intervals) and the enhancement layer, to efficiently calculate the conditional expectation that forms the optimal predictor. It was central to that approach that all layers reconstruct approximations to the same original transform coefficient. In spatial scalability, however, the layers encode different resolution versions of the signal. To approach optimality in enhancement layer prediction, this paper departs from existing spatially scalable codecs that employ pixel domain resampling to perform interlayer prediction. Instead, it incorporates a transform domain resampling technique that ensures that the base layer quantization intervals are accessible and usable at the enhancement layer despite their differing signal resolutions, which in conjunction with prior enhancement layer information, enable optimal prediction. A delayed prediction approach that complements this framework for spatial scalable video coding is then provided to further exploit future base layer frames for additional enhancement layer coding performance gains. Finally, a low-complexity variant of the proposed estimation-theoretic prediction approach is also devised, which approximates the conditional expectation by switching between three predictors depending on a simple condition involving information from both layers, and which retains significant performance gains. Simulations provide experimental evidence that the proposed approaches substantially outperform the standard scalable video codec and other leading competitors.

Content-adaptive Traffic Surveillance Video Coding with Extended Spatial Scalability

Regions of interest (ROI) or visually salient regions are rarely considered in spatial scalable video coding, thus visually important content can not be better adapted to lower display resolutions. In this paper, we propose a content-adaptive spatial scalable coding for traffic surveillance video. First, the background image is extracted by an improved single Gaussian method based on the spatio-temporal model and updated from the latest static image. Then a background subtraction algorithm is present for detecting and tracking vehicles, the motion window of the leading vehicle is commonly referred to as ROI in traffic surveillance, and ROI is as a cropping window in extended spatial scalability (ESS) of the scalable video coding (SVC). Moreover, we employ a tracking-aware compression algorithm to remove more low tracking interest bit rate by ROI-based quantization strategy and frequency coefficient suppression technique, so tracking accuracy is used instead of PSNR as the compression criterion. The experimental results show that compared with conventional scaling coding the proposed algorithm can greatly improve the visual perception of the decoded base layer video with limited loss in the rate-distortion performance, and allows for about 60% bit rate savings while maintaining comparable tracking accuracy.

Spatially Scalable Video Coding For HEVC

Spatially scalable video coding (SSVC) provides an efficient way to transmit one video at different resolutions. Based on the emerging High Efficiency Video Coding (HEVC), we propose an SSVC scheme to support both single-loop (SL) and multiloop (ML) solutions by enabling different interlayer prediction mechanisms. Specifically, we employ two interlayer prediction modes: quadtree-based prediction mode (Q-mode) and learning-based prediction mode (L-mode). The Q-mode is investigated to exploit the interlayer redundancy based on the quadtree coding structure of HEVC. Due to the high correlation between layers, Q-mode utilizes the coded information from the base layer quadtree, including coding unit split, prediction unit partition, motion information, and partial texture information of transform unit, to predict the enhancement layer quadtree. By enabling Q-mode, we provide a basic SL solution for low complexity applications. Besides the correlation explored in Q-mode, we employ an extra L-mode to further improve the coding performance. In L-mode, the temporal-spatial correlation is exploited simultaneously by visual patch-based learning and mapping at pixel level. This helps us achieve more accurate prediction signals based on the coarse base layer reconstruction within an ML structure. Experimental results show the effectiveness of our SSVC scheme compared with the simulcast case and other HEVC-based SSVC schemes.

Novel Rate-Quantization Model-Based Rate Control With Adaptive Initialization for Spatial Scalable Video Coding

A novel spatial-layer rate-control algorithm is proposed for scalable video coding (SVC). First, by analyzing the relationship among the best initial quantization parameter ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> ), channel bandwidth, and the initial frames' complexity measure, an adaptive <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -initialization model is introduced to determine the starting <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> value for not only the base layer but also the spatial enhancement layers of SVC. Then, a two-stage <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -determination scheme is designed to improve the rate-control performance for the spatial-layer SVC with an efficient frame-complexity prediction method and an adaptive model-parameter update technique employed. Experimental results demonstrate the effectiveness of the proposed <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -initialization scheme and the two-stage <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -determination algorithm. By comparison with two other benchmark rate-control algorithms, the proposed rate-control algorithm is able to control constrained bit rates accurately with better rate-distortion performance.

Fast Mode Decision Using All-Zero Block Detection for Fidelity and Spatial Scalable Video Coding

In scalable video coding (SVC) as an extension of H.264/advanced video coding (AVC), a computationally expensive exhaustive mode decision is employed to select the best coding mode for each macroblock (MB). In order to reduce computational complexity, we propose a fast mode decision algorithm for SVC which uses an all-zero block (AZB) detection. Based on the empirical analysis of the inter-layer correlation of the AZB, the MB at the enhancement layer (EL) is predicted whether to be the AZB. Then, only predicted MBs are examined by the AZB detection algorithm. Since the mode decision can be terminated by detecting the AZB, we determine the processing order of the candidate modes at the EL in order to terminate the mode decision in the early stage. The proposed algorithm can be combined with other conventional fast mode decision methods in order to further reduce the computational complexity of those methods. Experimental results show that the proposed algorithm can significantly speed up the encoding process, especially for low bit-rate video sequences, without deteriorating the coding efficiency of SVC.

Fast mode decision algorithm for spatial scalable video coding

Multimedia applications are ubiquitous in daily life. To satisfy multiple requests equipped with different devices and bandwidths, scalable video coding has been developed. However, it is difficult to realise spatial and quality scalabilities in practice owing to the increased coding complexity brought by inter-layer prediction. A novel fast mode decision algorithm for spatial scalable video coding is proposed in which macroblock modes in the base layer and the enhancement layer are predicted from each other. The experimental results show that the proposed algorithm provides up to 74% of time saving with ignorable quality degradation and acceptable bit rate increase.

Fast decoder for H.264 scalable video coding with selective up-sampling for spatial scalable video coding

Abstract. A simple and effective method is presented forfast decoding of H.264 scalable video coding SVC . Theup-sampling operation in H.264 SVC makes the decodervery complex, because convolution and complex memorytransactions are inevitable. The proposed method exploitscoded modes of neighboring macroblocks MBs for deter-mining up-sampling operation on MB by MB. The experi-mental validation shows considerable improvement in de-coding time, and the proposed method reduces thecomplexity by about 25% on average. © 2008 Society of Photo-Optical Instrumentation Engineers. DOI: 10.1117/1.2957042 Subject terms: video decoder; scalable video coding; upsampling . Paper 080173LR received Mar. 9, 2008; revised manuscriptreceived May 18, 2008; accepted for publication May 20, 2008;published online Jul. 16, 2008. 1 Introduction Although transcoding methods 1–4 support spatial, temporal,and quality adaptation, they require additional tools intransmission. Consequently, a new standard was proposedto provide scalable video models for scalable extension ofH.264/AVC H.264 scalable video coding SVC .

Spatial scalable video coding using a combined subband-DCT approach

A combined subband-DCT approach for spatial scalable video coding is presented. The high-resolution input signal is decomposed into four spatial subband signals. The low-frequency subband is used as the low-resolution signal and is separately coded in the base-layer bitstream, and the high-frequency subband signals are coded in the enhancement-layer bitstream. The low-resolution signal is reconstructed from the base-layer bitstream and the high-resolution signal is reconstructed using both the base- and the enhancement-layer bitstream. Similar to MPEG, DCT-based hybrid coding techniques are applied for the coding of the subband signals, but an improved motion-compensated prediction is used for the low-resolution signal. Additionally, SNR scalability is introduced to allow a flexible bit allocation for the base and the enhancement layer. Experimental results at a bit rate of 6 Mbit/s show that the reference coder MPEG spatial scalable profile (SSP) leads to a loss of more than 2.2-dB peak signal-to-noise ratio (PSNR) compared with nonscalable MPEG-2 coding at the same bit rate, whereas the proposed combined subband-DCT scheme is able to achieve a decrease of less than 0.4 dB in PSNR.