SNR Scalability Research Articles

CNN-based intra partitioning process for spatial and SNR scalability for SHVC

CNN-based intra partitioning process for spatial and SNR scalability for SHVC

Distributed Compressive Sensing for Cloud-Based Wireless Image Transmission

We consider efficient image transmission via time-varying channels. To improve the performance, we propose a new distributed compressive sensing (CS) scheme that can leverage similar images in the cloud. It is featured by channel SNR and bandwidth scalability, high efficiency, and low encoding complexity. For each image, a compressed thumbnail is first transmitted after forward error correction (FEC) and modulation to retrieve similar images and generate a side information (SI) in the cloud. The residual image after subtracting the decompressed thumbnail is then coded and transmitted by CS through a very dense constellation without FEC. The linearly and ratelessly generated CS measurements make it capable of achieving both graceful quality degradation (GD) with the channel SNR and bandwidth scalability in a universal scheme. A mode decision and transform-domain power allocation are introduced for better bandwidth usage and protection against channel errors. At the decoder, a two-step CS decoding is performed to recover the residual signal, where both the local and nonlocal correlations within the image and that with the SI are exploited. Simulations on landmark images and an AWGN channel show that the received image quality gracefully increases with the channel SNR and bandwidth. Furthermore, it outperforms existing schemes both subjectively and objectively by up to 11 dB gains compared with the state-of-the-art transmission scheme with GD, i.e. SoftCast.

Efficient temporal and interlayer parameter prediction for weighted prediction in scalable high efficiency video coding

Weighted prediction (WP) is an efficient video coding tool that was introduced since the establishment of the H.264/AVC video coding standard, for compensating the temporal illumination change in motion estimation and compensation. WP parameters, including a multiplicative weight and an additive offset for each reference frame, are required to be estimated and transmitted to the decoder by slice header. These parameters cause extra bits in the coded video bitstream. High efficiency video coding (HEVC) provides WP parameter prediction to reduce the overhead. Therefore, WP parameter prediction is crucial to research works or applications, which are related to WP. Prior art has been suggested to further improve the WP parameter prediction by implicit prediction of image characteristics and derivation of parameters. By exploiting both temporal and interlayer redundancies, we propose three WP parameter prediction algorithms, enhanced implicit WP parameter, enhanced direct WP parameter derivation, and interlayer WP parameter, to further improve the coding efficiency of HEVC. Results show that our proposed algorithms can achieve up to 5.83% and 5.23% bitrate reduction compared to the conventional scalable HEVC in the base layer for SNR scalability and 2× spatial scalability, respectively.

Scalable Extension of HEVC for Flexible High‐Quality Digital Video Content Services

This paper describes the scalable extension of High Efficiency Video Coding (HEVC) to provide flexible highquality digital video content services. The proposed scalable codec is designed on multi‐loop decoding architecture to support inter‐layer sample prediction and inter‐layer motion parameter prediction. Inter‐layer sample prediction is enabled by inserting the reconstructed picture of the reference layer (RL) into the decoded picture buffer of the enhancement layer (EL). To reduce the motion parameter redundancies between layers, the motion parameter of the RL is used as one of the candidates in merge mode and motion vector prediction in the EL. The proposed scalable extension can support scalabilities with minimum changes to the HEVC and provide average Bjøntegaard delta bitrate gains of about 24% for spatial scalability and of about 21% for SNR scalability compared to simulcast coding with HEVC.

A Novel VLSI Architecture for SPHIT Encoder

In this Paper we propose a highly scalable image compression scheme based on the set partitioning in hierarchical trees (SPIHT) algorithm. Our algorithm called highly scalable SPIHT (HS-SPIHT), supports spatial and SNR scalability and provides a bit stream that can be easily adapted (reordered) to given bandwidth and resolution requirements by a simple transcoder (parser). The HS-SPIHT algorithm adds the spatial scalability feature without sacrificing the SNR embeddedness property as found in the original SPIHT bit stream. HS-SPIHT finds applications in progressive Web browsing, flexible image storage and retrieval, and image transmission over heterogeneous networks. Here we have written the core processor Microblaze is designed in VHDL (VHSIC hardware description language), implemented using XILINX ISE 8.1 Design suite the algorithm is written in system C Language and tested in SPARTAN-3 FPGA kit by interfacing a test circuit with the PC using the RS232 cable. The test results are seen to be satisfactory. The area taken and the speed of the algorithm are also evaluated.

End-to-end transmission of scalable video contents: performance evaluation over EvalSVC—a new open-source evaluation platform

Scalable Video Coding (SVC) is the latest extension of the famous Advance Video Coding (AVC) standard. Scalability is important and useful because it is dedicated to the transmission of video contents over heterogeneous network conditions and terminals’ capabilities. Nevertheless, the multimedia service research community and industry have not been able to fully utilize the entire potential of this video coding standard extension because of the lack of a platform for evaluating the end-to-end transmission of SVC-contents. EvalSVC aims to foster SVC-based applications and research in multimedia services. It is capable of evaluating the end-to-end transmission of SVC bit-streams encoded with enhanced features (spatial, temporal, SNR, and combined scalability). The output results are both objective and subjective metrics of the video transmission. Interfaces with real networks and an overlay simulation platform are presented. Through these interfaces, the transmission performance of different types of SVC scalability and AVC bit-streams can be evaluated easily.

A 135 MHz 542 k Gates High Throughput H.264/AVC Scalable High Profile Decoder

To satisfy the requirement of application heterogeneities, the latest H.264/AVC based video coding standard called scalable video coding additional includes temporal, SNR, and spatial scalabilities for frame rate, quality, and frame resolution adaptation. However, these inclusions significantly increase chip design difficulties such as decoding time, memory bandwidth, and area cost. This paper presents an H.264/AVC scalable high profile decoder realization with several optimization techniques to provide high throughput video decoding. For decoding flow, this paper proposes an one-pass macroblock-based quality layer decoding flow for SNR scalability and 71% of external memory bandwidth and 66% of macroblock processing cycles can be saved. For texture padding in interlayer intra prediction, the modified padding flow can save 26% of decoding time. For interlayer predictor design, this paper proposes a centralized concept for accumulation-based calculation of corresponding spatial position, simplified poly-phase interpolator, and efficient motion vector generator to save area cost and decoding time. Furthermore, the residual reconstruction path with the parallel-pipeline architecture is also proposed to cope with the additional decoding complexity and thus leads to 54% of gate count savings compared to the traditional serial-pipeline architecture. Finally, the proposed H.264/AVC scalable high profile decoder design is implemented with 90 nm CMOS technology and it costs 542 k gate count and 39.66 Kbytes on-chip memory while is capable to decode 60 frames/s for resolution with three quality layers at 135 MHz operating frequency.

Fast Adaptive Hierarchical Prediction Algorithm for H.264/AVC Scalable Video Coding

The objective of scalable video coding (SVC) is to enable the generation of a unique bitstream that can adapt to various bit-rates, transmission channels and display capabilities. The scalability is categorised in terms of temporal, spatial, and quality. In order to improve encoding efficiency, the SVC scheme incorporates inter-layer prediction mechanisms to complement the H.264/AVC very refined Motion Estimation (ME) and mode decision processes. However, this further increases the overall encoding complexity of the scalable coding standard. In this paper several conditional probabilities are established relating motion estimation characteristics and the mode distribution at different layers of the H.264/SVC. An evaluation of these probabilities is used to structure a low-complexity prediction algorithm for Group of Pictures (GOP) in H.264/SVC, reducing computational complexity whilst maintaining similar RD performance. When compared to the JSVM software, the proposed algorithm achieves a significant reduction of encoding time, with a negligible average PSNR loss and bit-rate increase in temporal, spatial and SNR scalability. Experiments are conducted to provide a comparison between our method and recently developed fast mode selection algorithms. These demonstrate the proposed method achieves appreciable time savings for scalable spatial and scalable quality video coding, while maintaining similar PSNR and bit rate.

An efficient region of interest support in scalable multi-view video coding

In this paper, a scalable multi-view video coding (SMVC) scheme is proposed to efficiently realize region of interest (ROI) support in SMVC. In addition, in order to further improve the coding efficiency, we develop a novel bit allocation model which can implement reasonable bit allocation among views. Extensive experiments are conducted on the typical multi-view video (MVV) sequences, and the efficiency of the presented codec is demonstrated. Meanwhile, the addressed SMVC architecture can easily provide temporal, spatial, SNR, view, and content scalabilities.

Rate-distortion optimization of scalable video codecs

In this paper joint optimization of layers in the layered video coding is investigated. Through theoretical analysis and simulations, it is shown that, due to higher interactions between the layers in a SNR scalable codec, this type of layering technique benefits most from joint optimization of the layers. A method for joint optimization is then proposed, and its compression efficiency is contrasted against the separate optimization and an optimized single layer coder. It is shown that, in joint optimization of SNR scalable coders when the quantization step size of the enhancement layer is larger than half the step size of the base layer, an additional improvement is gained by not sending the enhancement zero valued quantized coefficients, provided they are quantized at the base-layer. This will result in a non-standard bitstream syntax and as an alternative for standard syntax, one may skip the inter coded enhancement macroblocks. Through extensive tests it is shown that while separate optimization of SNR coders is inferior to single layer coder by more than 2 dB, with joint optimization this gap is reduced to 0.3–0.5 dB. We have shown that through joint optimization quality of the base layer video is also improved over the separate optimization. It is also shown that spatial scalability like SNR scalability does benefit from joint optimization, though not being able to exploit the relation between the quantizer step sizes. The amount of improvement depends on the interpolation artifacts of upsampled base-layer and the residual quantization distortion of this layer. Hence, the degree of improvement depends on image contents as well as the bit rate budget. Simulation results show that joint optimization of spatial scalable coders is about 0.5–1 dB inferior to the single layer optimized coder, where its separate optimization counterpart like SNR scalability is more than 2 dB worse.

Architectures for Fast Transcoding of H.264/AVC to Quality-Scalable SVC Streams

The scalable extension of H.264/AVC (SVC) was recently standardized, and offers scalability at a minor penalty in rate-distortion efficiency when compared to single-layer H.264/AVC coding. In SVC, a scaled version of the original video sequence can easily be extracted by dropping layers from the stream. However, most of the video content nowadays is still produced in a single-layer format. While decoding and reencoding is a possible solution to introduce scalability in the existing bitstreams, this is an approach which requires a tremendous amount of time and effort. In this paper, we show that transcoding can be used to intelligently derive scalable bitstreams from existing single-layer streams. We focus on SNR scalability, and introduce techniques that are able to create multiple quality layers in the bitstreams. We also discuss bitstream rewriting from SVC to H.264/AVC, and examine how our newly proposed architectures can benefit from the changes that were introduced for bitstream rewriting. Architectures with different rate distribution flexibility and computational complexity are discussed. Rate-distortion performance of transcoding is shown to be comparable to that of reencoding at a fraction of the time needed for the latter.

JPEG 2000: A Potential Solution for Archiving HDTV Content

Selecting an adequate file format for archiving has always been a complex issue. Several formats are available from a variety of vendors but in addition to the issue of storage media deterioration there is always the serious threat of technology becoming obsolete or no longer being supported by the vendor. Consequently, users experience interoperability during content exchange. Furthermore, with the introduction of HDTV in broadcasting and multiple distribution platforms (Mobile TV, T-DMB, DVB-H, IPTV) and image formats (720p/50, 1080p/50, etc.) compression scalability is now more of a requirement than before. Together with a need for higher quality preservation capability, metadata preservation is also a strong requirement for today's archive formats. JPEG 2000 is a fairly recent ISO open standard, based on wavelet technology. It provides a set of features such as spatial, temporal, and SNR scalability; lossless and lossy compression; and a large support for metadata, all of which are highly valuable for archiving HDTV content. The EBU Technical department has been investigating the JPEG 2000 compression system and, in particular, the best parameters for visually lossless preservation of HDTV content, depending on the format. The subjective quality was assessed at expert viewing sessions using a split screen for quality comparisons. This paper describes features of the JPEG 2000 standard, as well as its advantages and disadvantages in an archival environment. A performance evaluation of JPEG 2000 and recommendation on the JPEG 2000 settings suitable for HDTV archival is given.

Concealment of Whole-Picture Loss in Hierarchical B-Picture Scalable Video Coding

H.264/AVC scalable video coding (H.264/AVC SVC), as the scalable extension of H.264/AVC, offers the flexible adaptivity in terms of spatial, temporal and SNR scalabilities for the generated bitstream. However, such compressed video still suffers from the bad playback quality when packet loss occurs over unreliable networks. In this paper, we present an error concealment algorithm to tackle the whole-picture loss problem in H.264/AVC SVC when hierarchical B-picture coding is used to support temporal scalability. In the proposed algorithm, by taking advantage of the temporal relationship among the adjacent video pictures, the motion information of the lost picture is derived simply and efficiently based on the principle of temporal direct mode. Utilizing the derived motion information, the lost picture is concealed by performing motion compensation on the correctly received temporally previous and future video pictures. The experimental results demonstrate that as a post-processing tool, the proposed error concealment algorithm is able to significantly improve both the objective and subjective qualities of the decoded video pictures in the presence of packet losses when compared to the error concealment algorithm used in H.264/AVC SVC reference software. The proposed method can also be applied to H.264/AVC with hierarchical B-picture coding for error concealment.

Bit-depth scalability compatible to H.264/AVC-scalable extension

While eight-bit playback and display devices will be dominating the marketplaces in the near future, superior visual quality by high bit-depth videos is desirable for applications such as high standard entertainment and healthcare. Hence, conventional eight-bit and high bit-depth digital imaging systems will coexist in marketplaces. Content distributors supporting both of them need to provide different contents for different users, e.g., simulcastly code the different representations for the same video content. This requires more storage or bandwidth for video content delivery. Bit-depth scalability is an efficient tool to solve this problem. However, video coding techniques can allow flexible usage of various versions of the same visual content that may have spatial resolutions and even alterations in color. In this paper, we propose a bit-depth scalable coding solution that is compatible to the scalable extension of H.264/AVC, also known as Scalable Video Coding (SVC). The proposed bit-depth scalable coding is capable of providing an 8-bit AVC main profile or high profile compliant base layer which is multiplexed with a high bit depth (e.g., 10-, 12-, or up to 14-bit) enhancement layer through macroblock level inter-layer bit-depth prediction. New decoding processes for inter-layer bit-depth prediction are introduced to enable bit-depth scalability. Combination with other types of scalability: temporal, spatial and SNR scalability, as well as single-loop decoding is also supported since our algorithm is implemented based on the up-to-date SVC reference software. Furthermore, the proposed solution supports adaptive inter-layer prediction to determine whether or not the inter-layer bit-depth prediction shall be invoked for each Macroblock. The coding efficiency of the proposed bit-depth scalable coding can be further improved by incorporating advanced inter-layer bit-depth prediction algorithms. Experimental results are presented for 8-bit to high bit (10 or 12) scalability and also the combined bit-depth and spatial scalability.

Scalable and Media Aware Adaptive Video Streaming over Wireless Networks

This paper proposes an advanced video streaming system based on scalable video coding in order to optimize resource utilization in wireless networks with retransmission mechanisms at radio protocol level. The key component of this system is a packet scheduling algorithm which operates on the different substreams of a main scalable video stream and which is implemented in a so-called media aware network element. The concerned type of transport channel is a dedicated channel subject to parameters (bitrate, loss rate) variations on the long run. Moreover, we propose a combined scalability approach in which common temporal and SNR scalability features can be used jointly with a partitioning of the image into regions of interest. Simulation results show that our approach provides substantial quality gain compared to classical packet transmission methods and they demonstrate how ROI coding combined with SNR scalability allows to improve again the visual quality.

In-Scale Motion Compensation for Spatially Scalable Video Coding

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> In existing pyramid-based spatially scalable coding schemes, such as H.264/MPEG-4 SVC (scalable video coding), video frame at a certain high-resolution layer is mainly predicted either from the same frame at the next lower resolution layer, or from the temporal neighboring frames within the same resolution layer. But these schemes fail to exploit both kinds of correlation simultaneously and therefore cannot remove the redundancies among resolution layers efficiently. This paper extends the idea of spatiotemporal subband transform and proposes a general in-scale motion compensation technique for pyramid-based spatially scalable video coding. Video frame at each high-resolution layer is partitioned into two parts in frequency. Prediction for the lowpass part is derived from the next lower resolution layer, whereas prediction for the highpass part is obtained from neighboring frames within the same resolution layer, to further utilize temporal correlation. In this way, both kinds of correlation are exploited simultaneously and the cross-resolution-layer redundancy can be highly removed. Furthermore, this paper also proposes a macroblock-based adaptive in-scale technique for hybrid spatial and SNR scalability. Experimental results show that the proposed techniques can significantly improve the spatial scalability performance of H.264/MPEG-4 SVC, especially when the bit-rate ratio of lower resolution bit stream to higher resolution bit stream is considerable. </para>

Traffic and Quality Characterization of the H.264/AVC Scalable Video Coding Extension

The recent scalable video coding (SVC) extension to the H.264/AVC video coding standard has unprecedented compression efficiency while supporting a wide range of scalability modes, including temporal, spatial, and quality (SNR) scalability, as well as combined spatiotemporal SNR scalability. The traffic characteristics, especially the bit rate variabilities, of the individual layer streams critically affect their network transport. We study the SVC traffic statistics, including the bit rate distortion and bit rate variability distortion, with long CIF resolution video sequences and compare them with the corresponding MPEG‐4 Part 2 traffic statistics. We consider (i) temporal scalability with three temporal layers, (ii) spatial scalability with a QCIF base layer and a CIF enhancement layer, as well as (iii) quality scalability modes FGS and MGS. We find that the significant improvement in RD efficiency of SVC is accompanied by substantially higher traffic variabilities as compared to the equivalent MPEG‐4 Part 2 streams. We find that separately analyzing the traffic of temporal‐scalability only encodings gives reasonable estimates of the traffic statistics of the temporal layers embedded in combined spatiotemporal encodings and in the base layer of combined FGS‐temporal encodings. Overall, we find that SVC achieves significantly higher compression ratios than MPEG‐4 Part 2, but produces unprecedented levels of traffic variability, thus presenting new challenges for the network transport of scalable video.

Optimal Multilayer Adaptation of SVC Video over Heterogeneous Environments

Scalable video coding (SVC) is a new video coding format which provides scalability in three‐dimensional (spatio‐temporal‐SNR) space. In this paper, we focus on the adaptation in SNR dimension. Usually, an SVC bitstream may contain multiple spatial layers, and each spatial layer may be enhanced by several FGS layers. To meet a bitrate constraint, the fine‐grained scalability (FGS) data of different spatial layers can be truncated in various manners. However, the contributions of FGS layers to the overall/collective video quality are different. In this work, we propose an optimized framework to control the SNR scalability across multiple spatial layers. Our proposed framework has the flexibility in allocating the resource (i.e., bitrate) among spatial layers, where the overall quality is defined as a function of all spatial layers′ qualities and can be modified on the fly.

Scalable Motion Vector Coding Based on CABAC for MC-EZBC

In the scalable video coder MC-EZBC, the motion-vector (MV) bitstream was not scalable in bitrate or resolution. In this short paper, we enhance MC-EZBC with a new scalable MV coder based on context adaptive binary arithmetic coding. Alphabet general partition of MV symbols is proposed to achieve accuracy or quality scalability of MVs. A selective layered structure is used to reduce the number of MVs transmitted when appropriate, mainly needed for resolution scalability. With these two additions, we have a layered temporal, SNR, and resolution scalability for the MV bitstream. Experimentally, we find that this gives significant visual and objective improvement for low bitrates and/or resolutions with only very slight PSNR loss and unnoticeable visual loss at high bitrates

Content-Aware Scalability-Type Selection for Rate Adaptation of Scalable Video

Scalable video coders provide different scaling options, such as temporal, spatial, and SNR scalabilities, where rate reduction by discarding enhancement layers of different scalability-type results in different kinds and/or levels of visual distortion depend on the content and bitrate. This dependency between scalability type, video content, and bitrate is not well investigated in the literature. To this effect, we first propose an objective function that quantifies flatness, blockiness, blurriness, and temporal jerkiness artifacts caused by rate reduction by spatial size, frame rate, and quantization parameter scaling. Next, the weights of this objective function are determined for different content (shot) types and different bitrates using a training procedure with subjective evaluation. Finally, a method is proposed for choosing the best scaling type for each temporal segment that results in minimum visual distortion according to this objective function given the content type of temporal segments. Two subjective tests have been performed to validate the proposed procedure for content-aware selection of the best scalability type on soccer videos. Soccer videos scaled from 600 kbps to 100 kbps by the proposed content-aware selection of scalability type have been found visually superior to those that are scaled using a single scalability option over the whole sequence.