Scalable Bitstream Research Articles

An efficient and Highly Scalable Listless SPIHT Image Compression Framework

The set partitioning in hierarchical trees is a powerful image compression algorithm. It has reasonable complexity and produces a rate scalable bit-stream. Unfortunately, SPIHT fails to explore the multi-resolution nature of the wavelet transform as the output bit-stream doesn't support resolution scalability. Moreover it requires huge memory and has complex memory management as it depends on utilizing lists with memory of about 2.5 the image size. This paper proposes three related algorithms. The first one modifies SPIHT to reduce its complexity and improve its efficiency especially at low rates. The second is the main contribution of the paper. It provides a simultaneous solution to the memory and scalability problems of SPIHT. Memory is reduced by utilizing status bits of average 2.5 bits per pixel instead of the lists. Resolution scalability is maintained by encoding the resolution levels in increasing order within each coding pass. Another important attribute of our algorithm is that it has very little increment in complexity in comparison with the original SPIHT algorithm. In contrast, the existing solutions have much more complexity, and/or more memory resources. The third has slightly lower complexity and memory than the second but at the same time, it has slightly lower performance.

SSSIC: Semantics-to-Signal Scalable Image Coding With Learned Structural Representations.

We address the requirement of image coding for joint human-machine vision, i.e., the decoded image serves both human observation and machine analysis/understanding. Previously, human vision and machine vision have been extensively studied by image (signal) compression and (image) feature compression, respectively. Recently, for joint human-machine vision, several studies have been devoted to joint compression of images and features, but the correlation between images and features is still unclear. We identify the deep network as a powerful toolkit for generating structural image representations. From the perspective of information theory, the deep features of an image naturally form an entropy decreasing series: a scalable bitstream is achieved by compressing the features backward from a deeper layer to a shallower layer until culminating with the image signal. Moreover, we can obtain learned representations by training the deep network for a given semantic analysis task or multiple tasks and acquire deep features that are related to semantics. With the learned structural representations, we propose SSSIC, a framework to obtain an embedded bitstream that can be either partially decoded for semantic analysis or fully decoded for human vision. We implement an exemplar SSSIC scheme using coarse-to-fine image classification as the driven semantic analysis task. We also extend the scheme for object detection and instance segmentation tasks. The experimental results demonstrate the effectiveness of the proposed SSSIC framework and establish that the exemplar scheme achieves higher compression efficiency than separate compression of images and features.

An MMT-Based Hierarchical Transmission Module for 4K/120fps Temporally Scalable Video

High frame rate (HFR) video is attracting strong interest since it is considered as a next step toward providing Ultra-High Definition video service. For instance, the Association of Radio Industries and Businesses (ARIB) standard, the latest broadcasting standard in Japan, defines a 120 fps broadcasting format. The standard stipulates temporally scalable coding and hierarchical transmission by MPEG Media Transport (MMT), in which the base layer and the enhancement layer are transmitted over different paths for flexible distribution. We have developed the first ever MMT transmitter/receiver module for 4K/120fps temporally scalable video. The module is equipped with a newly proposed encapsulation method of temporally scalable bitstreams with correct boundaries. It is also designed to be tolerant to severe network constraints, including packet loss, arrival timing offset, and delay jitter. We conducted a hierarchical transmission experiment for 4K/120fps temporally scalable video. The experiment demonstrated that the MMT module was successfully fabricated and capable of dealing with severe network constraints. Consequently, the module has excellent potential as a means to support HFR video distribution in various network situations.

Efficient and simple scalable image compression algorithms

Abstract Most wavelet-based scalable image compression algorithms are optimized for performance only, i.e., they try to produce a compressed bit-stream that gives the best image quality for the selected bit-rate. Unfortunately, the optimization of the coding method may result in a bit-stream that cannot be rearranged easily to support resolution scalability. This paper proposes two algorithms. The first algorithm produces a rate scalable bit-stream that is optimized for performance and scalability. That is, it has good performance and at the same time it produces bit-stream that can be easily rearranged to become highly (rate and resolution) scalable. To this end, the algorithm combines the low complexity and rate scalability features of the set partitioning coding approach, and it exploits the resolution scalability characteristics of the block-based systems. The second algorithm upgrades the first algorithm to produce a highly scalable bit-stream by only arranging and identifying the different resolutions within the bit-stream.

Real-Time Fidelity Measurement of JPEG2000 Coded Images

The progressive nature of the JPEG2000 coded bitstream allows the reconstruction of images of different qualities from a single coded bitstream. This feature is utilized in this work to estimate the mean-squared-error (MSE) of reconstructed images without requiring the original image. It is based on the fact that if the MSE between the original image and a lower quality image is known, the MSE for higher quality images can be estimated from a quality scalable bitstream. The proposed method is highly accurate and is very simple as no complex statistical modeling is needed. Therefore, it is suitable to measure the fidelity of JPEG2000 decoded images at any desired quality in a real-time scenario.

Bitstream-Oriented Protection for the H.264/Scalable Video Coding (SVC)

The newly standardized H.264/SVC enable multimedia suppliers to provide video bitstream with temporal, spatial and quality scalabilities to meet various needs of end users. Its bitstream scalability characteristic is also especially suitable for multimedia applications under merged heterogeneous networks with different network protocols, capacities and throughputs. Although the network environment, to some extent, provides security protection against the illegal users, it is not enough to protect against the authorized users from digesting contents beyond its authorization. Hence the issue of content protection for the H.264/SVC has aroused researchers’ interests in recent years. In this paper, we present an efficient bitstream-oriented protection scheme for the H.264/SVC in a compression friendly and format compliant manner. The encryption is implemented on the Network Abstraction Layer (NAL) unit level. To improve the computational efficiency as well as to provide sufficient security, selective cryptographic algorithms with different computation cost are employed for different content level of SVC according to its significance. The I slice NAL units from the base layer of SVC bitstream are encrypted with the symmetric AES algorithm in Cipher Block Chaining mode. And the other NAL units are protected by simplified XOR cipher. Furthermore, a robust key management (key generation and distribution) mechanism is also discussed in this paper. The security analysis and simulation results further verify that the proposed methods can effectively protect the H.264/SVC bitstream at low computational cost.

Soft decoding algorithms for optimized JPEG 2000 wireless transmission over realistic MIMO-OFDM systems

In this paper, we investigate a new cross layer PHYsical/APPlication (PHY-APP) communication strategy for scalable JPEG 2000 wireless image transmission over a realistic Multiple-Input Multiple-Output (MIMO) system. To exploit the channel diversity, we use a closed-loop MIMO-OFDM scheme. In the proposed scheme, the MIMO channel is decomposed into several hierarchical Single-Input Single-Output (SISO) subchannels by using a precoding approach. The scalable bitstream is divided into hierarchical quality layers which are passing through these SISO subchannels. In this paper, we propose a Joint Source-Channel (JSC) decoding approach based on soft-inputs decoding techniques to decrease the error rates at the reception without introducing extra redundancy. This scheme involves the serial concatenation of a soft-input soft-output Reed-Solomon (RS) decoder and a soft-input arithmetic decoder that were integrated into the JPEG 2000 wireless decoder. The objective of our approach is to guarantee the Quality of Service (QoS) required by the user for varying channel states. To this end, a link adaption strategy adjusting all the systems parameters of each SISO sub-channel (number of used subchannels, modulation order, Forward Error Correction (FEC) code capability, source coding rate) is also adopted in order to maximize the image visual quality at the reception. Thus, Unequal Error Protection (UEP), Unequal Power Allocation (UPA), adaptive modulation and source coding rate are provided for each quality layer. Simulation results of the optimized adaptive strategy illustrate good image quality improvements at the receiver side compared to a non adaptive strategy, with significant Peak Signal to Noise Ratio (PSNR) gains, especially for a realistic noisy channel provided by a 3D ray-tracing software.

An improved 3D wavelet-based scalable video coding codec for MC-EZBC

With the rapid growth of modern multimedia applications, 3D wavelet-based scalable video coding (SVC) codec has received considerable attention lately because of its high coding performance and flexibility in bitstream scalability. It combines the motion-compensated temporal filtering (MCTF) together with the spatial decomposition to produce an embedded bitstream offering various levels of video quality over the heterogeneous networks. However, in the existing 3D wavelet-based SVC schemes, where the block types for block matching algorithms are limited, weighting matrices for block-wise motion compensation are fixed, and variations in activities of temporal subbands are not considered in the selection of the Lagrange multiplier for mode decision. In this paper, our major contribution is to provide some recent extensions to the well-known scalable subband/wavelet video codec Motion-Compensated Embedded Zero Block Coding (MC-EZBC) using three novel and content adaptive algorithms. Firstly, the enhanced hierarchical variable size block matching (Enhanced HVSBM) algorithm is proposed for the variable block size motion estimation. Then, the rate-distortion optimization (RDO) based adaptive Lagrange multiplier selection model for mode decision is presented. Finally, we introduce the adaptive weighting matrices design for overlapped block motion compensation (OBMC). Experimental results show that all the three proposed algorithms significantly improve the overall coding performance of MC-EZBC. Comparisons with other popular wavelet-based SVC codecs demonstrate the effectiveness of our improved codec in terms of both video quality assessment and computational complexity.

English

Wavelet-based image compression algorithms have good performance and produce a rate scalable bitstream that can be decoded efficiently at several bit rates. Unfortunately, the discrete wavelet transform (DWT) has relatively high computational complexity. On the other hand, the discrete cosine transform (DCT) has low complexity and excellent compaction properties. Unfortunately, it is non-local, which necessitates implementing it as a block-based transform leading to the well-known blocking artifacts at the edges of the DCT blocks. This paper proposes a very fast and rate scalable algorithm that exploits the low complexity of DCT and the low complexity of the set partitioning technique used by the wavelet-based algorithms. Like JPEG, the proposed algorithm first transforms the image using block-based DCT. Then, it rearranges the DCT coefficients into a wavelet-like structure. Finally, the rearranged image is coded using a modified version of the SPECK algorithm, which is one of the best well-known wavelet-based algorithms. The modified SPECK consumes slightly less computer memory, has slightly lower complexity and slightly better performance than the original SPECK. The experimental results demonstrated that the proposed algorithm has competitive performance and high processing speed. Consequently, it has the best performance to complexity ratio among all the current rate scalable algorithms.

A pixel-based outlier-free motion estimation algorithm for scalable video quality enhancement

Scalable video quality enhancement refers to the process of enhancing low quality frames using high quality ones in scalable video bitstreams with time-varying qualities. A key problem in the enhancement is how to search for correspondence between high quality and low quality frames. Previous algorithms usually use block-based motion estimation to search for correspondences. Such an approach can hardly estimate scale and rotation transforms and always introduces outliers to the motion estimation results. In this paper, we propose a pixel-based outlier-free motion estimation algorithm to solve this problem. In our algorithm, the motion vector for each pixel is calculated with respect to estimate translation, scale, and rotation transforms. The motion relationships between neighboring pixels are considered via the Markov random field model to improve the motion estimation accuracy. Outliers are detected and avoided by taking both blocking effects and matching percentage in scale-invariant feature transform field into consideration. Experiments are conducted in two scenarios that exhibit spatial scalability and quality scalability, respectively. Experimental results demonstrate that, in comparison with previous algorithms, the proposed algorithm achieves better correspondence and avoids the simultaneous introduction of outliers, especially for videos with scale and rotation transforms.

Temporal video transcoding from H.264/AVC-to-SVC for digital TV broadcasting

Mobile digital TV environments demand flexible video compression like scalable video coding (SVC) because of varying bandwidths and devices. Since existing infrastructures highly rely on H.264/AVC video compression, network providers could adapt the current H.264/AVC encoded video to SVC. This adaptation needs to be done efficiently to reduce processing power and operational cost. This paper proposes two techniques to convert an H.264/AVC bitstream in Baseline (P-pictures based) and Main Profile (B-pictures based) without scalability to a scalable bitstream with temporal scalability as part of a framework for low-complexity video adaptation for digital TV broadcasting. Our approaches are based on accelerating the interprediction, focusing on reducing the coding complexity of mode decision and motion estimation tasks of the encoder stage by using information available after the H.264/AVC decoding stage. The results show that when our techniques are applied, the complexity is reduced by 98 % while maintaining coding efficiency.

H.264/AVC-to-SVC temporal video transcoder for video broadcasting in wireless networks

Over the last few decades multimedia usage has changed dramatically, with networks and terminals of diverse bandwidths and capabilities coexisting. To ensure a good quality of experience, this diverse environment requires adaptability of the video stream. If video streams were compressed using Scalable Video Coding (SVC) schemes, they would be able to adapt to these heterogeneous networks and a wide range of terminals. However, since the majority of multimedia contents are compressed using H.264/AVC, they cannot benefit from this scalability. This paper makes some proposals for improving a temporal H.264/AVC-to-SVC video transcoder so that it can convert an H.264/AVC bitstream without scalability to scalable bitstreams with temporal scalability. The idea behind the proposed transcoder is to accelerate interprediction, focusing on reducing the coding complexity of the mode decision and motion estimation tasks of the encoder stage by using information available after the H.264/AVC decoding stage. To accelerate the mode decision task, a decision tree for narrowing down the Macroblock (MB) types to be checked by the SVC encoder has been developed. On the other hand, to accelerate the motion estimation task, a reduced area is created dynamically for every MB and sub-MB. Moreover, time savings increase considerably when both proposals are adjusted to work together, as opposed to doing so in an isolated way. The results show that when our joint proposal is applied, the complexity is reduced by 98 % while maintaining coding efficiency.

BER-Driven Resource Allocation for Scalable Bitstreams Over OFDMA Networks

Conventional resource-allocation schemes in orthogonal frequency-division multiple-access (OFDMA) systems consider only one bit-error-rate (BER) requirement. When these schemes are applied to transmit bitstreams with multiple BER requirements, low resource utilization will occur since the target BER has to be the strictest one. We develop a BER-driven resource allocation for generic bitstreams (BRA-G) algorithm by exploiting multiple predetermined target BERs. Then, this scheme is extended to transmission of realistic scalable bitstreams (BRA-SB) where more important (MI) layers possess lower target BERs as compared with less important (LI) layers. The objective here is to maximize the expected quality of all JPEG 2000 (J2K) scalable bitstreams subject to a total power constraint. Instead of using static target BERs for different layers, BRA-SB water-fills over all layers to determine the number of transmit layers and the target BER of each layer simultaneously based on the importance of each layer in terms of reconstructed quality and average channel conditions. Bit-level results show that BRA-SB provides significant image quality improvement over a scheme that ignores the layer's importance in terms of target BER and a BRA-SB-like scheme with static target BERs. A suboptimal algorithm is also proposed to reduce the computational complexity and its performance is close to the optimal one.

Evaluation of H.264/AVC Coding Elements and New Improved Scalability/Adaptation Algorithm/Methods

In this paper, we evaluated the major coding elements/parameters of H.264/AVC and their significant differences with the prior standards. The design parameters of H.264/AVC codec are experimented with and analysed. The influences of the parameters and other coding elements on scalable bitstream and their level of adaptation over multi-channel networks are also evaluated. Objective and real-time analysis are employed in the evaluation. To enhance the flexibility and robustness of codec adaptation over heterogeneous networks, Qp Constraint Algorithm is developed that reveals bits reduction up to 100kbits depending on the current bitrates. The usage of slice groups is implemented and the result shows that, the implementation of slice groups can support flexible adaptation and data protection. We also propose and experimentally evaluate methods for multi-channels adaptation. Key-Words: Coding Elements, H.264/AVC, Scalability, Adaptation, Evaluation

Highly Scalable Single List Set Partitioning in Hierarchical Trees Image Compression

A highly scalable image compression system produces a compressed image that can be easily decoded at several bit-rates (qualities) and resolutions (sizes). This is a very interesting feature in the heterogeneous environments of the Internet to satisfy the different user's requests. This paper proposes a new algorithm termed Highly Scalable-Single List-SPIHT (HS-SLS). The proposed HS-SLS algorithm produces a highly scalable bit-stream that is both rate and resolution scalable. The flexible bit-stream of the HS-SLS encoder can easily be adapted to various resolution requirements at any bit-rate using a very simple scaling process that is performed on-the-fly and without the need to decode the bit-stream. A unique feature of the new algorithm is that it has low memory requirement and low complexity due to its simple memory management. In addition, the size of the usable memory is fixed and thus can be predefined which avoids the dynamic memory allocation problem. These features make the HS-SLS algorithm very suitable for hardware implementation. The price paid for these valuable features is very slight decrement in the algorithm's performance as compared to the original SPIHT algorithm. Index Terms: Highly Scalable Image Compression, Low Memory Image Compression, Rate scalability, Resolution scalability, Set Partitioning image Compression, SPIHT, Wavelet image compression, Zero-tree coding.

Scalable video transcoding for mobile communications

Mobile multimedia contents have been introduced in the market and their demand is growing every day due to the increasing number of mobile devices and the possibility to watch them at any moment in any place. These multimedia contents are delivered over different networks that are visualized in mobile terminals with heterogeneous characteristics. To ensure a continuous high quality it is desirable that this multimedia content can be adapted on-the-fly to the transmission constraints and the characteristics of the mobile devices. In general, video contents are compressed to save storage capacity and to reduce the bandwidth required for its transmission. Therefore, if these compressed video streams were compressed using scalable video coding schemes, they would be able to adapt to those heterogeneous networks and a wide range of terminals. Since the majority of the multimedia contents are compressed using H.264/AVC, they cannot benefit from that scalability. This paper proposes a technique to convert an H.264/AVC bitstream without scalability to a scalable bitstream with temporal scalability as part of a scalable video transcoder for mobile communications. The results show that when our technique is applied, the complexity is reduced by 98 % while maintaining coding efficiency.

Low-complexity transcoding algorithm from H.264/AVC to SVC using data mining

Nowadays, networks and terminals with diverse characteristics of bandwidth and capabilities coexist. To ensure a good quality of experience, this diverse environment demands adaptability of the video stream. In general, video contents are compressed to save storage capacity and to reduce the bandwidth required for its transmission. Therefore, if these compressed video streams were compressed using scalable video coding schemes, they would be able to adapt to those heterogeneous networks and a wide range of terminals. Since the majority of the multimedia contents are compressed using H.264/AVC, they cannot benefit from that scalability. This paper proposes a low-complexity algorithm to convert an H.264/AVC bitstream without scalability to scalable bitstreams with temporal scalability in baseline and main profiles by accelerating the mode decision task of the scalable video coding encoding stage using machine learning tools. The results show that when our technique is applied, the complexity is reduced by 87% while maintaining coding efficiency.

An Efficient Inter-Prediction Mode Decision Method for Temporal Scalability Coding With Hierarchical B-Picture Structure

Although the temporal scalability coding supported by both H.264/AVC and H.264/SVC provides a good means of adapting bitstream scalability in temporal dimension, it entails heavy computational complexity due to bidirectional motion estimation and compensation for B-pictures in temporal scalability levels. To make it more applicable, this paper introduces a fast inter-prediction mode decision method which is efficiently performed using statistical hypothesis testing. The hypothesis testing is performed on mean and variance of pixels in 16 × 16 and 8 × 8 blocks to decide early termination against further processing for their sub-blocks. Fallback check is employed to compromise the tradeoff between the compression efficiency and encoding time saving. The proposed method exhibits effective performance for early termination in the inter-prediction mode decision, thus leading to a significant reduction of the total encoding time up to average 64.8% with small increments in bit amounts and small degradation in visual quality.

Modeling of Rate and Perceptual Quality of Compressed Video as Functions of Frame Rate and Quantization Stepsize and Its Applications

This paper first investigates the impact of frame rate and quantization on the bit rate and perceptual quality of compressed video. We propose a rate model and a quality model, both in terms of the quantization stepsize and frame rate. Both models are expressed as the product of separate functions of quantization stepsize and frame rate. The proposed models are analytically tractable, each requiring only a few content-dependent parameters. The rate model is validated over videos coded using both scalable and nonscalable encoders, under a variety of encoder settings. The quality model is validated only for a scalable video, although it is expected to be applicable to a single-layer video as well. We further investigate how to predict the model parameters using the content features extracted from original videos. Results show accurate bit rate and quality prediction (average Pearson correlation >;0.99) can be achieved with model parameters predicted using three features. Finally, we apply rate and quality models for rate-constrained scalable bitstream adaptation and frame rate adaptive rate control. Simulations show that our model-based solutions produce better video quality compared with conventional video adaptation and rate control.

A game theoretic approach to minimum-delay scalable video transmission over P2P

In this paper we describe a game theoretic framework for scalable video streaming over a peer-to-peer network. The proposed system integrates minimum delay functionalities with an incentive provision mechanism for optimal resource allocation. First of all, we introduce an algorithm for packet scheduling that allows users to download a specific sub-set of the original scalable bit-stream, depending on the current network conditions. Furthermore, we present an algorithm that aims both at identifying free-riders and minimising the transmission delay. Uncooperative peers are cut out of this system, while users upload more data to those which have less to share, in order to fully exploit the resources of all peers. Experimental evaluation shows that the proposed model can effectively cope with free-riders and minimise the transmission delay for scalable video transmission by exploiting a packet scheduling algorithm, game theory, and a minimum-delay algorithm.