Rate-distortion Efficiency Research Articles

Optimal Adaptive Quantization Based on Temporal Distortion Propagation Model for HEVC.

Optimal adaptive quantization is one of the key points to optimize the coding efficiency of video encoders. The latest block-based video compression standards, such as high-efficiency video coding (HEVC), extensively use predictive coding techniques that create dependencies between blocks and increase the complexity of optimal block quantizers search. Specifically, the motion compensation is responsible for a dependency network connecting all blocks of the same GOP together. In this paper, this dependency network is estimated by a temporal distortion propagation model and an accurate estimation of Inter and Skip modes probabilities. Optimal quantizers are then designed per block in order to achieve global optimization in terms of rate-distortion efficiency. By implementing the algorithm into the HEVC reference model (HM), we report -16.51% PSNR-based and -26.26% SSIM-based average bitrate savings compared to no adaptive quantization. The proposed algorithm outperforms several related methods from the state-of-the-art. Moreover, along with the demonstration of an optimal quantizer solution, we propose an in-depth analysis of the algorithm behavior. This analysis includes, among others, the relative distribution of rates between frames and the control of quantizers dynamic range.

Proof-of-concept: role of generic content characteristics in optimizing video encoders

The influence of content characteristics on the efficiency of redundancy and irrelevance reduction in video coding is well known. Each new standard in video coding includes additional coding tools that potentially increase the complexity of the encoding process in order to gain further rate-distortion efficiency. In order to be versatile, encoder implementations often neglect the content dependency or they optimize the encoding complexity on a local scale, i.e. on a single frame or on the coding unit level without being aware of the global content type. In this contribution, an analysis is presented which coding tool settings of the recent High Efficiency Video Coding (HEVC) standard are most efficient for a given content type when balancing rate-distortion against computational complexity measured in encoding time. The content type is algorithmically determined, leading to a framework for rate-distortion-complexity based encoder parameter decision for any given video sequence. The implementability is demonstrated using a set of 35 Ultra-HD (UHD) sequences. The performance results and evaluations show that the encoding parameters may be predicted to optimize the video coding. For instance, predicting motion search range achieves complexity reduction of 36% on average when HEVC reference HM is used at a cost of bitrate (2%). When another HEVC coding standard software, x265, is used to predict the coding unit (CU) size, there is a reduction of 20% in bitrate and of 8% in distortion but there is a reduction of 6% in execution time.

Efficient Scalable Image Compression Algorithms with Low Memory and Complexity

The set partitioning embedded block (SPECK) image compression algorithm has excellent performance, low computational complexity, and produces a rate scalable compressed bitstream that can be decoded efficiently at multiple bit-rates. Unfortunately, it consumes a huge amount of computer memory due to employing lists that store the coordinates of the image pixels and the coordinates of the sets that are generated during the coding process. In addition, it has complex memory management due to using an array of random access linked lists to store these sets according to their sizes. In this paper, we propose two algorithms that are based on SPECK. The main contribution of the first algorithm is that, as compared to SPECK, the amount of the algorithm’s usable memory is reduced to about 75% and at the same time its processing speed is increased and its rate distortion efficiency is preserved as will be demonstrated. The second algorithm has higher processing speed but has slightly lower rate distortion performance than the first algorithm.

A parallel H.264/SVC encoder for high definition video conferencing

In this paper we present a video encoder specially developed and configured for high definition (HD) video conferencing. This video encoder brings together the following three requirements: H.264/Scalable Video Coding (SVC), parallel encoding on multicore platforms, and parallel-friendly rate control. With the first requirement, a minimum quality of service to every end-user receiver over Internet Protocol networks is guaranteed. With the second one, real-time execution is accomplished and, for this purpose, slice-level parallelism, for the main encoding loop, and block-level parallelism, for the upsampling and interpolation filtering processes, are combined. With the third one, a proper HD video content delivery under certain bit rate and end-to-end delay constraints is ensured. The experimental results prove that the proposed H.264/SVC video encoder is able to operate in real time over a wide range of target bit rates at the expense of reasonable losses in rate-distortion efficiency due to the frame partitioning into slices.

GPU-Based MPEG-2 to Secure Scalable Video Transcoding

Most of the high definition video content are still produced in a single-layer MPEG-2 format. Multiple-layers Scalable Video Coding (SVC) offers a minor penalty in rate-distortion efficiency when compared to single-layer coding MPEG-2. A scaled version of the original SVC bitstream can easily be extracted by dropping layers from the bitstream. This paper proposes a parallel transcoder from MPEG-2 to SVC video with Graphics Processing Unit (GPU), named PTSVC. The objective of the transcoder is to migrate MPEG-2 format video to SVC format video such that clients with different network bandwidth and terminal devices can seamlessly access video content. Meanwhile, the transcoded SVC videos are encrypted such that only authorized users can access corresponding SVC layers. Using various scalabilities SVC test sequences, experimental results on TM5 and JSVM indicate that PTSVC is a higher efficient transcoding system compared with previous systems and only causes little quality loss.

Transmission Policy Selection for Multi-View Content Delivery Over Bandwidth Constrained Channels.

I formulate an optimization framework for computing the transmission actions of streaming multi-view video content over bandwidth constrained channels. The optimization finds the schedule for sending the packetized data that maximizes the reconstruction quality of the content, for the given network bandwidth. Two prospective multi-view content representation formats are considered: 1) MVC and 2) video plus depth. In the case of each, I formulate directed graph models that characterize the interdependencies between the data units that comprise the content. For the video plus depth format, I develop a novel space-time error concealment strategy that reconstructs the missing content based on received data units from multiple views. I design multiple techniques to solve the optimization problem of interest, at varying degrees of complexity and accuracy. In conjunction, I derive spatiotemporal models of the reconstruction error for the multi-view content that I employ to reduce the computational requirements of the optimization. I study the performance of my framework via simulation experiments. Significant gains in terms of rate-distortion efficiency are demonstrated over various reference methods.

Reducing 3D video coding complexity through more efficient disparity estimation

3D video coding for transmission exploits the Disparity Estimation (DE) to remove the inter-view redundancies present within both the texture and the depth map multi-view videos. Good estimation accuracy can be achieved by partitioning the macro-block into smaller subblocks partitions. However, the DE process must be performed on each individual sub-block to determine the optimal mode and their disparity vectors, in terms of ratedistortion efficiency. This vector estimation process is heavy on computational resources, thus, the coding computational cost becomes proportional to the number of search points and the inter-view modes tested during the rate-distortion optimization. In this paper, a solution that exploits the available depth map data, together with the multi-view geometry, is proposed to identify a better DE search area; such that it allows a reduction in its search points. It also exploits the number of different depth levels present within the current macro-block to determine which modes can be used for DE to further reduce its computations. Simulation results demonstrate that this can save up to 95% of the encoding time, with little influence on the coding efficiency of the texture and the depth map multi-view video coding. This makes 3D video coding more practical for any consumer devices, which tend to have limited computational power.

Study of H.264/Avc Algorithm and It’s Implementation In Matlab

H.264/AVC(Advanced Video Coding) is a video coding standard of the ITU-T(International Telecommunication Union for Telecommunication Standardization Sector) Video Coding Experts Group and the ISO/IEC(International Standards Organization/International Electrotechnical Commission) Moving Picture Experts Group. The main goals of the H.264/AVC standardization effort have been enhanced compression performance and provision of a network friendly video representation addressing conversational (video telephony) and no conversational (storage, broadcast, or streaming) applications. H.264/AVC has achieved a significant improvement in rate-distortion efficiency relative to existing standards. This article provides an overview of the technical features of H.264/AVC, describes profiles and applications for the standard, and compared it with H.261 and H.263.

A complexity-scalable hybrid H.264/AVC-to-SVC transcoder

Transcoding of H.264/AVC bitstreams is frequently used for adapting video streams to varying network conditions or different end-user device characteristics. Transcoding requires a relatively high complexity (and thus high operational cost) for each adaptation operation, and possibly reduces the video quality. In comparison, transcoding the input bitstream to SVC requires only one adaptation step. Afterwards, the resulting SVC bitstream can be adapted instantly. To reduce the H.264/AVC-to-SVC transcoding complexity, a transcoding architecture which combines existing open-loop transcoding techniques with optimized closed-loop transcoding is proposed. Furthermore, this approach allows to scale the complexity of the system. Doing so, compared with a cascaded decoder-encoder only 8.48% of the complexity is required for the highest complexity, while a high rate-distortion efficiency is maintained and a high degree of scalability is guaranteed. When only limited resources are available, the transcoding complexity can be scaled such that over 99% of the complexity is reduced compared to a cascaded decoder-encoder.

Consistent Visual Quality Control in Video Coding

Visual quality consistency is one of the most important issues in video quality assessment. When people view a sequential video, they may have an unpleasant perceptual experience if the video has an inconsistent visual quality even though the average visual quality of the video is not compromised. Thus, consistent visual quality control is mostly expected in general video encoding with limited channel bandwidth and buffer resources. However, there still has not been enough study on such an issue. In this paper, a new objective visual quality metric (VQM) is proposed first, which can easily be incorporated into video coding for guiding video coding. Second, a VQM-based window model is proposed to handle the tradeoff between visual quality consistency and buffer constraint in video coding. Third, a window-level rate control algorithm is developed to accomplish visual quality control based on the above two proposals. Finally, experimental results prove that consistent visual quality, high rate-distortion efficiency, accurate bit control, and compliant buffer constraint can be achieved by the proposed rate control algorithm.

Multi-path content delivery: Efficiency analysis and optimization algorithms

The paper studies the benefits of multi-path content delivery from a rate-distortion efficiency perspective. We develop an optimization framework for computing transmission schedules for streaming media packets over multiple network paths that maximize the end-to-end video quality, for the given bandwidth resources. We comprehensively address the two prospective scenarios of content delivery with packet path diversity. In the context of sender-driven systems, our framework enables the sender to compute at every transmission instance the mapping of packets to network paths that meets a rate constraint while minimizing the end-to-end distortion. In receiver-driven multi-path streaming, our framework enables the client to dynamically decide which packets, if any, to request for transmission and from which media servers, such that the end-to-end distortion is minimized for a given transmission rate constraint. Via simulation experiments, we carefully examine the performance of the scheduling framework in both multi-path delivery scenarios. We demonstrate that the optimization framework closely approaches the performance of an ideal streaming system working at channel capacity with an infinite play-out delay. We also show that the optimization leads to substantial gains in rate-distortion performance over a conventional content-agnostic scheduler. Through the concept of error-cost performance for streaming a single packet, we provide another useful insight into the operation of the optimization framework and the conventional scheduling system.

Generalized Hybrid Intra and Wyner-Ziv Video Coding

The hybrid Wyner-Ziv (WZ) and intra video coding system has many interesting features, like the possibility to perform real motion estimation, at the decoder, and low latency coding. In this letter, we present an improved system and we will generalize it to include the other types of WZ coding. Thus, we propose a new quantization and a new cyclic redundancy check checking technique and, most importantly, we propose many new coding modes to produce a more efficient system that exploits the strong features of each type. The resulting system can thus produce many types of hybrid frames and allows mode adaptive coding. The simulation results show an improvement of the rate-distortion efficiency and can largely outperform those of state of the art.

Rate Distortion Analysis for Spatially Scalable Video Coding

In this paper, we derive the rate distortion lower bounds of spatially scalable video coding techniques. The methods we evaluate are subband and pyramid motion compensation where temporal redundancies in the same spatial layer as well as interlayer spatial redundancies are exploited in the enhancement layer encoding. The rate distortion bounds are derived from rate distortion theory for stationary Gaussian signals where mean square error is used as the distortion criteria. Assuming that the base layer is encoded by a non-scalable video coder, we derive the rate distortion functions for the enhancement layer, which depend on the power spectral density of the input signal, the motion prediction error probability density function and the base layer encoding performance. We will show that pyramid and subband methods are expected to outperform independently encoding the enhancement layer using motion-compensated prediction, in terms of rate distortion efficiency, when the base layer is encoded at a relatively higher quality or less accurate displacement estimation happens in the enhancement layer.

Sub-pixel motion estimation using kernel methods

Modern video codecs such as MPEG2, MPEG4-ASP and H.264 depend on sub-pixel motion estimation to optimise rate-distortion efficiency. Sub-pixel motion estimation is implemented within these standards using interpolated values at 1/2 or 1/4 pixel accuracy. By using these interpolated values, the residual energy for each predicted macroblock is reduced. However this leads to a significant increase in complexity at the encoder, especially for H.264, where the cost of an exhaustive set of macroblock segmentations needs to be estimated for optimal mode selection. This paper presents a novel scheme for sub-pixel motion estimation based on the whole-pixel SAD distribution. Both half-pixel and quarter-pixel searches are guided by a model-free estimation of the SAD surface using a two dimensional kernel method. While giving an equivalent rate distortion performance, this approach approximately halves the number of quarter-pixel search positions giving an overall speed up of approximately 10% compared to the EPZS quarter-pixel method (the state of the art H.264 optimised sub-pixel motion estimator).

Architectures for Fast Transcoding of H.264/AVC to Quality-Scalable SVC Streams

The scalable extension of H.264/AVC (SVC) was recently standardized, and offers scalability at a minor penalty in rate-distortion efficiency when compared to single-layer H.264/AVC coding. In SVC, a scaled version of the original video sequence can easily be extracted by dropping layers from the stream. However, most of the video content nowadays is still produced in a single-layer format. While decoding and reencoding is a possible solution to introduce scalability in the existing bitstreams, this is an approach which requires a tremendous amount of time and effort. In this paper, we show that transcoding can be used to intelligently derive scalable bitstreams from existing single-layer streams. We focus on SNR scalability, and introduce techniques that are able to create multiple quality layers in the bitstreams. We also discuss bitstream rewriting from SVC to H.264/AVC, and examine how our newly proposed architectures can benefit from the changes that were introduced for bitstream rewriting. Architectures with different rate distribution flexibility and computational complexity are discussed. Rate-distortion performance of transcoding is shown to be comparable to that of reencoding at a fraction of the time needed for the latter.

Idempotent H.264 intraframe compression

This paper studies H.264 intraframe multi-generation characteristics and presents four mechanisms that contribute to the generation degradation. We analyze the H.264 transform and quantization and reveal that for some quantization parameters there always exists at least one clipping compensation matrix to make the H.264 intraframe multi-generation coding idempotent as long as the same prediction mode and the same quantization parameter are selected for each coding generation. In addition, an idempotent H.264 intraframe multi-generation coding procedure is presented. The procedure has three additional steps to achieve idempotence: Prediction Mode and QP Restriction, Prediction Mode and QP Identification, and Clipping Compensation. Experiment results show that the proposed idempotent coding produces no generation PSNR loss and the rate-distortion efficiency is almost same as that of traditional H.264.

Video transcoding from H.264/AVC to MPEG-2 with reduced computational complexity

This paper addresses video transcoding from H.264/AVC into MPEG-2 with reduced complexity and high rate-distortion efficiency. While the overall concept is based on a cascaded decoder–encoder, the novel adaptation methods developed in this work have the advantage of providing very good performance in H.264/AVC to MPEG-2 transcoding. The proposed approach exploits the similarities between the coding tools used in both standards, with the objective of obtaining a computationally efficient transcoder without penalising the signal quality. Fast and efficient methods are devised for conversion of macroblock coding modes and translation of motion information in order to compute the MPEG-2 coding format with a reduced number of operations, by reusing the corresponding data embedded in the incoming H.264/AVC coded stream. In comparison with a cascaded decoder–encoder, the fast transcoder achieves computational complexity savings up to 60% with slightly better peak signal-to-noise ratio (PSNR) at the same bitrate.

VLSI Architecture Design of Fractional Motion Estimation for H.264/AVC

The H.264/AVC Fractional Motion Estimation (FME) with rate-distortion constrained mode decision can improve the rate-distortion efficiency by 2–6 dB in peak signal-to-noise ratio. However, it comes with considerable computation complexity. Acceleration by dedicated hardware is a must for real-time applications. The main difficulty for FME hardware implementation is parallel processing under the constraint of the sequential flow and data dependency. We analyze seven inter-correlative loops extracted from FME procedure and provide decomposing methodologies to obtain efficient projection in hardware implementation. Two techniques, 4×4 block decomposition and efficiently vertical scheduling, are proposed to reuse data among the variable block size and to improve the hardware utilization. Besides, advanced architectures are designed to efficiently integrate the 6-taps 2D finite impulse response, residue generation, and 4×4 Hadamard transform into a fully pipelined architecture. This design is finally implemented and integrated into an H.264/AVC single chip encoder that supports realtime encoding of 720×480 30fps video with four reference frames at 81 MHz operation frequency with 405 K logic gates (41.9% area of the encoder).

Efficient Macroblock Mode Derivation in ESS Based on Local Characteristics of Video Objects

To improve the efficiency of inter-layer prediction on macblock (MB) mode in extended spatial scalability (ESS), a mode complexity parameter derived from local MBs in the base layer is used to early terminate the MB mode derivation in the enhancement layer. Novel criteria to merge enhancement-layer 4times4 blocks into a large MB sub-partition are also proposed. Experimental results show that our method can averagely increase the number of MBs in 16 times 16 mode in the enhancement layer more than 30% and reduce the number of MBs in small size partition mode with almost no rate-distortion efficiency loss compared to joint video team (JVT)-V108 under the test conditions specified in JVT-V302.

Fast mode decision on H.264/AVC baseline profile for real-time performance

In this paper a new fast mode decision (FMD) algorithm is proposed for the recent H.264/AVC video coding standard, aiming to reduce its computational load without loosing coding efficiency. This algorithm identifies redundancy and selects the minimum sub-set of modes for each macroblock (MB) required to provide high rate-distortion (RD) efficiency. It is based on a fast analysis of the histogram of the difference image between frames which classifies the areas of each frame as active or non-active by means of an adaptive thresholding technique. More coding effort is devoted to active areas with the selection of a large sub-set of Modes, as these areas are expected to be the most relevant in terms of RD cost. Results show reduction values around 35–65% of motion estimation (ME) time, preserving the RD cost for the Baseline Profile, by using P-Slices and without needing B-Slices. Moreover, the strategy works as an intelligent tool for real-time applications with constrained number of operations per frame: it wisely uses the given operational resources distributing them among those MBs that need it.