Intra-frame Encoder Research Articles

Partition Map Prediction for Fast Block Partitioning in VVC Intra-Frame Coding

The Versatile Video Coding (VVC) standard introduces a block partitioning structure known as quadtree plus nested multi-type tree (QTMTT), which allows more flexible block partitioning compared to its predecessors, like High Efficiency Video Coding (HEVC). Meanwhile, the partition search (PS) process, which is to find out the best partitioning structure for optimizing the rate-distortion cost, becomes far more complicated for VVC than for HEVC. Also, the PS process in VVC reference software (VTM) is not friendly to hardware implementation. We propose a partition map prediction method for fast block partitioning in VVC intra-frame encoding. The proposed method may replace PS totally or be combined with PS partially, thereby achieving adjustable acceleration of the VTM intra-frame encoding. Different from the previous methods for fast block partitioning, we propose to represent a QTMTT-based block partitioning structure by a partition map, which consists of a quadtree (QT) depth map, several multi-type tree (MTT) depth maps, and several MTT direction maps. We then propose to predict the optimal partition map from the pixels through a convolutional neural network (CNN). We propose a CNN structure, known as Down-Up-CNN, for the partition map prediction, where the CNN structure emulates the recursive nature of the PS process. Moreover, we design a post-processing algorithm to adjust the network output partition map, so as to obtain a standard-compliant block partitioning structure. The post-processing algorithm may produce a partial partition tree as well; then based on the partial partition tree, the PS process is performed to obtain the full tree. Experimental results show that the proposed method achieves 1.61× to 8.64× encoding acceleration for the VTM-10.0 intra-frame encoder, with the ratio depending on how much PS is performed. Especially, when achieving 3.89× encoding acceleration, the compression efficiency loss is 2.77% in BD-rate, which is a better tradeoff than the previous methods.

An H.264 High-Profile Intra-Prediction with Adaptive Selection Between the Parallel and Pipelined Executions of Prediction Modes

A high-profile H.264 intra-frame encoder is suitable for low-cost and low-power applications and capable of providing enhanced compression efficiency. The high-profile is targeting the high-resolution videos. Thus, the encoding speed should be faster than or comparable to the baseline-profile. In previous work related to a hardware-based baseline-profile intra-frame encoder, a speed-up is achieved by the early termination of the intra modes and by an increase in the rate of hardware utilization only under one of the serialized and parallel schedules. This paper proposes a novel pipeline schedule for a hardware-based high-profile intra-prediction scheme in which the $8 \times 8$ prediction is performed in Stage 1 and $4 \times 4$ , $16 \times 16$ and chroma predictions are executed during Stage 2. The processing time of Stage 2 is efficiently accelerated based on the result of the $8 \times 8$ prediction in Stage 1. According to the distribution of each mode, the schedule is adaptively selected between parallel and pipeline schedules. To increase the hardware utilization of the $8 \times 8$ prediction, the order of prediction modes and the inverse vertical transform is adaptively adjusted. In addition, early termination of the prediction modes is employed for a fast $8 \times 8$ prediction. The proposed $8 \times 8$ intra-prediction is implemented and verified as an entire intra-frame encoder. Experimental results show that the average number of cycles necessary to process one MB for videos with resolutions of $1920 \times 1080$ and $3840 \times 2160$ are only 269 and 253 cycles, respectively. Compared to JM13.2, the bitrate is increased by 1.13% on average with a small PSNR degradation of 0.06 dB. The difference in the rate-distortion performance between the proposed high-profile intra-prediction scheme and JM 13.2 is not significant, whereas the achieved speed-up due to the proposed schemes is considerable compared to the conventional hardware-based intra-prediction encoders.

A new approach to DVC combined with ROI extraction and residual video coding for multimedia sensor networks

The Wyner-Ziv distributed video coding scheme is characterized for its intraframe encoder and interframe decoder which can also approach the efficiency of an interframe encoder-decoder system. In Wyner-Ziv residual coding of video, the residual of a frame with respect to a reference frame is Wyner-Ziv encoded, which can reduces the input entropy and leads to a higher coding efficiency than directly encoding the original frame. In this paper, we propose a new approach of residual coding combined with Region Of Interest (ROI) extraction. Experimental results show that, the proposed scheme achieves better rate-distortion performance compared to conventional Wyner-Ziv coding scheme.

A Low-Power High-Performance H.264/AVC Intra-Frame Encoder for 1080pHD Video

H.264/AVC intra-frame encoding contains several computation-intensive coding tools that form a long data dependency loop that is difficult to speed up. In this paper, we present a low-power and high-performance H.264/AVC intra-frame encoder. We propose several novel approaches to alleviate the performance bottleneck caused by the long data dependency loop among 4 × 4 luma blocks, integrate an efficient CABAC entropy encoder, and apply a clock-gating technique to reduce power consumption. Synthesized into a TSMC 0.13 μm CMOS cell library, our design requires 265.3 K gates at 114 MHz and consumes 23.56 mW to encode 1080pHD (1920 × 1088) video sequences at 30 frames per second (fps). It also delivers the same video quality as the H.264/AVC reference software. Compared with all state-of-the-art designs, our design has a lower working frequency and achieves both better bit-rate saving and lower power consumption.

WEBP, a new web oriented image format (english version)

Recently, Google has proposed a new image encoding format called WEBP that it is specifically targeted to encode images that are posted on websites. This new WEBP format is a lossy compression one that is based on the intraframe encoder of the VP8 video compressor.

Distributed Video Coding

Distributed coding is a new paradigm for video compression, based on Slepian and Wolf's and Wyner and Ziv's information-theoretic results from the 1970s. This paper reviews the recent development of practical distributed video coding schemes. Wyner-Ziv coding, i.e., lossy compression with receiver side information, enables low-complexity video encoding where the bulk of the computation is shifted to the decoder. Since the interframe dependence of the video sequence is exploited only at the decoder, an intraframe encoder can be combined with an interframe decoder. The rate-distortion performance is superior to conventional intraframe coding, but there is still a gap relative to conventional motion-compensated interframe coding. Wyner-Ziv coding is naturally robust against transmission errors and can be used for joint source-channel coding. A Wyner-Ziv MPEG encoder that protects the video waveform rather than the compressed bit stream achieves graceful degradation under deteriorating channel conditions without a layered signal representation.