Loss Of Coding Efficiency Research Articles

Efficient VVC Encoding Using Hierarchical Parallelization: A Comprehensive Analysis

This paper presents and analyzes different parallelization strategies in VVenC, an open and optimized software encoder implementation of the Versatile Video Coding (VVC) standard. VVC has been developed to address the increasing demand for higher compression of digital video data, and it reduces the bitrate by around 50% for the same perceived quality compared to its predecessor, the High-Efficiency Video Coding (HEVC) standard. However, this increase in compression efficiency comes with an increase in computational complexity, particularly on the encoder side. VVenC integrates algorithmic optimizations for each coding tool in VVC and defines a set of five presets from faster to slower that provide Pareto-optimal tradeoffs between runtime and efficiency. Multithreading is employed to further reduce runtime while preserving most of the compression efficiency of each preset. With a hierarchical combination of pre-processing, picture-level and in-picture parallelization, VVenC achieves a 4× speedup for four threads. Further speedup using a higher number of threads depends on the video resolution and used encoder preset. For 16 threads, it ranges from 6 to 9 for high definition to 10-12 for ultra-high-definition video. Compared to previous work, the usage of temporal prediction for the adaptive loop filter reduces the associated coding efficiency loss from 0.4% to almost zero. A better scaling for higher numbers of threads can be achieved at the cost of a higher coding efficiency loss. In the presented framework, increased speedup by smaller block Coding Tree Unit (CTU) sizes, a combination of wavefront parallel processing and various tiles picture partitioning configurations is examined. Furthermore, results on a 20-core ARM-based Apple M1 computer indicate a better scaling for multithreading compared to × 86-based architectures. The analysis is complemented by profiling, which exhibits the overhead by idle threads and identifies the mode estimation as the main bottleneck of the presented framework.

Hybrid Strategies for Efficient Intra Prediction in Spatial SHVC

With multi-layer encoding and Inter-layer prediction, Spatial Scalable High Efficiency Video Coding (SSHVC) has extremely high coding complexity. It is very crucial to speed up its coding to promote widespread and cost-effective SSHVC applications. Specifically, we first reveal that the average RD cost of Inter-layer Reference (ILR) mode is different from that of Intra mode, but they both follow the Gaussian distribution. Based on this discovery, we apply the classic Gaussian Mixture Model and Expectation Maximization to determine whether ILR mode is the best mode thus skipping Intra mode. Second, when coding units (CUs) in enhancement layer use Intra mode, it indicates very simple texture is presented. We investigate their Directional Mode (DM) distribution, and divide all DMs into three classes, and then develop different methods with respect to classes to progressively predict the best DMs. Third, by jointly considering rate distortion costs, residual coefficients and neighboring CUs, we propose to employ the Conditional Random Fields model to early terminate depth selection. Experimental results demonstrate that the proposed algorithm can significantly improve coding speed with negligible coding efficiency losses.

QTMT‐LNN: A fast intra CU partition using lightweight neural network for 360‐degree video coding on VVC

In the newest generation of video coding standard, Versatile Video Coding (VVC), a new technique called Quad Tree with nested Multi-type Tree (QTMT) structure is introduced. QTMT significantly improves the coding efficiency, but the improvement in compression performance comes at the cost of drastically increased complexity. This paper proposes a fast intra partition algorithm using Lightweight Neural Network (LNN) to skip QTMT partition steps which are unlikely to be chosen as the best split modes. Specifically, five LNNs (for five QTMT split modes) are trained, using features that consider the characteristic of 360-degree videos. The experimental results demonstrate that the proposed QTMT-LNN can reduce the encoding time from 52.28% to 72.17% on average, with coding efficiency losses ranging from 0.71% to 1.78%. Compared to other fast algorithms for intra coding unit (CU) partition, the method outperforms the related works in terms of Bjontegaard delta bit-rate (BDBR) and encoding time reduction.

Configurable Fast Block Partitioning for VVC Intra Coding Using Light Gradient Boosting Machine

This article presents a configurable fast block partitioning decision for Versatile Video Coding (VVC) intra-frame prediction using Light Gradient Boosting Machine (LGBM). VVC further improves the coding efficiency by introducing a Quadtree with nested Multi-Type Tree (QTMT), enabling five split types allowing square and rectangular Coding Unit (CU) sizes. However, this improvement in the coding efficiency comes at the cost of a high computational burden since several combinations of block sizes and prediction modes are evaluated through the costly Rate-Distortion Optimization (RDO) process. In this article, we propose a partitioning decision using LGBM classifiers to avoid the exhaustive RDO process and skip the evaluation of split types that are unlikely to be chosen as the best one. For this purpose, five classifiers (one for each split type) were offline trained with an efficient training process and using effective features of texture, coding, and context information. The proposed solution is highly configurable and can provide several operation points with different tradeoffs between timesaving and coding efficiency, according to the application requirements. Considering five operation points, the configurable solution can reduce the encoding time from 35.22% to 61.34%, with coding efficiency losses from 0.46% to 2.43%. Compared to the state-of-the-art, our solution is able to outperform the related works in terms of combined rate-distortion and timesaving.

FastInter360: A Fast Inter Mode Decision for HEVC 360 Video Coding

This paper presents FastInter360, a fast inter mode decision algorithm for accelerating the encoding of ERP 360 videos. The development of FastInter360 involves an in-depth and comprehensive set of evaluations performed to understand the differences in the encoder’s behavior when encoding 360 and conventional videos. These evaluations showed that due to the texture distortions resulting from projection, the encoder presents a specific behavior when encoding 360 videos, making it more likely to use a recurrent set of encoding modes when processing 360 videos. Besides, the coding efficiency is less sensible to approximations in some encoding steps depending on the frame region. FastInter360 is then proposed to reduce the encoding complexity by exploiting these differences. FastInter360 comprises three algorithms that accelerate the encoding by performing early decision by SKIP mode, reducing integer motion estimation search range, and adjusting fractional motion estimation precision. Furthermore, each of these algorithms behaves according to distortion intensity, performing greater complexity reduction in more distorted regions. When employed altogether, these algorithms compose FastInter360, which is able to achieve an average complexity reduction of 22.84% with a coding efficiency loss of 0.652% BD-BR, on average, making FastInter360 competitive with literature works.

Complexity-scalable HEVC-to-AV1 video transcoding based on partition inheritance

After the AOMedia Video 1 (AV1) bitstream specification was launched by the Alliance for Open Media, the need for converting legacy content encoded with the state-of-the-art High Efficiency Video Coding (HEVC) standard to the new format has arisen in several scenarios. However, transcoding is a complex task composed of a decoding and an encoding process in sequence, which requires long processing times and high energy consumption. This paper presents a complexity-scalable HEVC-to-AV1 transcoding scheme, which comprises 25 configuration modes chosen based on a Pareto Optimization strategy and built upon the correlation between block size decisions in HEVC and AV1. The configurations allow the AV1 encoder to inherit partitioning information from the HEVC bitstream to constrain the AV1 reencoding process, thus speeding up the transcoding operation. Experimental results showed that the proposed transcoding complexity reduction ranges from 0.41 up to 35.06%, on average, with coding efficiency losses that vary between 0.0536 and 5.3801%.

Fast Grid-Based Refining Segmentation Method in Video-Based Point Cloud Compression

The video-based point cloud compression (V-PCC, ISO/IEC 23090-5) is the state-of-the-art international standard for compressing dynamic point clouds developed by the moving picture experts group (MPEG). It has been achieved good rate-distortion (RD) performance by employing the 2D-based dynamic point cloud compression. As a brief look, V-PCC first converts the 3D input point cloud into a set of 2D patches followed by a packing process. The packing process then maps the patches into a 2D grid. Such a way allows compressing the patches utilizing the existing video coding standards. Besides the RD performance, complexity is another vital factor to consider in performance evaluations. In the V-PCC encoder, the self-time accounts for on average 15.9% and a maximum of 48.2% of the total-time, which can be a hindrance to realizing real-time V-PCC applications. One of the most computationally intensive modules of V-PCC is the grid-based refining segmentation (G-RS). Thus this paper proposes a fast G-RS method that can adaptively select voxels that need the refining segmentation. More concretely, the proposed method classifies the voxels based on the projection plane indices of 3D points and only applies the refining process to the selected voxels. Experimental results demonstrate that the proposed method reduces the complexity of the refining steps in G-RS, on average, by 60.7% and 62.5% without coding efficiency loss compared to the test model for category 2 (TMC2) version 12.0 reference software under the random access (RA) and all-intra (AI) configurations, respectively.

Memory-aware Workload Balancing Technique based on Decision Trees for Parallel HEVC Video Coding

Video coding applications demand high computational effort to achieve high compression rates at a low perceptual quality expense. In order to reach acceptable encoding time for such applications, modern video coding standards have been em-ploying parallelism approaches to exploit multiprocessing plat-forms, such as the tiling tool from HEVC standard. When employing Tiles, each frame is divided into rectangular-shaped regions which can be encoded independently. However, alt-hough it is possible to distribute the data equally among the processing units when using Tiles, balancing the workload among processing units poses great challenges. Therefore, this paper proposes a workload balancing technique aiming to speed up the HEVC parallel encoding using Tiles. Different from other literature works, the proposed solution uses a novel approach employing static uniform tiling to avoid memory management difficulties that may emerge when dynamic tiling solutions are employed. The proposed technique relies on workload distribution history of previous frames to predict the workload distribution of the current frame. Then, the pro-posed technique balances the workload among Tiles by em-ploying a workload reduction scheme based on decision trees in the coding process. Experimental tests show that the pro-posed solution outperforms the standard uniform tiling and it is competitive with related works in terms of speedup. Moreo-ver, the solution optimizes resources usage in multiprocessing platforms, presents a negligible coding efficiency loss and avoids increasing memory bandwidth usage by 9.8%, on aver-age, when compared to dynamic tiling solutions, which can impact significantly the performance in memory-constrained platforms.

6WR: A Hardware Friendly 3D-HEVC DMM-1 Algorithm and its Energy-Aware and High-Throughput Design

This brief presents the Six Wedgelets and six Refinements (6WR), a hardware friendly algorithm targeting the Depth Modeling Mode 1 (DMM-1) encoding tool of the 3D-High Efficiency Video Coding (3D-HEVC) standard. This brief also presents the high-throughput and energy-aware hardware design for the 6WR. The 6WR algorithm reduces 98.5% of the evaluated wedge lets by exploring the edges gradients, with average coding efficiency losses between 1.2% and 2.8%. The hardware design implements the Bresenham algorithm to avoid the use of memory. The synthesis results show that the 6WR architecture can process up to nine views in 3D full HD 1080p videos at 30 frames per second, with a power dissipation of 263.7 mW. When compared with related works, the 6WR architecture reached the highest throughput and the best results of coding efficiency and energy efficiency when the same target throughput is considered.

Tile Adaptation for Workload Balancing of 3D-HEVC Encoder in Homogeneous Multicore Systems

This paper proposes a tile adaptation for workload balancing to speed up the 3D-High Efficiency Video Coding (3D-HEVC) encoder. Experiments were done to evaluate the usage of tiles when encoding texture and depth maps considering a static uniform partitioning. Based on these evaluations, we observed that the encoding workload of neighboring frames in the same view tends to be similar. Moreover, the encoding workload of dependent views can be predicted using the information of previously encoded frames of the same and base views. Therefore, the proposed solution adaptively defines the boundaries of the tiles based on inter-frame and inter-view correlations to balance the workloads of the tiles, resulting in a higher speedup when executing in homogeneous multicore systems. The proposed method was implemented in 3D-HEVC Test Model and evaluated according to the Common Test Conditions. The results demonstrated that the proposed solution speeds up texture and depth map coding from 9.94% to 18.21%, when compared to the uniform-sized tiles, without coding efficiency loss. To the best of the authors' knowledge, this is the first work proposing a complete solution for speeding up the 3D-HEVC video encoder by balancing the tiles workload dynamically.

3D-HEVC Bipartition Modes Encoder and Decoder Design Targeting High-Resolution Videos

This paper presents hardware designs for the encoder and decoder of the 3D-High Efficiency Video Coding (3D-HEVC) bipartition modes targeting real-time processing of high-resolution videos. These hardware designs include the depth modeling modes 1 (DMM-1) and 4 (DMM-4). The encoder was designed using a simplification in the DMM-1 algorithm with the advantage of better using the hardware resources and without coding efficiency loss. The encoder architecture was designed in a scalable structure supporting different block sizes and reaching different throughputs according to the application requirements. The decoder was designed sharing resources between DMM-1 and DMM-4 execution and supporting all available block sizes. Both architectures were synthesized for 65- and 28-nm technologies, being capable of achieving real-time processing for ${1920}\times {1080}$ videos at 30 frames/s.

Speedup-Oriented History-Based Tiling Algorithm for the HEVC Standard Targeting an Efficient Parallelism Exploration

This paper proposes a history-based tiling algo-rithm intending to increase the speedup when using Tiles. The algorithm is composed of two independent steps that use work-load history information to define the vertical and horizontal boundaries of the Tiles. The workload distribution of previous frames is used as reference to perform the tiling of the current frame, exploiting the temporal similarity between neighboring frames, focusing on improving the speedup. Experimental re-sults show that the proposed algorithm outperforms the speedup when compared to uniform tiling by up to 38%, whereas posing no significant complexity increase nor coding efficiency losses. When compared to related works the proposed solution also presents better results.

A fast CU partition and mode decision algorithm for HEVC intra coding

The latest video coding standard H.265/HEVC is a milestone because it can achieve up to 50 % coding performance improvement compared to the widely used H.264/AVC, owing to the newly introduced coding tools. However, these new techniques also introduce dramatic computational complexity increase. In order to release the computational burden while maintaining the coding efficiency, a two-part fast intra coding algorithm is proposed in this paper. First a CTU depth range prediction and two Bayesian decision rules are applied to accelerate the CU partition process. The depth search range of the current CTU is restricted by the neighboring CTUs on account of the spatial correlation. And one early split and one early termination strategies is presented based on the minimum Bayesian decision rule. Next, a three-step fast intra mode decision algorithm is performed to speed up the intra coding, both the depth information and the orientation information of the current CU are used to skip some unnecessary intra mode check, and the order of Most Probable Modes (MPM) is adjusted to further remove redundant calculation. As demonstrated in the experimental results, our proposed algorithm can reduce 52.9% encoding time on average with the cost of an negligible coding efficiency loss under the All-Intra configuration.

Fast Algorithm and VLSI Architecture of Rate Distortion Optimization in H.265/HEVC

In H.265/high efficiency video coding (HEVC) encoding, rate distortion optimization (RDO) is an important cost function for mode decision and coding structure decision. Despite being near-optimum in terms of coding efficiency, RDO suffers from a high complexity. To address this problem, this paper presents a fast RDO algorithm and its very large scale implementation (VLSI) for both intra- and inter-frame coding. The proposed algorithm employs a quantization-free framework that significantly reduces the complexity for rate and distortion optimization. Meanwhile, it maintains a low degradation of coding efficiency by taking the syntax element organization and probability model of HEVC into consideration. The algorithm is also designed with hardware architecture in mind to support an efficient VLSI implementation. When implemented in the HEVC test model, the proposed algorithm achieves 62% RDO time reduction with 1.85% coding efficiency loss for the “all-intra” configuration. The hardware implementation achieves 1.6 × higher normalized throughput relative to previous works, and it can support a throughput of 8k@30fps (for four fine-processed modes per prediction unit) with 256 k logic gates when working at 200 MHz.

Hardware implementation-oriented fast intra-coding based on downsampling information for HEVC

This paper proposes a downsampling information-based intra-coding scheme which consists of two parts, preprocessing stage and fast intra-coding stage. Three downsampling information-based fast decision algorithms are proposed in fast intra-coding stage. Moreover, a parallelized architecture for the proposed fast intra-coding scheme is proposed. The preprocessed downsampling stage can be executed with intra-coding stage in parallel. The proposed architecture fully makes use of this feature to improve the throughput and fragment data dependency. Experimental results demonstrate that the proposed algorithms achieves on average 60.4% reduction in encoding time with negligible coding efficiency loss, compared with original HEVC.

CU encoding depth prediction, early CU splitting termination and fast mode decision for fast HEVC intra-coding

High Efficiency Video Coding (HEVC) is a new video coding standard achieving about a 50% bit rate reduction compared to the popular H.264/AVC High Profile with the same subjective reproduced video quality. Better coding efficiency is attained, however, at the cost of significantly increased encoding complexity. Therefore, fast encoding algorithms with little loss in coding efficiency is necessary for HEVC to be successfully adopted for real applications. In this paper we propose a fast encoding technique applicable to HEVC all intra encoding. The proposed fast encoding technique consists of coding unit (CU) search depth prediction, early CU splitting termination, and fast mode decision. In CU search depth prediction, the depth of encoded CU in the current coding tree unit (CTU) is limited to predicted range, which is mostly narrower than the full depth range. Early CU splitting skips mode search of its sub-CUs when rate distortion (RD) cost of current CU is below the estimated RD cost at the current CU depth. RD cost and encoded CU depth distribution of the collocated CTU of the previous frame are utilized both to predict the encoding CU depth search range and to estimate the RD cost for CU splitting termination. Fast mode decision reduces the number of candidate modes for full rate distortion optimized quantization on the basis of the low complexity costs computed in the preceding rough mode decision step. When all these three methods are applied, proposed fast HEVC intra encoding technique reduces the encoding time of the reference encoder by 57% on the average, with only 0.6% of coding efficiency loss in terms of Bjontegaard delta (BD) rate increase under the HEVC common test conditions.

HEVC Encoding Optimization Using Multicore CPUs and GPUs

Although the High Efficiency Video Coding (HEVC) standard significantly improves the coding efficiency of video compression, it is unacceptable even in offline applications to spend several hours compressing 10 s of high-definition video. In this paper, we propose using a multicore central processing unit (CPU) and an off-the-shelf graphics processing unit (GPU) with 3072 streaming processors (SPs) for HEVC fast encoding, so that the speed optimization does not result in loss of coding efficiency. There are two key technical contributions in this paper. First, we propose an algorithm that is both parallel and fast for the GPU, which can utilize 3072 SPs in parallel to estimate the motion vector (MV) of every prediction unit (PU) in every combination of the coding unit (CU) and PU partitions. Furthermore, the proposed GPU algorithm can avoid coding efficiency loss caused by the lack of a MV predictor (MVP). Second, we propose a fast algorithm for the CPU, which can fully utilize the results from the GPU to significantly reduce the number of possible CU and PU partitions without any coding efficiency loss. Our experimental results show that compared with the reference software, we can encode high-resolution video that consumes 1.9% of the CPU time and 1.0% of the GPU time, with only a 1.4% rate increase.

A fast mode decision algorithm applied to Coarse-Grain quality Scalable Video Coding

A fast mode decision algorithm is proposed for a Coarse-Grain Scalable (CGS) video encoder based on the encoding characteristics of quality Scalable Video Coding (SVC). First, candidate modes and coding orders are predicted, based on inter-layer and spatial correlations. Three early termination methods are then proposed based on CGS encoding structure. Finally, all candidate modes are checked sequentially, according to their predicted order with three early termination conditions, to improve the coding speed. Experimental results have demonstrated that the proposed algorithm could reduce the encoding time by an average of 84.39%, with negligible coding efficiency losses.

High Throughput Entropy Coding in the HEVC Standard

Context-based adaptive binary arithmetic coding (CABAC) is a single entropy coding mode of the newest video coding standard, High Efficiency Video Coding (HEVC). Despite the high coding efficiency of CABAC, its data dependencies prevent effective parallelization, which also means limited throughput. Accordingly, during the HEVC standardization activity of entropy coding, both coding efficiency and throughput were considered. This paper highlights the key techniques that were proposed and adopted up to HEVC Draft International Standard (DIS). In addition, a new method is proposed for high throughput entropy coding. This method reduces the worst-case complexity of entropy coding without significant coding efficiency loss. From the experimental results, in terms of throughput improvement, we confirm that the proposed method reduces the number of context-coded bins up to 42.1 and 15.9 % on average.

적응적 탐색 영역을 이용한 HEVC 고속 움직임 탐색 방법

본 논문에서는 HEVC 부호화의 연산 복잡도를 줄이기 위한 고속 움직임 탐색 방법을 제안한다. 고속움직임 탐색을 위한 종래의 방법은 현재 영상과 참조 영상 간의 거리에 따라 탐색 영역의 크기를 조절하는 반면, 제안 방법은 예측 단위 간의 움직임 벡터 차의 유사성을 이용하여 예측 단위에 따라 움직임 벡터 탐색 영역을 적응적으로 조절한다. 실험을 통해 제안 방법은 random access 실험 조건에서 움직임 예측의 연산 시간을 약 10.7% 감소시키는 반면, 약 0.1%의 낮은 부호화 성능 하락을 가지는 것을 확인하였다. This paper proposes a fast motion estimation method which can reduce the computational complexity of HEVC encoding process. While the previous method determines its search range based on a distance between a current and a reference pictures to accelerate the time-consuming motion estimation, the proposed method adaptively sets the search range according to motion vector difference between prediction units. Experimental results show that the proposed method achieves about 10.7% of reduction in processing time of motion estimation under the random access configuration whereas its coding efficiency loss is less than 0.1%.