High Efficiency Video Coding Intra Coding Research Articles

Transformer-Based Data-Driven Video Coding Acceleration for Industrial Applications

With the exploding development of edge intelligence and smart industry, deep learning-based intelligent industrial solutions are promptly applied in the manufacturing process. Many intelligent industrial solutions such as automatic manufacturing inspection are computer vision based and require fast and efficient video encoding techniques so that video streams can be processed as quickly as possible either at the edge cluster or over the cloud. As one of the most popular video coding standards, the high efficiency video coding (HEVC) standard has been applied to various industrial scenes. However, HEVC brings not only a higher compression rate but also a significant increase in encoding complexity, which hinders its practical application in industrial scenarios. Fortunately, a large amount of video coding data makes it possible to accelerate the encoding process in the industry. To speed up the video coding process in some industrial scenes, this paper proposes a data-driven fast approach for coding tree unit (CTU) partitioning in HEVC intracoding. First, we propose a method to represent the partition result of a CTU as a column vector of length 21. Then, we employ lots of encoding data produced in normal industry scenes to train transformer models used to predict the partitioning vector of the CTU. Finally, the final partitioning structure of the CTU is generated from the partitioning vector after a postprocessing operation and used by an industrial encoder. Compared with the original HEVC encoder used by some industrial applications, experiment results show that our approach achieves 58.77% encoding time reduction with 3.9% bit rate loss, which indicates that our data-driven approach for video coding has great capacity working in industrial applications.

Intra Prediction-Based Hologram Phase Component Coding Using Modified Phase Unwrapping

In this paper, we propose a method for compressing the phase component of a full-complex hologram. The JPEG (Joint Photographic Experts Group) Pleno is undergoing standardization for compressing full-complex holograms. If the full-complex hologram is compressed in the form of amplitude and phase components, the three-dimensional information of the hologram may be better preserved. Therefore, in order to solve the disadvantages of the method of independently compressing real and imaginary parts, we propose a method for directly compressing phase information. We select the HEVC (High Efficiency Video Coding), which has the best performance in compressing holograms from previous studies, as the anchor codec, and propose an algorithm for converting the phase information into the form suitable for the HEVC. Since the phase component is very random, we propose a modified phase unwrapping technique to improve this. In addition, in order to make good use of the property of HEVC Intra coding, the phase unwrapping considering Intra prediction is applied, and the most suitable HEVC Intra coding condition is searched. Compared with the result of compressing the phase using the HEVC, the hologram was improved by 2 dB or more and the reconstruction result was improved by more than 4 dB at a compression ratio of 80:1. If the compression ratio is increased, the proposed method has better results.

Learned Fast HEVC Intra Coding.

In High Efficiency Video Coding (HEVC), excellent rate-distortion (RD) performance is achieved in part by having a flexible quadtree coding unit (CU) partition and a large number of intra-prediction modes. Such an excellent RD performance is achieved at the expense of much higher computational complexity. In this paper, we propose a learned fast HEVC intra coding (LFHI) framework taking into account the comprehensive factors of fast intra coding to reach an improved configurable tradeoff between coding performance and computational complexity. First, we design a low-complex shallow asymmetric-kernel CNN (AK-CNN) to efficiently extract the local directional texture features of each block for both fast CU partition and fast intra-mode decision. Second, we introduce the concept of the minimum number of RDO candidates (MNRC) into fast mode decision, which utilizes AK-CNN to predict the minimum number of best candidates for RDO calculation to further reduce the computation of intra-mode selection. Third, an evolution optimized threshold decision (EOTD) scheme is designed to achieve configurable complexity-efficiency tradeoffs. Finally, we propose an interpolation-based prediction scheme that allows for our framework to be generalized to all quantization parameters (QPs) without the need for training the network on each QP. The experimental results demonstrate that the LFHI framework has a high degree of parallelism and achieves a much better complexity-efficiency tradeoff, achieving up to 75.2% intra-mode encoding complexity reduction with negligible rate-distortion performance degradation, superior to the existing fast intra-coding schemes.

Effective Data Driven Coding Unit Size Decision Approaches for HEVC INTRA Coding

High Efficiency Video Coding (HEVC) INTRA coding improves compression efficiency by adopting advanced coding technologies, such as multi-level quad-tree block partitioning and up to 35-mode INTRA prediction. However, it significantly increases the coding complexity, memory access, and power consumption, which goes against its widely applications, especially for ultra-high definition and/or mobile video applications. To tackle this problem, we propose effective data driven coding unit (CU) size decision approaches for HEVC INTRA coding, which consists of two stages of support vector machine-based fast INTRA CU size decision schemes at four CU decision layers. At the first stage classification, a three output classifier with offline learning is developed to early terminate the CU size decision or early skip checking the current CU depth. As for the samples that neither early skipped nor early terminated, the second stage of binary classification, which learns online from previous coded frames, is proposed to further refine the CU size decision. Representative features for the CU size decision are explored at different decision layers and stages of classifications. Finally, the optimal parameters derived from the training data are achieved to reasonably allocate complexity among different CU layers at given total rate-distortion degradation constraint. Extensive experiments show that the proposed overall algorithm can achieve 27.95%–80.53% and 52.48% on average complexity reduction for the CU size decision as compared with the original HM16.7 model. Meanwhile, the average Bjonteggard delta peak-signal-to-noise ratio degradation is only −0.08 dB, which is negligible. The overall performance of the proposed algorithm outperforms the state-of-the-art benchmark schemes.

Fast HEVC Intra Coding by Predicting the Rate-distortion Cost for a Low-complexity Encoder

High Efficiency Video Coding (HEVC) intra coding achieves significant improvements in coding efficiency compared with H.264/AVC intra coding by adopting 35 spatial intra prediction modes and a quadtree-based block partitioning structure. However, the encoding complexity is extremely high from performing rate-distortion optimization (RDO) on each mode and depth level. This paper proposes fast intra mode decision algorithms for real-time HEVC encoding. For a fast intra prediction mode decision, we estimate the lower bound on the rate-distortion (RD) cost of a prediction unit with non–most-probable modes (non-MPMs). If the RD cost of the MPM is lower than the lower bound, the MPM is selected as the best mode. For a fast intra coding unit (CU) size decision, we predict the RD cost of four split sub-CUs from the RD of the current CU using the relationship between the RDs of the current CU and its split sub-CUs. In the HEVC Test Model software, the proposed algorithm saves 48.56% of the encoding time, on average, while the RD loss is only 0.59%. In x265, the proposed algorithm improves the encoding speed to 71.95 fps, on average, from 49.46 fps. It achieves 1.8 times faster speed with a bit increase of only 0.44%, compared to x265.

Low Complexity HEVC INTRA Coding for High-Quality Mobile Video Communication

INTRA video coding is essential for high quality mobile video communication and industrial video applications since it enhances video quality, prevents error propagation, and facilitates random access. The latest high-efficiency video coding (HEVC) standard has adopted flexible quad-tree-based block structure and complex angular INTRA prediction to improve the coding efficiency. However, these technologies increase the coding complexity significantly, which consumes large hardware resources, computing time and power cost, and is an obstacle for real-time video applications. To reduce the coding complexity and save power cost, we propose a fast INTRA coding unit (CU) depth decision method based on statistical modeling and correlation analyses. First, we analyze the spatial CU depth correlation with different textures and present effective strategies to predict the most probable depth range based on the spatial correlation among CUs. Since the spatial correlation may fail for image boundary and transitional areas between textural and smooth areas, we then present a statistical model-based CU decision approach in which adaptive early termination thresholds are determined and updated based on the rate-distortion (RD) cost distribution, video content, and quantization parameters (QPs). Experimental results show that the proposed method can reduce the complexity by about 56.76% and 55.61% on average for various sequences and configurations; meanwhile, the RD degradation is negligible.

Just Noticeable Difference Based Fast Coding Unit Partition in HEVC Intra Coding

As an ongoing video compression standard, High Efficiency Video Coding (HEVC) has achieved better rate distortion performance than H.264, but it also leads to enormous encoding complexity. In this paper, we propose a novel fast coding unit partition algorithm in the intra prediction of HEVC. Firstly, instead of the time-consuming rate distortion optimization for coding mode decision, just-noticeable-difference (JND) values can be exploited to partition the coding unit according to human visual system characteristics. Furthermore, coding bits in HEVC can also be considered as assisted information to refine the partition results. Compared with HEVC test model HM10.1, the experimental results show that the fast intra mode decision algorithm provides over 28% encoding time saving on average with comparable rate distortion performance.

Fast CU Splitting and Pruning for Suboptimal CU Partitioning in HEVC Intra Coding

High Efficiency Video Coding (HEVC), a new video coding standard currently being established, adopts a quadtree-based Coding Unit (CU) block partitioning structure that is flexible in adapting various texture characteristics of images. However, this causes a dramatic increase in computational complexity compared to previous video coding standards due to the necessity of finding the best CU partitions. In this paper, a fast CU splitting and pruning method is presented for HEVC intra coding, which allows for significant reduction in computational complexity with small degradations in rate-distortion (RD) performance. The proposed fast splitting and pruning method is performed in two complementary steps: 1) early CU split decision and 2) early CU pruning decision. For CU blocks, the early CU splitting and pruning tests are performed at each CU depth level according to a Bayes decision rule method based on low-complexity RD costs and full RD costs, respectively. The statistical parameters for the early CU split and pruning tests are periodically updated on the fly for each CU depth level to cope with varying signal characteristics. Experimental results show that our proposed fast CU splitting and pruning method reduces the computational complexity of the current HM to about 50% in encoding time with only 0.6% increases in BD rate.