H.265/High Efficiency Video Coding Research Articles

Fast CU size decision and intra-prediction mode decision method for H.266/VVC

AbstractH.266/Versatile Video Coding (VVC) is the most recent video coding standard developed by the Joint Video Experts Team (JVET). The quad-tree with nested multi-type tree (QTMT) architecture that improves the compression performance of H.266/VVC is introduced. Moreover, H.266/VVC contains a greater number of intra-prediction modes than H.265/High Efficiency Video Coding (HEVC), totalling 67. However, these lead to extremely the coding computational complexity. To cope with the above issues, a fast intra-coding unit (CU) size decision method and a fast intra-prediction mode decision method are proposed in this paper. Specifically, the trained Support Vector Machine (SVM) classifier models are utilized for determining CU partition mode in a fast CU size decision scheme. Furthermore, the quantity of intra-prediction modes added to the RDO mode set decreases in a fast intra-prediction mode decision scheme based on the improved search step. Simulation results illustrate that the proposed overall algorithm can decrease 55.24% encoding runtime with negligible BDBR.

Quality Enhancement with Frame-wise DLCNN using High Efficiency Video Coding in 5G Networks

In the present situation, applications related to multimedia are discovered to be comfortable with the use of video. The number of end consumers who use video continues to rise every day. People are presently searching for videos with better quality among the ones that are currently available there. This results in the launch and dissemination of HD (high definition) videos. Ultra high-definition (UHD) videos are becoming more and more popular as a result of this advancement and need. However, as video communication keeps expanding, there is an upsurge in network traffic because of the limited bandwidth, especially among smart cities. Different advancement codecs have been suggested to deal with the data stream to overcome this hazardous circumstance. However, the fact that modern UHD videos have huge amounts of data makes the available codecs even more complicated. UHD videos can be processed with the latest improvement codec, H.265/High-efficiency video coding (HEVC). Nevertheless, it is impacted by increased power consumption and intricate calculations. Limitations in the codec's functionality confine its use to specific applications, preventing its application in wireless, mobile, or portable settings. Hence, this research concentrates on implementing frame-level quality enhancement through a deep learning network known as FQE-Net. The deep learning convolutional neural network (DLCNN) is specifically crafted to manage films with resolutions up to 16K. Its primary objectives include reducing complexity, minimizing artifacts, enhancing the efficiency of the HEVC codec, and compacting energy consumption. To achieve superior efficiency, it is imperative to replace the DWT transforms within the HEVC codec with a DLCNN model. Additionally, incorporating the Content Block Search Algorithm for Motion Estimation and Compensation, alongside filtering techniques like Sample Adaptive Filter and Deblocking Filter, becomes essential. The simulation results showed that the suggested FQE-Net performed better than the conventional techniques.

A Hierarchical Multiscenario H.265/HEVC Video Encryption Scheme

With the pervasive application of video streaming, the security of streaming media information continually faces new challenges. Most existing encryption methods employ uniform criteria for encrypting all scenarios, leading to unnecessary mutual inhibition of algorithm security and coding efficiency. Furthermore, several encryption algorithms are inadequate for high-resolution videos. A novel, independently hierarchical video cryptosystem for H.265/high efficiency video coding (HEVC) is proposed that develops a scene-adaptive encryption strategy tailored for multiscenario videos. Additionally, we fully consider the seeds of pseudo-random number generators, syntax components in compressed codes, and scenario indicators. We analyze the visibility of encrypted videos in different scenarios and the encryption performance of various syntax parameters based on the integration of encryption for three distinct scenario categories to further enhance encryption efficiency and security. The method’s versatility is demonstrated using a diverse array of videos with significant and insignificant inter-frame motion information across varying resolutions. The experimental results from the video datasets indicate that our scheme effectively balances security and coding efficiency. Furthermore, the scene-adaptive approach can be tailored flexibly according to subscriber needs.

Bandwidth-Aware High-Efficiency Video Coding Design Scheme on a Multiprocessor System on Chip

H.265/high-efficiency video coding (HEVC) provides a multitude of video data compression to minimize data storage and data transmission while preserving video coding quality and ameliorating coding bit rates. However, HEVC encoder chips are frequently integrated into mobile multiprocessor system-on-chip (MPSoC) systems that adopt intelligent thermal and power management techniques for heat- and power-dissipation reductions. Consequently, the accessible coding bandwidth (CB) for HEVC encoder chip use is not fixed, and the compressed video data for data transmission within MPSoCs are restricted to time-altering wireless transmission bandwidths (TBs). Therefore, the proposed bandwidth-aware H.265/HEVC controller design solves the video coding problems of limited CB and TB by jointly using a machine-learning method and convex optimization. The ancillary experimental and implementation results demonstrate that the proposed CB-TB rate-coding distortion (CB-TB-R-D) algorithm modeling and the very large-scale integration (VLSI) hardware architecture are applicable for CB- and TB-constrained HEVC encoder design within MPSoC.

Visual Perception Based Intra Coding Algorithm for H.266/VVC

The latest international video coding standard, H.266/Versatile Video Coding (VVC), supports high-definition videos, with resolutions from 4 K to 8 K or even larger. It offers a higher compression ratio than its predecessor, H.265/High Efficiency Video Coding (HEVC). In addition to the quadtree partition structure of H.265/HEVC, the nested multi-type tree (MTT) structure of H.266/VVC provides more diverse splits through binary and ternary trees. It also includes many new coding tools, which tremendously increases the encoding complexity. This paper proposes a fast intra coding algorithm for H.266/VVC based on visual perception analysis. The algorithm applies the factor of average background luminance for just-noticeable-distortion to identify the visually distinguishable (VD) pixels within a coding unit (CU). We propose calculating the variances of the numbers of VD pixels in various MTT splits of a CU. Intra sub-partitions and matrix weighted intra prediction are turned off conditionally based on the variance of the four variances for MTT splits and a thresholding criterion. The fast horizontal/vertical splitting decisions for binary and ternary trees are proposed by utilizing random forest classifiers of machine learning techniques, which use the information of VD pixels and the quantization parameter. Experimental results show that the proposed algorithm achieves around 47.26% encoding time reduction with a Bjøntegaard Delta Bitrate (BDBR) of 1.535% on average under the All Intra configuration. Overall, this algorithm can significantly speed up H.266/VVC intra coding and outperform previous studies.

Low-complexity intra coding scheme based on Bayesian and L-BFGS for VVC

H.266/Versatile Video Coding (VVC) is the latest generation of video coding standards. Compared with H.265/High Efficiency Video Coding (HEVC), the number of intra modes expanded from 35 to 67, and the Quad Tree with nested Multi type Tree (QTMT) split structure is adopted. These encoding tools endow VVC with better compression performance at the cost of significantly increased computational complexity. To reduce computational complexity, a Low-complexity intra coding scheme is proposed in this paper, which includes two parts: fast Coding Unit (CU) split decision and fast intra mode decision. Firstly, a Bayesian-based fast CU split decision method is proposed for skipping vertical split and terminating split. This method improves the accuracy of probability estimation by combining parameter eigenvectors and non-parametric eigenvectors of multiple features. Secondly, a fast intra mode decision method based on Limit-Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) is proposed. This method reduces the number of Rough Mode Decision (RMD) and Rate Distortion Optimization (RDO) by searching for angle mode in a certain direction. Experimental results showed that the proposed scheme reduces the encoding time by 59.52%, while the BDBR increases by 1.13%.

Fast CU Size Decision Method Based on Just Noticeable Distortion and Deep Learning

With the development of broadband networks and high-definition displays, people have higher expectations for the quality of video images, which also brings new requirements and challenges to video coding technology. Compared with H.265/High Efficiency Video Coding (HEVC), the latest video coding standard, Versatile Video Coding (VVC), can save 50%-bit rate while maintaining the same subjective quality, but it leads to extremely high encoding complexity. To decrease the complexity, a fast coding unit (CU) size decision method based on Just Noticeable Distortion (JND) and deep learning is proposed in this paper. Specifically, the hybrid JND threshold model is first designed to distinguish smooth, normal, or complex region. Then, if CU belongs to complex area, the Ultra-Spherical SVM (US-SVM) classifiers are trained for forecasting the best splitting mode. Experimental results illustrate that the proposed method can save about 52.35% coding runtime, which can realize a trade-off between the reduction of computational burden and coding efficiency compared with the latest methods.

Fast CU Partition Decision Based on Texture for H.266/VVC

With the development of multimedia equipment and the increasing demand for high-quality video applications, the traditional video coding standard, H.265/High Efficiency Video Coding (HEVC), can no longer effectively satisfy the requirements. To promote the development of high-quality video, a new generation video coding standard, H.266/Versatile Video Coding (H.266/VVC), is established, and it is the inheritance and development of H.265/HEVC. It not only retains many mature technologies and methods in HEVC but also adds some new coding tools, such as wide-angle prediction and Multitype Tree (MTT) partition structure. The MTT partition structure brings a more flexible partition method of Coding Unit (CU), but the accompanying increase in computational complexity is unacceptable. In order to ensure an effective balance between coding efficiency and coding quality, a fast CU partition algorithm based on texture is proposed in this paper. First, the texture complexity of the neighboring CU is used as a threshold for evaluating the complexity of the current CU, so as to skip the unpromising depth. Then, the gradient features are extracted to determine whether the Quad-Tree (QT) partition is executed. Finally, the improved Canny operator is used to extract edge features, and the partition mode in the horizontal or vertical direction is excluded. The algorithm was embedded in VTM7.0, and the video sequences with different resolutions were tested under general experimental configuration. Simulation experiment results show that the average time saving of this method reached 50.56% compared with the anchor algorithm. At the same time, the average BDBR is increased by 1.31%.

A Real-Time Low-Power Coding Bit-Rate Control Scheme for High-Efficiency Video Coding in a Multiprocessor System-on-Chip

A real-time high-performance transmission bandwidth-aware (TB-aware) coding bit-rate (CBR) controller design with low power consumption and low hardware complexity is presented in this article for H.265/high-efficiency video coding (HEVC) in a multiprocessor system-on-chip (MPSoC). Previous TB-aware motion estimation designs with CBR-control capability in video coding have focused on algorithm development with precise CBR models, which require a complicated algorithmic derivation according to the system on-demand CBR and are difficult to realize in very large scale integration (VLSI) due to their lack of consideration for hardware implementation and modeling. Consequently, we present a hardware-oriented CBR-control algorithm that uses simple CBR control functions instead of requiring root and exponential operations to realize the real-time low-power design objective for HEVC applications within a mobile MPSoC. Then, an adequate hardware architecture with low hardware complexity is exploited to accomplish a low-power and high-speed VLSI design of a CBR controller for our proposed algorithm. Using diverse video-sized sequences under on-demand system coding-bit-rate constraints, the experimental outcomes demonstrate that the introduced design is capable of low power consumption and high speeds and can utilize low-complexity hardware.

Hybrid Distortion-Based Rate-Distortion Optimization and Rate Control for H.265/HEVC

Mean squared error (MSE) is adopted as the distortion metric for the rate-distortion optimization (RDO) and rate control (RC) in most of the existing video coding standards, e.g., H.265/High-Efficiency Video Coding (HEVC) and H.264/Advanced Video Coding (AVC), because the pixel-wise MSE can represent the image fidelity efficiently. As videos are usually used for visual enjoyment, recently, researchers found that the MSE is not always consistent with perceptual quality, and proposed some perceptual quality-based video coding methods by exchanging the MSE with other perceptual quality metrics completely, like the structure similarity index (SSIM), just noticeable distortion (JND), etc. We believe that a sophisticated video encoder should guarantee both the subject quality and the data fidelity. By investigating the properties of the human visual system (HVS), hybrid distortion-based RDO and RC methods are proposed to improve subjective quality while avoiding fidelity degradation. Experimental results demonstrate that the proposed methods can improve subjective quality efficiently, while preserving the image fidelity of the reconstructed videos as much as possible.

Near-Instantaneously Adaptive Multi-Set Space-Time Shift Keying for UAV-Aided Video Surveillance

Unmanned Aerial Vehicles (UAV) are more suitable for surveillance systems than their traditional stationary counterparts. To meet the associated demanding video compression requirements, the H.265/High Efficiency Video Coding (HEVC) codec is invoked. Its scalable video coding extension is capable of supporting diverse video resolutions by encoding the video clips captured into multiple layers, namely the Base Layer (BL) and several Enhancement Layers (EL). In this treatise, we propose a multi-functional Multiple-Input Multiple-Output (MIMO) transceiver relying on Multi-Set Space-Time Shift Keying (MS-STSK) for UAV-aided near-instantaneously adaptive layered video streaming, which is capable of promptly varying the throughput as a function of both the estimated channel quality as well as of the UAV mobility, while maintaining near-flawless video quality. Additionally, we invoke EXtrinsic Information Transfer (EXIT) charts for characterizing our Unequal Error Protection (UEP) based video transceiver with the aid of near-capacity channel coding. Some of the perceptually less important ELs may have to be dropped during instances of low channel quality to retain the robustness of the transmitted video stream when a low-throughput MIMO configuration is activated. The simulation results show that our proposed UEP assisted adaptive system is capable of attaining the best Y-Peak SNR (PSNR) performance in comparison to both its non-adaptive and Equal Error Protection (EEP) counterparts.

An improved R-λ rate control model based on joint spatial-temporal domain information and HVS characteristics

With the popularization of smart terminals and multimedia technologies, the video coding standard — H.264/Advanced Video Coding (AVC) and H.265/High Efficiency Video Coding (HEVC) have been unable to meet the needs of various high-definition videos, so the next generation standard —H.266/ Versatile Video Coding (VVC) is under study. In the actual transmission of a video communication channel, rate control plays an important role. However, HEVC rate control based on R-λ model does not adequately take into account the characteristics of the human visual system (HVS). Also, the convergence speed of Least Mean Square (LMS) in HEVC is too slow. In this paper, an improved R-λ(Lambda) rate control model based on joint spatial-temporal domain information and HVS characteristics (IRLRC) is established. In this model, the joint spatial-temporal domain information based on gradient information is used to guide bit allocation for frame and CTU level, where the temporal coefficient is corrected adaptively. What’s more, the Broyden Fletcher Goldfarb Shanno (BFGS) algorithm is introduced, which speeds up the convergence of the proposed model. The experimental results have clearly shown that the proposed IRLRC can achieve better coding performance than HEVC, VVC and other models. In particular, the video sequence based on the proposed IRLRC can meet the needs of HVS and achieve higher optimization for subjective quality.

Fast CU partition decision for H.266/VVC based on the improved DAG-SVM classifier model

One of the biggest changes in H.266/Versatile Video Coding (VVC) is introduced quad-tree with nested multi-type tree (QTMT) coding tree architecture, where the multi-type tree (MTT) structure in H.266/VVC includes binary tree (BT) and ternary tree (TT). Compared with H.265/High Efficiency Video Coding (HEVC) which only is divided by quad-tree (QT), the QTMT architecture makes the coding unit (CU) partition procedure more complexity. In this paper, we design a fast CU partition decision algorithm based on the improved Directed Acyclic Graph Support Vector Machine (DAG-SVM) model to reduce the complexity of CU partition. The video sequences are first encoded on the H.266/VVC and Test Model 4.0 (VTM 4.0), and the characteristics of the video sequences are extracted for training through the improved F-score method, where the correlation between a feature and CU partition is high. Then, the offline training is used for the improved DAG-SVM model. Finally, the trained DAG-SVM model is embedded in VTM 4.0 to early forecast the optimal CU partition modes. Simulation results indicate that the proposed method increases the time savings to 54.74% while maintaining the encoding performance. Furthermore, the proposed method exceeds the latest methods of H.266/VVC.

Cross-Component Prediction in HEVC

Video coding in the YCbCr color space has been widely used, since it is efficient for compression, but it can result in color distortion due to conversion error. Meanwhile, coding in the RGB color space maintains high color fidelity, having the drawback of a substantial bitrate increase with respect to YCbCr coding. Cross-component prediction (CCP) efficiently compresses video content by decorrelating color components while keeping high color fidelity. In this scheme, the chroma residual signal is predicted from the luma residual signal inside the coding loop. This paper gives a description of the CCP scheme from several points of view, from theoretical background to practical implementation. The proposed CCP scheme has been evaluated in standardization communities and adopted into H.265/High Efficiency Video Coding (HEVC) Range Extensions. The experimental results show significant coding performance improvements for both natural and screen content video, while the quality of all color components is maintained. The average coding gains for natural video are 17% and 5% bitrate reduction in the case of intra coding and 11% and 4% in the case of inter coding for RGB and YCbCr coding, respectively, while the average increment of encoding and decoding times in the HEVC reference software implementation are 10% and 4%, respectively.

Steerable-Discrete-Cosine-Transform (SDCT): Hardware Implementation and Performance Analysis.

In the last years, the need for new efficient video compression methods grown rapidly as frame resolution has increased dramatically. The Joint Collaborative Team on Video Coding (JCT-VC) effort produced in 2013 the H.265/High Efficiency Video Coding (HEVC) standard, which represents the state of the art in video coding standards. Nevertheless, in the last years, new algorithms and techniques to improve coding efficiency have been proposed. One promising approach relies on embedding direction capabilities into the transform stage. Recently, the Steerable Discrete Cosine Transform (SDCT) has been proposed to exploit directional DCT using a basis having different orientation angles. The SDCT leads to a sparser representation, which translates to improved coding efficiency. Preliminary results show that the SDCT can be embedded into the HEVC standard, providing better compression ratios. This paper presents a hardware architecture for the SDCT, which is able to work at a frequency of 188, reaching a throughput of 3.00 GSample/s. In particular, this architecture supports 8k UltraHigh Definition (UHD) (7680 × 4320) with a frame rate of 60 , which is one of the best resolutions supported by HEVC.

Optimized Rate Control Algorithm of High-Efficiency Video Coding Based on Region of Interest

Aiming at the problems that the strategy of target bit allocation at the CTU layer has deviations from the human subjective observation mechanism, and the update phase of parametric model has a higher complexity in the JCTVC-K0103 rate control algorithm of ITU-T H.265/high efficiency video coding (HEVC) standard. Optimized rate control (ORC) algorithm of ITU-T H.265/HEVC based on region of interest (ROI) is proposed. Firstly, the algorithm extracts the region of interest of video frames based on time and space domains by using the improved Itti model. Then, the weight of target bits w is recalculated based on space-time domains to improve the rate control accuracy, and the target bits are distributed based on ROI by the adaptive weight algorithm once again to make the output videos more attuned with the human visual attention mechanism. Finally, the quasi-Newton algorithm is used to update the rate distortion model, which reduces the computational complexity in the update phase of the parametric model. The experimental results show that the ORC algorithm can obtain a better subjective quality in the compressed results with less bit error compared with the other two algorithms. Meanwhile, the rate distortion performance of the ORC algorithm is better on the premise of guaranteeing rate control performance.

Fast CU Partition and Intra Mode Decision Method for H.266/VVC

The Versatile Video Coding (H.266/VVC) standard has developed by Joint Video Exploration Team (JVET). Compared with the previous generation video coding standard, the H.266/VVC is more outstanding. Since the H.266/VVC introduces multi-type tree (MTT) structure including binary tree (BT) and ternary tree (TT), which brings the significant coding efficiency but increases coding complexity. Moreover, the intra prediction modes have increased from 35 to 67, which can provide more accurate prediction than H.265/High Efficiency Video Coding (HEVC). Therefore, these can improve the encoding quality, but increase computational complexity. To reduce the computational complexity, this paper designs a fast coding unit (CU) partition and intra mode decision algorithm, which includes fast CU partition based on random forest classifier (RFC) model and fast intra prediction modes optimization based on texture region features. Simulation results indicate that the proposed scheme can save 54.91% encoding time with only 0.93% increase in BDBR.

Efficient Algorithm Adaptations and Fully Parallel Hardware Architecture of H.265/HEVC Intra Encoder

The growing demand for high-performance ultra-high-definition video coding leads to H.265/high-efficiency video coding (HEVC), where the increased computational complexity and data/timing dependence hinder its coding throughput. To address these challenges, this paper presents four algorithm adaptations and a fully parallel hardware architecture for an H.265/HEVC intra encoder. To the best of our knowledge, this is the first fully parallel H.265/HEVC intra encoder. This design supports 35 prediction modes and all coding tree unit partitions. All PUs are independently processed in four prediction engines for high parallelism. An appropriate set of intra prediction modes, RDO candidates, and CABAC rate estimate instances is assigned to each prediction engine, where internal computational tasks are pipelined and scheduled to maximize the processing throughput. Compared with the HM-15.0 software, the proposed algorithm adaptations lead to a reduction of 27% in computational workload, while the average BD-rate and BD-PSNR are 4.39% and -0.21 dB, respectively. This BD-rate is lower than the existing designs with the same video resolution. FPGA implementation of the proposed design shows that it operates at 120 MHz and supports 45 fps of 1080P video sequences using 201-K logic elements and 120-KB on-chip SRAM. ASIC implementation of the proposed design in TSMC 90-nm technology shows that its clock frequency reaches 320 MHz with a hardware gate count of 2288 K, and that it supports real-time encoding of 30 fps of 4-K video sequences. Compared with the state-of-the-art designs, our proposed design demonstrates advantages in computational complexity, bit rate, video quality, throughput, reliability, and flexibility.

Design of High-Speed H.265 Sample Adaptive Offset Estimation for Ultra-Hd TV Encoding using Clock Synchronization Code

Sample adaptive offset (SAO) is a recently presented in-circle separating segment in H.265/High Efficiency Video Coding (HEVC). SAO adds to a striking coding effectiveness enhancement; the estimation of SAO parameters commands the multifaceted nature of in-circle sifting in HEVC encoding. Double clock engineering that procedures statistics collection (SC) and parameter decision (PD), the two principle useful squares of SAO estimation, at high-and low-speed timekeepers, separately. Such a technique decreases the general zone by 56% by tending to the heterogeneous information streams of SC and PD. This exploration work endeavors to ad lib the working clock speed by adjusting a double clock synchronizer VLSI structure of H.265 ultra HD encoder and control minimization. To additionally enhance the territory and power productivity, calculation engineering co-improvements are connected, including a coarse range selection (CRS) and an accumulator bit width reduction (ABR). They together may accomplish another 25% territory decrease. The proposed VLSI configuration is fit for handling 8k at 120-outlines/s encoding.

Quality-Oriented Perceptual HEVC Based on the Spatiotemporal Saliency Detection Model.

Perceptual video coding (PVC) can provide a lower bitrate with the same visual quality compared with traditional H.265/high efficiency video coding (HEVC). In this work, a novel H.265/HEVC-compliant PVC framework is proposed based on the video saliency model. Firstly, both an effective and efficient spatiotemporal saliency model is used to generate a video saliency map. Secondly, a perceptual coding scheme is developed based on the saliency map. A saliency-based quantization control algorithm is proposed to reduce the bitrate. Finally, the simulation results demonstrate that the proposed perceptual coding scheme shows its superiority in objective and subjective tests, achieving up to a 9.46% bitrate reduction with negligible subjective and objective quality loss. The advantage of the proposed method is the high quality adapted for a high-definition video application.