High Efficiency Video Coding Encoder Research Articles

Complexity control algorithm based on adaptive mode selection for interframe coding in high efficiency video coding

The latest high efficiency video coding (HEVC) standard significantly increases the encoding complexity for improving its coding efficiency. Due to the limited computational capability of handheld devices, complexity constrained video coding has drawn great attention in recent years. A complexity control algorithm based on adaptive mode selection is proposed for interframe coding in HEVC. Considering the direct proportionality between encoding time and computational complexity, the computational complexity is measured in terms of encoding time. First, complexity is mapped to a target in terms of prediction modes. Then, an adaptive mode selection algorithm is proposed for the mode decision process. Specifically, the optimal mode combination scheme that is chosen through offline statistics is developed at low complexity. If the complexity budget has not been used up, an adaptive mode sorting method is employed to further improve coding efficiency. The experimental results show that the proposed algorithm achieves a very large complexity control range (as low as 10%) for the HEVC encoder while maintaining good rate-distortion performance. For the lowdelayP condition, compared with the direct resource allocation method and the state-of-the-art method, an average gain of 0.63 and 0.17 dB in BDPSNR is observed for 18 sequences when the target complexity is around 40%.

Unimodal Stopping Model-Based Early SKIP Mode Decision for High-Efficiency Video Coding

High-efficiency video coding (HEVC) can greatly improve coding efficiency compared with the prior video coding standard H.264/AVC by adopting advanced hierarchical coding structures such as coding unit (CU), prediction unit (PU), and transform unit. For each CU, an exhaustive mode decision strategy is adopted to achieve the best rate distortion (RD) cost, which simultaneously results in enormous computational complexity. In this paper, an early SKIP mode decision algorithm is proposed for the HEVC encoder to speed up the process of mode decision. Each CU size is categorized into either rare used or frequent used by exploiting the correlation of CU depth, which is estimated from the temporally colocated CUs. For the rare-used CU size, the SKIP mode is directly selected as the optimal mode and the remaining mode decision process is early terminated. For the frequent-used CU size, a unimodal stopping model is designed for its early SKIP mode decision by exploiting both hierarchical mode structure and RD cost property. Experimental results show that the proposed early SKIP mode decision method achieves average 58.5% and 54.8% encoding time savings, while the Bjontegaard Delta bit rate only increases average 0.8% and 0.8% for various test sequences under the random access and the low delay B conditions, respectively.

Low complexity based ultra-high quality video compression method for multimedia-centric internet of things (IoT) services

Internet of things (IoT) enables a number of embedded systems to interact for the purpose of various IoT applications such as home security, medical, smart surveillance, and etc. It is expected that the need of multimedia computing for ultra-high quality video delivery has further increased the importance of fast and complexity-awareness video compression algorithm under low-complexity and low-power IoT systems. High efficiency video coding (HEVC) is the state-of-the-art video coding technology that can provide powerful video compression performance under limited bandwidth conditions for transmission or storage. Although HEVC adopted newly advanced video coding tools to achieve a bitrate reduction of 50% with similar video quality compared to the previous method, H.264/AVC, these cause heavy computational encoding complexity resulting from inter prediction process of HEVC encoder. In this paper, we propose a complexity scalable SKIP/MERGE encoding algorithm to design a low complexity inter prediction. Experimental results show that the proposed method is much faster than those of HEVC test model (47.42%) and previous method (13.95%) in terms of total encoding time, on average.

Fast intra coding unit decision for high efficiency video coding based on statistical information

The latest video coding compression standard is known as high–efficiency video coding (HEVC). It supports high-resolution video sequences and has better coding performance than the previous standard H.264/AVC. A quad-tree based coding unit (CU) partitioning process is one of the most efficient technologies used in an HEVC encoder. A coding tree unit (typically 64×64) can be split into smaller CUs based on rate-distortion optimization, allowing various types of video content to be adaptively compressed. In addition, intra prediction of the HEVC standard supports 35 prediction modes (planar, DC, and 33 angular modes) to improve coding efficiency. However, the computational complexity of HEVC encoder becomes a critical problem when implement with an encoder. Thus, a fast CU size decision algorithm for intra prediction of an HEVC encoder is proposed in this study. We utilize image complexity and an adaptive depth prediction for early split CU decision making. In addition, the Bayesian decision rule and quadratic discriminant analysis are used for early termination of the CU partitioning process. Experimental results show that our proposed algorithm considerably reduces encoding time by approximately 55.47% with only a small BD-BR loss (1.01%) compared to the HEVC reference software HM 16.0.

Fast CU size and prediction mode decision algorithm for HEVC based on direction variance

As the latest video coding standard, High Efficiency Video Coding (HEVC) achieves high coding performance, up to 50% bit rate savings compared with the previous standard, H.264/Advanced Video Coding while maintaining the same output video quality. Intra-coding in the HEVC significantly improves the compression efficiency, due to the various kinds of coding unit (CU) sizes, and high density of angular prediction modes. However, the improvement of the coding performance is obtained at the expense of the extraordinary computation complexity, which obstructs the usefulness of HEVC encoder for real-time applications. This paper presents a novel intra-encoding algorithm for HEVC, which is composed of an efficient CU partitioning technique and a fast intra-mode decision method based on direction variance. We utilize a preprocessing method in which the texture complexity and direction energy distribution along a special direction are extracted from a macroblock. According to the texture complexity obtained, an early CU splitting termination is proposed to decide whether a CU should be decomposed into four lower-dimensions CUs or not. Furthermore, a subset of prediction modes in accordance with the dominant direction energy distribution is chosen for the further rough mode decision and rate–distortion optimization process. The simulation results indicate the algorithm based on image understanding achieves better trade-off between rate–distortion performance and complexity reduction than the previous algorithms. Compared with the reference software HM16.7, the proposed algorithm can save 57% coding time on average, with a negligible bit rate increase of 0.65%, and quality losses lower 0.08 dB, respectively.

A QoE-Driven Encoder Adaptation Scheme for Multi-User Video Streaming in Wireless Networks

Video streaming over networks has grown rapidly in recent years. Increasing focus has gradually turned from quality of service awareness to end viewer's quality of experience (QoE) awareness. For multi-user video streaming services, videos are encoded and transmitted through a bandwidth-limited wireless networks. Therefore, proper encoder settings for videos are required to satisfy the total channel rate constraint. In this paper, a QoE-driven High Efficiency Video Coding (HEVC) encoder adaptation scheme is proposed, aiming to maximize QoE of all the users for wireless networks. First, the influence of HEVC encoder on video streaming is investigated to generate an encoder parameter model. Then, the effect of network impairment together with HEVC encoder is taken into account to derive a subjective QoE prediction model. Afterward, we formulate a QoE-maximized encoder adaptation with total channel rate constraint into an optimization problem based on the obtained encoder parameter model and the subjective QoE model. The problem is solved and incorporated into an online encoder adaptation scheme. Compared with conventional rate allocation methods, the proposed QoE-driven encoder adaptation scheme achieves significant QoE gains independent of wireless network channel rate and video content. Simulation results demonstrate that the proposed encoder adaptation scheme can improve the QoE by 31.4% with fixed number of users or increase the number of satisfied users by 35.5% with variable number of users.

A Bit-Plane Decomposition Matrix-Based VLSI Integer Transform Architecture for HEVC

In this brief, a new very-large-scale integrated (VLSI) integer transform architecture is proposed for the High Efficiency Video Coding (HEVC) encoder. The architecture is designed based on the signed bit-plane transform (SBT) matrices, which are derived from the bit-plane decompositions of the integer transform matrices in HEVC. Mathematically, an integer transform matrix can be equally expressed by the binary weighted sum of several SBT matrices that are only composed of binary 0 or ±1. The SBT matrices are very simple and have lower bit width than the original integer transform in the form. The SBT matrices are also sparse and there are many zero elements. The sparse characteristic of SBT matrices is very helpful for saving the addition operators of SBT. In the proposed architecture, instead of the original integer transform in high bit width, the video data can be respectively transformed with the SBT matrices in lower bit width. As a result, the delay of the transform unit circuit can be significantly reduced with the proposed SBT. Moreover, exploiting the redundant element characteristic of SBT matrices, in which the elements are 0 or ±1, the adder reuse strategy is proposed for our transform architecture, which can save the circuit area efficiently. The simulation results show that by employing the proposed strategies the VLSI transform architecture can be synthesized in a proper area with a high working frequency and low latency. The architecture can support all HEVC encoders coding ultra high-definition video sequences in real time.

Towards an FPGA-Based HEVC Encoder: A Low-Complexity Rate Distortion Scheme for AMVP

Advanced motion vector prediction (AMVP) is a new technique adopted in the latest high efficiency video coding (HEVC) standard. AMVP block predicts an initial motion vector of the current block from a given set of candidates by means of rate distortion (RD) optimization process. Due to the large number of different-sized blocks, simplification of RD optimization process in AMVP block is highly appreciated. Therefore, we present a new RD optimization technique for AMVP block in HEVC encoder. The proposed RD calculation approach finds the best AMVP candidate by processing less number of feature pixels per every block. Experimental results show notable speedup in terms of AMVP processing time with tolerable quality degradation (PSNR) and bitrate requirement. The proposed RD calculation technique reduces the RD computational complexity of the AMVP block by 87.5% as maximum (i.e. 1.7% of the whole encoder complexity). This improvement is accompanied with a modest average PSNR loss of 0.10 dB and an increase by 2.4% in terms of bitrate. On the other hand, we present an FPGA-based architecture for AMVP unit in HEVC encoder. The proposed architecture was prototyped, simulated and synthesized on Xilinx Virtex-7 XC7VX550T FPGA. At 188 MHz clock frequency, the proposed architecture processes 8 K (7680 \(\times \) 4320) YCrCb resolution at 60 fps while utilizing less than 1% of the FPGA resources.

Enhanced Test Zone Search Algorithm for High Efficiency Video Coding Encoders

Enhanced Test Zone Search Algorithm for High Efficiency Video Coding Encoders

A computation and energy reduction technique for HEVC intra prediction

Intra prediction algorithm used in High Efficiency Video Coding (HEVC) standard has very high computational complexity. Therefore, in this paper, a novel technique is proposed for reducing amount of computations performed by HEVC intra prediction algorithm and, therefore, reducing energy consumption of HEVC intra prediction hardware. The proposed technique significantly reduced the amount of computations performed by 4x4, 8x8, 16x16 and 32x32 luminance angular prediction modes. The proposed technique does not affect the PSNR and bit rate. In this paper, a low energy HEVC intra prediction hardware for 4x4, 8x8, 16x16 and 32x32 angular prediction modes is also designed and implemented using Verilog HDL. The proposed technique significantly reduced the energy consumption of the HEVC intra prediction hardware. Therefore, it can be used in portable consumer electronics products that require a real-time HEVC encoder.

HEVC 부호화기의 적응적 참조 구조 변경 방법

본 논문에서는 HEVC (High Efficiency Video Coding) 부호화기의 성능 향상을 위한 적응적 참조 구조 변경 방법을 제안한다. 제안하는 방법은 입력 영상 내에 장면 전환, 장면 회전, Light on/off, Fade in/out 등의 이벤트 발생 시 저하되는 화면 간 예측의 성능을 향상시키기 위해 참조 픽쳐 리스트를 수정하여 참조 구조를 변경한다. GOP (Group Of Pictures) 단위로 한정된 계층적 참조 구조에서 영상 이벤트가 발생한 픽쳐를 기준으로 두 개의 서브 그룹으로 분할하고 각각의 서브 그룹 내에서 참조 픽쳐를 결정한다. 이를 통해, 시간적 중복성이 높은 참조 픽쳐를 화면 간 예측에서 사용하여 부호화 효율을 향상시킨다. 또한, HEVC의 계층적 부호화 구조 특징을 이용하여 이벤트 발생 후 처음으로 부호화되는 픽쳐를 CRA (Clean Random Access) 픽쳐로 변경하여 부호화 속도를 향상시키는 방법을 제안한다. 제안하는 방법은 이벤트 발생 후 처음으로 부호화되는 프레임은 화면 내 예측이 매우 높은 확률로 선택된다는 통계적 특징을 기반으로 화면 간 예측을 수행하지 않는다. 실험 결과 본 논문에서 제안하는 적응적 참조 구조 변경 방법은 HM 16.0 대비 CTC (Common Test Condition)에서 평균 0.3%의 BD-rate를 개선하였으며 부호화 속도는 평균 4.9% 향상시켰다. 또한, 이벤트 발생에 따른 픽쳐 타입 변경 방법은 평균 0.11%의 BD-rate 저하가 있었지만 부호화 속도를 평균 12.2% 향상시킬 수 있었다. This paper proposes adaptive reference structure decision method to improve the performance of HEVC (High Efficiency Video Coding) encoder. When an event occurs in the input sequence, such as scene change, scene rotation, fade in/out, or light on/off, the proposed algorithm changes the reference structure to improve the inter prediction performance. The proposed algorithm divides GOP (Group Of Pictures) into two sub-groups based on the picture that has such event and decides the reference pictures in the divided sub-groups. Also, this paper proposes fast encoding method which changes the picture type of first encoded picture in the GOP that has such event to CRA (Clean Random Access). With the statistical feature that intra prediction is selected by high probability for the first encoded picture in the GOP carrying such event, the proposed fast encoding method does not operate inter prediction. The experimental result shows that the proposed adaptive reference structure decision method improves the BD-rate 0.3% and reduces encoding time 4.9% on average under the CTC (Common Test Condition) for standardization. In addition, the proposed reference structure decision method with the picture type change reduces the average encoding time 12.2% with 0.11% BD-rate loss.

Gradient-based pre-processing for intra prediction in High Efficiency Video Coding

In order to reach higher coding efficiency compared to its predecessor, a state-of-the-art video compression standard, the High Efficiency Video Coding (HEVC), has been designed to rely on many improved coding tools and sophisticated techniques. The new features are achieving significant coding efficiency but at the cost of huge implementation complexity. This complexity has increased the HEVC encoders’ need for fast algorithms and hardware friendly implementations. In fact, encoders have to perform the different encoding decisions, overcoming the real-time encoding constraint while taking care of coding efficiency. In this sense, in order to reduce the encoding complexity, HEVC encoders rely on look-ahead mechanisms and pre-processing solutions. In this context, we propose a gradient-based pre-processing stage. We investigate particularly the Prewitt operator used to generate the gradient and we propose necessary approaches that enhance the gradient performance of detecting the HEVC intra modes. We also set different probability scenarios, based on the gradient information, in order to speed up the mode search process. Moreover, we propose a gradient-based estimation of the texture complexity that we use for coding unit decision. Results show that the proposed algorithm achieves a reduction of 42.8% in encoding time with an increase in BD rate of only 1.1%.

Design and implementation of a highly efficient fractional motion estimation for the HEVC encoder

This paper presents the VLSI architecture and implementation of the highly efficient fractional motion estimation (FME) for High Efficiency Video Coding (HEVC) systems. In this design, the processing sequence of input pixels is highly optimized so that large parts of the hardware resources in the interpolator circuit are shared and the area complexity is greatly reduced. In order to further enhance the efficiency of hardware utilization and to achieve high throughput, the sum of absolute transformed differences circuit is realized using a pipelined time-multiplexing scheme, and the hardware-sharing structure is utilized to reduce the required computation components. Furthermore, comparing to the literature based on HEVC systems, this design enhances the rate-distortion performance by supporting more prediction modes. The proposed design has been synthesized, placed, and routed through cell-based design flow using TSMC 90-nm technology. The post-layout estimations show that, occupying the area complexity of 525.4 kGE, the presented FME architecture achieves 39 frames per second (fps) with resolution of 3840 × 2160 and 29 fps with resolution of 7680 × 4320. In addition, comparing to the prior art that supports 30 fps and the same number of prediction modes, this design achieves a 2 × improvement of hardware efficiency.

Visual tracking in HEVC compressed videos

We present a robust visual tracking algorithm to track a rigid object in high efficiency video coding (HEVC) compressed‐domain. Based on a Markov random field model, we estimate the target object state by using motion vector extracted from compressed bitstream. Especially, we employ residual discrete cosine transform (DCT) coefficient (RDC) in the bitstream as the shape information of the target object to prevent the drift problem. The measurements of motion vector and RDC are adaptively reflected through our potential functions for the target state. Our experiments on test sequences compressed by an HEVC encoder show that it is robust to track rigid objects. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

A DSP-Based Implementation of HEVC Encoder

The new High Efficiency Video Coding (HEVC) standard will sooner replace the H.264/AVC standard and will be embedded in the majority of multimedia devices (smartphones, digital camera, UHD TV). It is expected to reduce the bit rate by half for the same quality compared to its predecessor H.264/AVC. However, the complexity and computational time for HEVC are much higher makinghard to achieve a real-time embedded solution for HEVC encoder with classic embedded processor technology. Consequently, new technologies of programmable processors such multicore Digital signal Processor (DSP) offer a very promising solution to overcome these constraints. In this paper, an optimized implementation of HEVC video encoder on a single core among the eight cores of TMS320C6678 DSP is presented in order to move afterwards to a multicore implementation. Based on the DSP architectural features, various Single-instruction-multiple-data (SIMD) optimizations are adopted to optimize the encoding time of the most consuming functions. Combined with the integrated fast encoding configuration in the reference HEVC software HM12.1, experimental results show that the whole proposed optimizations on a single core running at 1.25 GHz allow saving up to 78% of the encoding time with an acceptable distortion in terms of PSNR and bit-rate. The obtained results will make possible to achieve a real-time implementation when exploiting the multicore features.

Parallelization and performance evaluation of open-source HEVC codecs

High Efficiency Video Coding (HEVC) was developed by the Joint Collaborative Team on Video Coding (JCT-VC) to replace the current H.264/Advanced Video Coding (AVC) standard, which has dominated digital video services in all segments of the domestic and professional markets for over 10 years. In terms of rate-distortion, HEVC roughly doubles the compression performance of H.264/AVC, but at a cost of extremely high computational complexities during encoding. Parallelizing HEVC encoding is an efficient way of fulfilling this computational requirement. Since its standardization, several open-source HEVC video codecs have been developed with parallel encoding capabilities. This paper presents a rate-distortion/complexity analysis of the open-source HEVC codecs using objective measures of assessment to analyze their real parallelization capabilities. Experimental results show that DivX265 and x265 obtain the best parallel performance on average for the high-quality encoding scenario. In the case of the high-speed encoding scenario, x265 obtains the best parallel performance. In both scenarios, the coding efficiency of the encoders remains virtually unaffected regardless of the amount of parallelism.

Fast Prediction Mode Decision in HEVC Using a Pseudo Rate-Distortion Based on Separated Encoding Structure

A novel fast algorithm is suggested for a coding unit (CU) mode decision using pseudo rate-distortion based on a separated encoding structure in High Efficiency Video Coding (HEVC). A conventional HEVC encoder requires a large computational time for a CU mode prediction because prediction and transformation procedures are applied to obtain a rate-distortion cost. Hence, for the practical application of HEVC encoding, it is necessary to significantly reduce the computational time of CU mode prediction. As described in this paper, under the proposed separated encoder structure, it is possible to decide the CU prediction mode without a full processing of the prediction and transformation to obtain a rate-distortion cost based on a suitable condition. Furthermore, to construct a suitable condition to improve the encoding speed, we employ a pseudo rate-distortion estimation based on a Hadamard transformation and a simple quantization. The experimental results show that the proposed method achieves a reduction in the total encoding time with a similar coding performance to that of the HEVC reference model.

Inter-Prediction Optimizations for Video Coding Using Adaptive Coding Unit Visiting Order

The flexible partitioning scheme and increased number of prediction modes in the high efficiency video coding (HEVC) standard are largely responsible for both its high compression efficiency and computational complexity. In typical HEVC encoder implementations, coding units (CUs) in a coding tree unit (CTU) are visited from top to bottom at each level of recursion to select the optimal coding configuration. In this paper, a novel approach is presented in which CUs in a CTU can be adaptively visited also in a reverse, bottom to top visiting order. This reverse CU (RCU) visiting order allows for different algorithmic optimizations for further complexity reduction of many HEVC encoding steps, especially under challenging conditions, such as highly textured or fast moving content. In particular, algorithms to reduce complexity of HEVC depth selection, mode decision, and inter-prediction are presented here based on the coding information obtained from higher depths when using the RCU visiting order. Experimental results show that enabling different stages of the proposed algorithm can achieve average speed-ups from $16.3\%$ to $36.6\%$ compared to fast reference HEVC implementation with pre-built speed-ups enabled (up to $51.2\%$ in some cases), for $0.3\%$ to $2.2\%$ BD-rate penalty.

Fast reference frame selection based on content similarity for low complexity HEVC encoder

The high efficiency video coding (HEVC) is the state-of-the-art video coding standard, which achieves about 50% bit rate saving while maintaining the same visual quality as compared to the H.264/AVC. This achieved coding efficiency benefits from a set of advanced coding tools, such as the multiple reference frames (MRF) based interframe prediction, which efficiently improves the coding efficiency of the HEVC encoder, while it also increases heavy computation into the HEVC encoder. The high encoding complexity becomes a bottleneck for the high definition videos and HEVC encoder to be widely used in real-time and low power multimedia applications. In this paper, we propose a content similarity based fast reference frame selection algorithm for reducing the computational complexity of the multiple reference frames based interframe prediction. Based the large content similarity between the parent prediction unit (Inter_2N×2N) and the children prediction units (Inter_2N×N, Inter_N×2N, Inter_N×N, Inter_2N×nU, Inter_2N×nD, Inter_nL×2N, and Inter_nR×2N), the reference frame selection information of the children prediction units are obtained by learning the results of their parent prediction unit. Experimental results show that the proposed algorithm can reduce about 54.29% and 43.46% MRF encoding time saving for the low-delay-main and random-access-main coding structures, respectively, while the rate distortion performance degradation is negligible.

Online learning early skip decision method for the HEVC Inter process using the SVM‐based Pegasos algorithm

High efficient video coding (HEVC), the latest coding standard, has an encoding complexity much higher compared with H.264/advanced video coding (AVC). The greater efficiency in HEVC is obtained at much greater computational cost compared with AVC. A fast coding unit (CU) splitting algorithm is proposed for the HEVC encoder, which early terminates the CU partitioning process based on an adaptive classification model. This model is generated by an online learning method based on a Pegasos (primal estimated sub-gradient solver for SVM) algorithm. The proposed method is implemented over the HEVC reference implementation on its version 16.7. Experimental results show that the proposed method reduces the computational complexity of HEVC encoder to 35% without any loss, resulting in a 1% of Bjøntegaard Delta-rate gain in the low delay B configuration without any offline training phase.