Real-time High Efficiency Video Coding Research Articles

A Noval Approach Of An Area Efficient Variable Size Comparison For Fixed Point 1D -DCT, 2D-DCT & IDCT Architecture For HEVC Encoding

The implementation of integer discrete cosine transform (DCT) of various lengths for use in High Efficiency Video Coding (HEVC) are proposed. Low strength hardware acceleration cores for integration into real-time High Efficiency Video Coding (HEVC) codec for clever phones, tablets, camcorders, and televisions are in exquisite demand. This want motivates one for an green attention of Discrete Cosine Transform (DCT) and Inverse-DCT (IDCT) for HEVC. My Implementation layout with low strength, area, and electricity functions may be protected in a real-time HEVC codec for HEVC-compliant purchaser digital devices. This end result is received via way of means of comparing extraordinary DCT factorizations to decide which one is good to be used in HEVC encoder. In comparison to latest implementations, simulation outcomes display that the proposed answer gives decreased fee losses & decreased complexity.

High-Level Synthesis Implementation of an Embedded Real-Time HEVC Intra Encoder on FPGA for Media Applications

High Efficiency Video Coding (HEVC) is the key enabling technology for numerous modern media applications. Overcoming its computational complexity and customizing its rich features for real-time HEVC encoder implementations, calls for automated design methodologies. This article introduces the first complete High-Level Synthesis (HLS) implementation for HEVC intra encoder on FPGA. The C source code of our open-source Kvazaar HEVC encoder is used as a design entry point for HLS that is applied throughout the whole encoder design process, from data-intensive coding tools like intra prediction and discrete transforms to more control-oriented tools such as context-adaptive binary arithmetic coding (CABAC). Our prototype is run on Nokia AirFrame Cloud Server equipped with 2.4 GHz dual 14-core Intel Xeon processors and two Intel Arria 10 PCIe FPGA accelerator cards with 40 Gigabit Ethernet. This proof-of-concept system is designed for hardware-accelerated HEVC encoding and it achieves real-time 4K coding speed up to 120 fps. The coding performance can be easily scaled up by adding practically any number of network-connected FPGA cards to the system. These results indicate that our HLS proposal not only boosts development time, but also provides previously unseen design scalability with competitive performance over the existing FPGA and ASIC encoder implementations.

A fast integrated deblocking filter and sample-adaptive-offset parameter estimation architecture for HEVC

Low power hardware acceleration cores for integration into real-time High Efficiency Video Coding (HEVC) codec for smartphones, tablets, camcorders, and televisions are in great demand. This demand motivates one for an efficient approximation of important power-consuming modules of HEVC including in-loop filters. This paper presents a hardware-efficient implementation of integrated deblocking filter (DBF) and sample adaptive offset (SAO) parameter estimation architecture for 16×16, 32×32, and 64×64 coding tree units (CTU) in HEVC. When the architecture is extended to HEVC-Test-Model (HM) Software, the luminance peak-signal-to-noise-ratio gets increased by 0.02 decibel, and the execution time of DBF and SAO gets decreased by at most 35% and 38%, respectively while compared to the reference algorithm. Moreover, it delivers rate–distortion performance comparable to the HEVC standard and reports mean-squared-error, structural-similarity (SSIM) index, and multi-scale SSIM (MS-SSIM) index of values 0.15, 0.9984, and 1, respectively for 4K video sequences. The architecture consumes minimum power, area, and energy equal to 9.83 milliwatts, 162 kilo-gate-equivalents, and 44 picojoules, respectively while supporting up to forty-six 8K frames per second. Additionally, it reports 78% smaller area and requires 75% less clock-cycles-per-largest-coding-unit as compared to the separate implementation of DBF and SAO with reference to HEVC. Such designs with low power, area, and energy can be integrated into a real-time HEVC codec for portable HEVC-compliant consumer electronic devices.

Hardware-Efficient 2D-DCT/IDCT Architecture for Portable HEVC-Compliant Devices

Low power hardware acceleration cores for integration into real-time High Efficiency Video Coding (HEVC) codec for smartphones, tablets, camcorders, and televisions are in great demand. This need motivates one for an efficient realization of Discrete Cosine Transform (DCT) and Inverse-DCT (IDCT) for HEVC. This paper presents an algorithm to calculate the required minimum number of low-frequency DCT-output/IDCT-input coefficients for 4, 8, 16, and 32-point DCT/IDCT in HEVC, such that there is a slight decrease in peak-signal-to-noise-ratio (<; 0.15 decibel) and a minor increment in bitrate (<; 1.5%) as compared to the reference HEVC-Test-Model (HM) Software. However, the encoding time gets reduced at most by 17.95% for Class-A type sequences, while reporting mean-squared-error and structural-similarity of 1.42 and 0.9913, respectively for 4K ultra-high-definition videos. Moreover, HEVC-compliant computationally efficient architectures are introduced for n-point DCT/IDCT. The presented flexible Transpose Memory architecture uses only sixteen random-access-memories to support all transform-unit sizes in HEVC. The proposed two-dimensional DCT/IDCT architecture can process up to 288@$4K$ frames-per-second, and it consumes the minimum power, energy, and area of 11.23 milliwatts, 2.34 picojoules, and 120 kilo-gate-equivalents, respectively. Such design with low power, area, and energy features can be included in a real-time HEVC codec for HEVC-compliant consumer electronic devices.

Video bit-rate requirements for 8K 120-Hz HEVC/H.265 temporal scalable coding: experimental study based on 8K subjective evaluations

Abstract 8K video parameters were designed to provide an immersive experience; meanwhile, special considerations are necessary to assess the entire screen subjectively. This paper studies the video bit-rate required for 8K 119.88-Hz (120-Hz) and 59.94-Hz (60-Hz) the high efficiency video coding (HEVC)/H.265 temporal scalable coding based on subjective evaluation experiments. To investigate the appropriate bit-rate for both 8K 120- and 60-Hz videos for broadcasting purposes, we compress 8K 120-Hz test sequences using software that emulates our real-time HEVC encoder and conduct two types of experiments. The experimental results demonstrate that the required video bit-rate for 8K 120-Hz temporal scalable coding is estimated to be 85–110 Mbps, which is equivalent to the practical bit-rate for 8K 60-Hz videos, and the appropriate bit-rate for 8K 60-Hz video in 8K 120-Hz video at 85 Mbps is assumed to be ~80 Mbps. From the analyses of the encoded videos, it is confirmed that the experimental results are primarily influenced by the image quality on the slice boundary positioned at the middle of the screen height. When conducting the experiments, we determined settings referring to an initial 8K subjective assessment; we further mention requirements for future 8K subjective evaluations from the experimental results.

A Novel Algorithmic Approach for Efficient Realization of 2-D-DCT Architecture for HEVC

Low power realization of high efficiency video coding (HEVC) standard, which has found widespread application in mobile devices, tablets, and high-definition televisions, calls for an efficient approximation of discrete cosine transform (DCT) technique. In this paper, a novel algorithm is proposed to determine the minimum number of low-frequency DCT coefficients required for transform and quantization block in HEVC. Moreover, it introduces hardware efficient 1-D architectures for 4, 8, 16, and 32-point DCT, that make use of the proposed algorithm and conform to the HEVC standard. When these architectures are extended to the HEVC test model (HM), it leads to a minor reduction of PSNR (<;0.15 dB) and a slight increase in bitrate (<;1.5%). The encoding time of HM encoder is reduced up to 16.02% and produces the rate-distortion performance comparable to the reference algorithm. A new memory efficient addressing scheme for transpose memory is presented for 2D-DCT. The proposed 2D-DCT architecture has a low power dissipation and energy consumption of 8.62 mW and 1.77 pJ respectively and can process at least sixty 8K ($7630\times 4830$ ) frames/s. Such architecture with low power and energy characteristics can be integrated into a real-time HEVC encoder for portable consumer electronic products.

A 120 fps High Frame Rate Real-time HEVC Video Encoder with Parallel Configuration Scalable to 4K

This paper describes a new 120 fps (frames per second) real-time HEVC (High Efficiency Video Coding) encoder for HFR (high frame rate) video encoding and transmission. HFR provides more immersive viewing experience features by solving the problems created by fast moving scenes. Temporally scalable encoding with backward compatibility for legacy non-HFR systems is suitable for the rapid spread of HFR content delivery, avoiding the need to distribute multiple bitstreams of the same video with different frame rates. Such temporal scalability requires flexible encoder control functionalities to support newly-customized reference picture structures and dual-stream bitrate control. In this paper, modification in the customizable software architecture of encoder LSIs makes it possible to achieve 120 fps temporally scalable HEVC encoding for existing 60 fps-based systems. The encoder also achieves ${4\mathrm{K}/ 120\;\mathrm{fps}}$ video encoding in real time through the synchronized operation of multiple ${2\mathrm{K}/ 120\;\mathrm{fps}}$ encoders working in parallel. Our evaluations show that the bitrate increase rate from 60 fps to 120 fps under the same objective image quality condition are within the range of less than 57.2 percent in all video sequences and its average value is 53.8 percent. Both values are lower than that of the HM (HEVC reference software). The proposed encoder systems will open the door to the next generation high frame rate UHDTV (ultra high definition television) services.

Fast HEVC Intra Coding by Predicting the Rate-distortion Cost for a Low-complexity Encoder

High Efficiency Video Coding (HEVC) intra coding achieves significant improvements in coding efficiency compared with H.264/AVC intra coding by adopting 35 spatial intra prediction modes and a quadtree-based block partitioning structure. However, the encoding complexity is extremely high from performing rate-distortion optimization (RDO) on each mode and depth level. This paper proposes fast intra mode decision algorithms for real-time HEVC encoding. For a fast intra prediction mode decision, we estimate the lower bound on the rate-distortion (RD) cost of a prediction unit with non–most-probable modes (non-MPMs). If the RD cost of the MPM is lower than the lower bound, the MPM is selected as the best mode. For a fast intra coding unit (CU) size decision, we predict the RD cost of four split sub-CUs from the RD of the current CU using the relationship between the RDs of the current CU and its split sub-CUs. In the HEVC Test Model software, the proposed algorithm saves 48.56% of the encoding time, on average, while the RD loss is only 0.59%. In x265, the proposed algorithm improves the encoding speed to 71.95 fps, on average, from 49.46 fps. It achieves 1.8 times faster speed with a bit increase of only 0.44%, compared to x265.

CABAC-based selective encryption for HEVC using RC6 in different operation modes

This paper proposes a selective encryption (SE) algorithm for the almost recently video compressing technique that is called High-Efficiency Video Coding (HEVC). The improvement of our HEVC SE algorithm is to secure real-time HEVC streaming using low computational overhead, same bit rate and ensuring the video format compliance. This contribution is achieved with low computational RC6 algorithm to encrypt subset of binstrings that is binarized using the bypass binary arithmetic coding mode in the context adaptive binary arithmetic coding (CABAC) process of HEVC. This binstrings subset is the non-zero discrete cosine transform (DCT) coefficients sign bits, motion vector difference (MVD) sign bits, remaining absolute values suffix of DCT, MVD absolute values suffix, Sample adaptive offset (SAO) sign bit, residual size, reference picture index and the delta Quantization Parameter (QP). Also, this paper investigates the performance evaluation of using the RC6 with its operation modes within the proposed HEVC CABAC SE. This investigation is done for choosing the best operation mode for RC6 to be suitable for being used in the HEVC real-time application. The security analysis like histogram analysis, correlation coefficients test and key sensitivity test are presented to make sure the protection of HEVC CABAC SE algorithm against brute force and statistical attacks. The performance analysis results prove that the HEVC CABAC SE is highly secure and can be used in real-time HEVC applications.

A computation and energy reduction technique for HEVC intra prediction

Intra prediction algorithm used in High Efficiency Video Coding (HEVC) standard has very high computational complexity. Therefore, in this paper, a novel technique is proposed for reducing amount of computations performed by HEVC intra prediction algorithm and, therefore, reducing energy consumption of HEVC intra prediction hardware. The proposed technique significantly reduced the amount of computations performed by 4x4, 8x8, 16x16 and 32x32 luminance angular prediction modes. The proposed technique does not affect the PSNR and bit rate. In this paper, a low energy HEVC intra prediction hardware for 4x4, 8x8, 16x16 and 32x32 angular prediction modes is also designed and implemented using Verilog HDL. The proposed technique significantly reduced the energy consumption of the HEVC intra prediction hardware. Therefore, it can be used in portable consumer electronics products that require a real-time HEVC encoder.

Implementation of 4kUHD HEVC-content transmission

The Internet of things (IoT) has received a great deal of attention in recent years, and is still being approached with a wide range of views. At the same time, video data now accounts for over half of the internet traffic. With the current availability of beyond high definition, it is worth understanding the performance effects, especially for real-time applications. High Efficiency Video Coding (HEVC) aims to provide reduction in bandwidth utilisation while maintaining perceived video quality in comparison with its predecessor codecs. Its adoption aims to provide for areas such as television broadcast, multimedia streaming/storage, and mobile communications with significant improvements. Although there have been attempts at HEVC streaming, the literature/implementations offered do not take into consideration changes in the HEVC specifications. Beyond this point, it seems little research exists on real-time HEVC coded content live streaming. Our contribution fills this current gap in enabling compliant and real-time networked HEVC visual applications. This is done implementing a technique for real-time HEVC encapsulation in MPEG-2 Transmission Stream (MPEG-2 TS) and HTTP Live Streaming (HLS), thereby removing the need for multi-platform clients to receive and decode HEVC streams. It is taken further by evaluating the transmission of 4k UHDTV HEVC-coded content in a typical wireless environment using both computers and mobile devices, while considering well-known factors such as obstruction, interference and other unseen factors that affect the network performance and video quality. Our results suggest that 4kUHD can be streamed at 13.5 Mb/s, and can be delivered to multiple devices without loss in perceived quality.

A computation and energy reduction technique for HEVC Discrete Cosine Transform

In this paper, a novel computation and energy reduction technique for High Efficiency Video Coding (HEVC) Discrete Cosine Transform (DCT) for all Transform Unit (TU) sizes is proposed. The proposed technique reduces the computational complexity of HEVC DCT significantly at the expense of slight decrease in PSNR and slight increase in bit rate by only calculating several pre-determined low frequency coefficients of TUs and assuming that the remaining coefficients are zero. It reduced the execution time of HEVC HM software encoder up to 12.74%, and it reduced the execution time of DCT operations in HEVC HM software encoder up to 37.27%. In this paper, a low energy HEVC 2D DCT hardware for all TU sizes is also designed and implemented using Verilog HDL. The proposed hardware, in the worst case, can process 53 Ultra HD (7680x4320) video frames per second. The proposed technique reduced the energy consumption of this hardware up to 18.9%. Therefore, it can be used in portable consumer electronics products that require a real-time HEVC encoder. 1

4K Real-Time HEVC Decoder on an FPGA

With the popularization of a quad high-definition/4K video being dependent on the availability of real-time High Efficiency Video Coding (HEVC) decoders, hardware implementations have become more appealing due to their superior performance and low power consumption. In this paper, a field-programmable gate array (FPGA)-based hardware implementation of a 4K 30 frames/s real-time HEVC decoder is presented. An elastically pipelined decoder architecture is used to absorb variations in processing time and each pipeline stage is optimized to use available FPGA primitives. FPGA-specific challenges in managing critical path delays to achieve a target operating frequency of 150 MHz required many architectural novelties, such as exploitation of the sparsity of transformed coefficient matrix, single-cycle reference pixel processing in intra prediction, and flexible <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8 \times 8$ </tex-math></inline-formula> block ordering in deblocking filter/sample adaptive offset filter. A high-throughput latency-aware cache architecture was used to reduce the external dynamic RAM access bandwidth by 70%. This work is compliant with the HEVC main profile at level 5 of the HEVC standard and only consumes 126K lookup tables, 58K registers, and 335 18-kb block RAMs when implemented on Xilinx Zynq 7045.

Low power HEVC software decoder for mobile devices

In the context of mobile handheld devices, energy consumption is a primary concern and the process of video decoding is often among the most resource-intensive applications. Recent embedded processors are equipped with advanced features such as dynamic voltage frequency scaling (DVFS) in order to reduce their power consumption. These features can be used to perform low power video decoding when no hardware decoding support is available for a given standard. High efficiency video coding (HEVC) is a recent video standard offering state-of-the-art compression rates and advanced parallel processing solutions. This paper presents strategies for the power optimization of a real-time software HEVC decoder on NEON architecture. These strategies include the exploitation of data and task-level parallelism, as well as the use of a new frequency control system to optimize the processor DVFS, based on an estimation of the decoding complexity. Extensive power measurement results, based on a multi-core ARM big.LITTLE processor, are provided and compared to state-of-the-art. These results show that the proposed open-source implementation can reach an energy consumption below 21 nJ/px for HD decoding at 2.2 Mbits/s.

Efficient algorithm for transferring a real-time HEVC stream with haptic data through the internet

It is widely accepted that the growth of Internet and the improvement of Internet's network conditions helped real-time applications to flourish. The demand for Ultra-High Definition video is constantly increasing. Apart from video and sound, a new kind of real-time data is making its appearance, haptic data. The efficient synchronization of video, audio, and haptic data is a rather challenging effort. The new High-Efficiency Video Coding (HEVC) is quite promising for real-time ultra-high definition video transferring through the Internet. This paper presents related work on High-Efficiency Video Coding. It points out the challenges and the synchronization techniques that have been proposed for synchronizing video and haptic data. Comparative tests between H.264 and HEVC are undertaken. Measurements for the network conditions of the Internet are carried out. The equations for the transferring delay of all the inter-prediction configurations of the HEVC are defined. Finally, it proposes a new efficient algorithm for transferring a real-time HEVC stream with haptic data through the Internet.

A computation and energy reduction technique for HEVC intra mode decision

High Efficiency Video Coding (HEVC) intra mode decision algorithm has very high computational complexity. Therefore, in this paper, a computation and energy reduction technique is proposed for reducing the amount of computations performed by Sum of Absolute Transformed Difference (SATD) calculations in HEVC intra mode decision, and therefore reducing the energy consumption of HEVC SATD calculation hardware without any PSNR loss and bit rate increase. The proposed technique reduced the energy consumption of HEVC SATD calculation hardware up to 64.6%. Therefore, it can be used in portable consumer electronics products that require a real-time HEVC encoder.

A low energy HEVC inverse transform hardware

In this paper, a novel energy reduction technique for High Efficiency Video Coding (HEVC) Inverse Discrete Cosine Transform (IDCT) and Inverse Discrete Sine Transform (IDST) for all transform unit (TU) sizes is proposed. The proposed technique calculates IDCT and IDST only for DC coefficient if the values of several predetermined forward transformed low frequency coefficients in a TU are smaller than a threshold. The proposed technique reduces the computational complexity of IDCT and IDST significantly. It increases the bit rate slightly for most video frames. It decreases the PSNR slightly for some video frames, and it increases the PSNR slightly for some video frames. In this paper, a low energy HEVC 2D inverse transform (IDCT and IDST) hardware for all TU sizes is also designed and implemented using Verilog HDL. In the worst case, the proposed hardware can process 48 Quad HD (3840x2160) video frames per second. The proposed technique reduced the energy consumption of this hardware up to 32%. Therefore, the proposed hardware can be used in portable consumer electronics products that require a real-time HEVC encoder.

FPGA implementation of a full HD real-time HEVC main profile decoder

High Efficiency Video Coding (HEVC) is the newest video coding standard approved by the ISO/IEC and ITU-T in January 2013. By providing a video coding efficiency gain up to 50 % compared to H.264/MPEG-4 AVC high profile, the complexity of the used algorithms has raised significantly. Targeting video formats with higher spatial and temporal resolutions - e.g. 4Kp60 in broadcast applications - make implementing encoders and decoders a challenging task. A few software based implementations on DSPs and general purpose CPUs are known from the literature which suffer from real-time constraints, power dissipation and hardware costs of these systems. In this paper a pure hardware implementation of a Main Profile H.265/MPEG-HEVC Full HD capable decoder is presented solving both real-time and power constraints respectively. As a first implementation approach a state-of-the-art FPGA technology is chosen as a prototyping platform. This design can be used as a starting point for an ASIC implementation.

Complexity scalability for real-time HEVC encoders

The high efficiency video coding (HEVC) standard achieves improved compression efficiency in comparison to previous standards at the cost of much higher computational complexity and consequently longer processing times, which may compromise real-time software-based video encoding, especially at high resolutions. This article addresses the problem of enabling complexity scalability in HEVC encoders by trading-off processing time for rate---distortion (R---D) performance in a controlled manner. The proposed method is based on dynamic constraining of HEVC coding treeblocks (CTBs) by limiting the prediction block (PB) shapes and the maximum tree depth used in each CTB, to decrease the number of R---D evaluations performed in the optimization process. The complexity-scalable encoder is capable of adjusting the processing time used in each group of pictures, according to a predefined target. The results show that processing times can be scaled down to 50 % with negligible R---D performance losses and down to 20 % at a maximum BD-PSNR decrease of 1.41 dB, which is acceptable in many applications and in power constrained devices. The simplicity of the scaling algorithm and the possibility of continuous adjustment of the scaling factor make it amenable to control real-time software-based HEVC video encoders.

소프트웨어 기반 실시간 HEVC 인코더 구현을 위한 병렬화 기법에 관한 연구

ISO/IEC MPEG+ITU-T VCEG!,-./JCT-VC (Joint Collaborative Team on Video Coding)01)2 3HEVC (High Efficiency Video Coding)4H.264/AVC 567289:;!% ?@A1.1)0BCDE. FGH, IJK2L40MN OP<Q+RSKTUVW <XYZ[04\]^_`abc-.GKCdeE. HEVC  <XYZ2f g FhEidjkdlmn!bC de-o, djkdlm-.pqe4jZr%H-.HEVC  </stF 4uv!eE. wxay4R1)0z{|HEVC  <FhSIMD }~2!!€q) , CPU GPU2!‚ƒ„ † +‡:Ei  E. ˆ, !‰ nHEVC  KF ^KŠ ‹ ŒŽ5Fh E. w ybHM (HEVC reference model) 10.0K+832×480 ‘%<’“20~30fps<TUjZ2o”•-–, 1920×1080 ‘%<’“5~10fps<TUjZ2o”—DE.AbstractJoint Collaborative Team on Video Coding (JCT-VC), which have founded ISO/IEC MPEG and ITU-T VCEG, has standardized High Efficiency Video Coding (HEVC). Standardization of HEVC has started with purpose of twice or more coding performance compared to H.264/AVC. However, flexible and hierarchical coding block and recursive coding structure are problems to overcome of HEVC standard. Many fast encoding algorithms for reducing computational complexity of HEVC encoder have been proposed. However, it is hard to implement a real-time HEVC encoder only with those fast encoding algorithms. In this paper, for implementation of software-based real-time HEVC encoder, data-level parallelism using SIMD instructions and CPU/GPU multi-threading methods are proposed. And we also proposed appropriate operations and functional modules to apply the proposed methods on HM 10.0 software. Evaluation of the proposed methods implemented on HM 10.0 software showed 20-30fps for 832×480 sequences and 5-10fps for 1920×1080 sequences, respectively. Keyword : HEVC, encoder, real-time, parallelization˜#xa(Regular Paper)™,š›xaGb18œb6U, 201311ž(JBE Vol. 18, No. 6, November 2013)http://dx.doi.org/10.5909/JBE.2013.18.6.835ISSN 2287-9137 (Online) ISSN 1226-7953 (Print)