Wavefront Parallel Processing Research Articles

Efficient VVC Encoding Using Hierarchical Parallelization: A Comprehensive Analysis

This paper presents and analyzes different parallelization strategies in VVenC, an open and optimized software encoder implementation of the Versatile Video Coding (VVC) standard. VVC has been developed to address the increasing demand for higher compression of digital video data, and it reduces the bitrate by around 50% for the same perceived quality compared to its predecessor, the High-Efficiency Video Coding (HEVC) standard. However, this increase in compression efficiency comes with an increase in computational complexity, particularly on the encoder side. VVenC integrates algorithmic optimizations for each coding tool in VVC and defines a set of five presets from faster to slower that provide Pareto-optimal tradeoffs between runtime and efficiency. Multithreading is employed to further reduce runtime while preserving most of the compression efficiency of each preset. With a hierarchical combination of pre-processing, picture-level and in-picture parallelization, VVenC achieves a 4× speedup for four threads. Further speedup using a higher number of threads depends on the video resolution and used encoder preset. For 16 threads, it ranges from 6 to 9 for high definition to 10-12 for ultra-high-definition video. Compared to previous work, the usage of temporal prediction for the adaptive loop filter reduces the associated coding efficiency loss from 0.4% to almost zero. A better scaling for higher numbers of threads can be achieved at the cost of a higher coding efficiency loss. In the presented framework, increased speedup by smaller block Coding Tree Unit (CTU) sizes, a combination of wavefront parallel processing and various tiles picture partitioning configurations is examined. Furthermore, results on a 20-core ARM-based Apple M1 computer indicate a better scaling for multithreading compared to ×86-based architectures. The analysis is complemented by profiling, which exhibits the overhead by idle threads and identifies the mode estimation as the main bottleneck of the presented framework.

Lensless Wavefront Parallel Processing of Vector Beams by Self‐Images of a Self‐Organized Q‐Plates Microarray

Wavefront shaping of structured light beams is attracting considerable attention in optics and related technologies. Due to the potential benefits, it is crucial to develop versatile methods for controlling the wavefront in a compact space with the capacity for high‐throughput parallel processing. Herein, a unique concept is introduced for converting the wavefront of multiple structured light beams, which are periodically packed in a microscopic area. An experimental demonstration is provided by using transmissive planar optical elements consisting of a microarray of geometric phase based on nematic liquid crystals. The periodic microstructure is fabricated by the directed self‐organization of topological defects present in a nanoscale relief obtained by area‐selective surface modification. The conversion of multiple wavefronts is realized by using the Talbot self‐imaging effect of the microarray. The integrated geometric phase microarray has potential applications in vortex beam emitters to harness structured light, such as for optical manipulation and lithography.

High-efficiency parallelism solution for a Multiview High-Efficiency Video Coding decoder

Thus far, Multiview High-Efficiency Video Coding (MV-HEVC) can only use a central processing unit (CPU) to perform decompression on a personal computer (PC) or workstation. Because MV-HEVC is much more complex than High-Efficiency Video Coding (HEVC), decompressors need higher parallelism to decompress in real time. Therefore, this study presents a parallel method based on MV-HEVC. Inter-view complete parallelism is realized according to the dependent relationship between other MV-HEVC views, and a search range is not required to avoid the data dependence between frames. Based on the dependencies of each task in MV-HEVC, an advanced wavefront parallel processing method is proposed to achieve higher intra-frame parallelism. The parallel structure of the proposed method is compatible with that of the single-instruction multiple-data acceleration method. The results showed that the proposed method can decompress MV-HEVC with 20 threads in real time for 1088p video with three views.

Scalable Wavefront Parallel Streaming Deblocking Filter Hardware for HEVC Decoder

The proposed work aims to design a Wavefront Parallel Processing (WPP) based streaming Deblocking Filter (DBF) architecture for High-Efficiency Video Coding (HEVC). This architecture supports scalable pipeline stages such as 1, 2, 4 and 8 with Coding Unit (CU) sizes, i.e., $8\times 8$ , $16\times 16$ , $32\times 32$ and $64\times 64$ Largest Coding Unit (LCU) processing respectively. Based on the requirements of speed and area of a consumer electronic application, one of the aforementioned sizes of CU or LCU is selected. The hardware uses an intelligent Memory Organization (MO) based on WPP technique with restructured CUs/LCU size without having any neighboring block dependencies. The proposed designs are implemented on an Application-Specific Integrated Circuit (ASIC) using 180-nm technology and Field Programmable Gate Array (FPGA). Experimental results show that the $32\times 32$ and $64\times 64$ block processing hardware decreases the processing cycles (128, 96) with the gate count of 286.47K and 744.13K respectively. Similarly, CUs $8\times 8$ , $16\times 16$ consume 512 and 176 processing clock cycles with an equivalent gate count of 90.72K and 194.97K respectively. The performance of proposed hardware compared with the previous works in terms of area and speed. The results show that the proposed hardware can process 4K Ultra High Definition (UHD) video frames at the rate of 50 fps at 300 MHz.

A Multi-Grained Parallel Solution for HEVC Encoding on Heterogeneous Platforms

To improve the parallel processing capability of video coding, the emerging high efficiency video coding (HEVC) standard introduces two parallel techniques, i.e., Wavefront Parallel Processing (WPP) and Tiles , to make it much more parallel-friendly than its predecessors. However, these two techniques are designed to explore coarse-grained parallelism in HEVC encoding on multicore Central Processing Unit (CPU) platforms. As the computing architecture undergoes a trend toward heterogeneity in the last decade, multi-grained parallel computing methods can be designed to accelerate HEVC encoding on heterogeneous systems. In this paper, a multi-grained parallel solution (MPS) is proposed to optimize HEVC encoding on a typical heterogeneous platform. A massively parallel motion estimation algorithm is employed by MPS to parallelize part of HEVC encoding on Graphic Processing Unit (GPU). Meanwhile, several other HEVC encoding modules are accelerated on CPU through the cooperation of WPP and an adaptive parallel mode decision algorithm. The parallelism between CPU and GPU is well designed and implemented to guarantee an efficient concurrent execution of HEVC encoding on multi-grained parallel levels. The effectiveness of the proposed MPS for HEVC encoding is verified on a number of experiments.

A Collaborative Scheduling-Based Parallel Solution for HEVC Encoding on Multicore Platforms

In order to meet the high computational demand to achieve superior coding efficiency and to explore the parallelism of parallel processing architectures, the emerging high efficiency video coding (HEVC) standard has been designed to be more parallelizable than previous video coding standards. However, it is still desirable to design an efficient parallel HEVC encoder to fully exploit the parallelism of the increasingly powerful multicore platforms, especially when considering the amount of parallelism, the scalability of parallelization, and the coding efficiency. In this work, a performance model of HEVC encoding is first introduced to investigate the speedup and the limitations of the technique of wavefront parallel processing (WPP) under various conditions. Then, a collaborative scheduling-based parallel solution (CSPS) for HEVC encoding is proposed, which includes adaptive parallel mode decision, asynchronous frame-level pixel interpolation, and multigrained task scheduling. The goal of the proposed CSPS is to defeat the disadvantages of WPP and further improve the parallelization of HEVC encoding on multicore platforms. Extensive experimental results demonstrate the efficiency of the proposed CSPS for parallelizing HEVC encoding as the computing resources of multicore architectures can be fully utilized.

Contents-aware partitioning algorithm for parallel high efficiency video coding

We introduce a new parallelization method for high-efficiency video coding (HEVC), which resolves the shortcomings of the existing tile-based parallel processing method. The parallel HEVC performs encoding by dividing a frame into numerous parallel units. This decreases the compression efficiency compared with sequential HEVC, because it artificially breaks the data correlation within a frame, which is called the parallelization overhead. The traditional parallel techniques such as Tiles and wavefront parallel processing (WPP) inherently introduce a high parallelization overhead because they simply divide a frame statically without considering the contents of the frame. The proposed new parallel encoding scheme resolves such problems by partitioning a frame based on the meaningful contents. In order to analyze the correlations within a frame and define the contents, the features within a frame are first extracted and clustered. In the feature clustering algorithm, two factors are considered to balance the workload between parallel units: (1) the number of features in each cluster and (2) the number of coding tree units (CTU) occupied by each cluster. The frame is partitioned based on the result of clustering, and the partitions are encoded in parallel. The proposed scheme achieves a bit-saving of up to 7.21%, with an average of 3.71%, along with an average time-saving of 20.50% compared to the Tiles technique.

Multi-level complexity reduction for HEVC multiview coding

Standardized in 2014, multiview extension of high efficiency video coding (MV-HEVC) offers significantly better compression performance of up to 50% for multiview and 3D videos compared to multiple independent single view HEVC coding. However, the extreme high computational complexity of MV-HEVC demands significant optimization of the encoder. In this work, we propose a series of optimization techniques at various levels of abstraction: non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) for prediction units, fractional and bidirectional ME/DE, quantization parameter-based early termination of coding tree unit (CTU), and optimized resource-scheduled wave front parallel processing for CTU. When evaluated over three views for all available official multiview video coding test sequences, proposed optimization outperforms the anchor encoder by average factor of 5.4 at the cost of 4.4% bitrate (DBR) increase at no loss in PSNR, or alternatively a PSNR degradation of 0.12 dB at no change to the DBR.

A Parallel HEVC Intra Prediction Algorithm for Heterogeneous CPU+GPU Platforms

The high efficiency video coding standard provides excellent coding performance but is also very complex. Especially, the intra mode decision is very time-consuming due to the large number of available prediction modes and the flexible block partitioning scheme. In this paper, a highly parallel intra prediction algorithm for heterogeneous CPU+graphics processing unit (GPU) platforms is proposed, which accelerates the encoder dramatically. It is targeted toward high-quality high definition (HD) and ultra HD applications and utilizes prediction based on original samples (POSs), where the reference samples are generated from original pixels. This makes it possible to perform intramode prediction for all prediction blocks of a video frame concurrently. In addition, parallel-friendly cost functions are proposed which enable parallel rate distortion optimization with no synchronization overhead. A detailed statistical analysis of both POS and the proposed GPU intramethod is provided and the coding performance of the presented prototype is evaluated based on a large amount of experimental data. It is shown that the complexity of the intramode selection on the CPU is reduced by up to 78.03%. This translates to significant encoding time reductions of up to 64.52% for a single-threaded encoder and up to 94.82% in combination with wavefront parallel processing. In high bitrate ranges, average rate increases of only 2.11%–4.26% and 0.80%–2.34% are observed for the proposed high-speed and high-quality configurations, respectively. Furthermore, GPU intra is shown to be extremely efficient in lossless coding scenarios, where up to 53.37% time is saved with an average bitrate increase of only 0.55% among all test cases.

A Novel Wavefront-Based High Parallel Solution for HEVC Encoding

With a lot of enhanced coding tools introduced, High Efficiency Video Coding (HEVC) achieves significant improvement in coding efficiency at the cost of increased computational complexity. To efficiently reduce the encoding time of HEVC, a wavefront-based high parallel (WHP) solution integrating novel data-level and task-level methods is proposed in this paper. On data level, optimal single-instruction-multiple-data algorithms are designed for the enhanced coding tools, i.e., replacing the multiplication in motion compensation by add and shift operations with reduced instruction cycles, removing the transpose in transform via realignment of coefficients, and minimizing the memory access in sum of absolute difference/sum of squared differences calculation by fully reusing the registers. On task level, a novel inter-frame wavefront (IFW) method is developed by effectively decreasing the dependence of wavefront parallel processing (WPP). In addition, a coding tree block level parallelism analysis method is presented to prove the superior of IFW method compared with other HEVC representative parallel methods. Besides, a three-level thread management scheme is proposed to best exploit the parallelism of IFW method and achieve corresponding encoding speedup. Extensive experimental results show that, the overall WHP solution can bring up to $57.65\times $ , $65.55\times $ , and $88.17\times $ speedup for HEVC encoding of Wide Video Graphics Array, 720p and 1080p standard test sequences, while maintaining the same coding performance as with WPP. The proposed solution is also applied in several leading video companies in China, providing HEVC video service for more than 1.3 million users everyday.

Efficient Software H.264/AVC to HEVC Transcoding on Distributed Multicore Processors

The latest High Efficiency Video Coding (HEVC) standard achieves a significant compression efficiency improvement over the H.264/Advanced Video Coding (AVC) standard, but with a much higher computational complexity. In this paper, we propose a novel framework for software-based H.264/AVC to HEVC transcoding, integrated with tools such as wavefront parallel processing that are useful for achieving higher levels of parallelism on multicore processors and distributed systems. By utilizing information extracted from the input H.264/AVC bitstream, the transcoding process can be greatly accelerated with a visual quality loss that is modest for many applications. Based on the HEVC HM 14.0 reference software and using standard HEVC test bitstreams, the proposed transcoder can achieve up to $60\times $ speedup on a Quad Core 8-thread server over decoding–re-encoding based on FFMPEG and the HM software with a BD-rate loss of 15%–20%. By implementing a group of picture-level task distribution on a distributed system with nine processing units, the proposed software transcoder can achieve a speed for transcoding 720 p at 30 Hz in real time.

Efficient In-Loop Filtering Across Tile Boundaries for Multi-Core HEVC Hardware Decoders With 4 K/8 K-UHD Video Applications

HEVC is a next generation video coding standard designed with modern coding techniques to be especially efficient for coding high-resolution video such as 4 K/8 K-ultra high- definition (UHD) video. Among the advanced coding tools of HEVC, tiles and wavefront parallel processing (WPP) have been newly adopted for parallel processing of such high-resolution (4 K/8 K-UHD) video. To realize UHD video services over portable devices with limited battery power, it is essential to implement multi-core-based and dedicated HEVC hardware decoders that support the tile- and wavefront-based parallel processing. By doing so, each frame is divided into a multiple number of picture partitions which can then be processed by multiple hardware decoder cores in parallel. However, in-loop filtering (ILF) at tile boundaries cannot be easily parallelized by a multi-core HEVC hardware decoder because of the data dependency between samples in different tiles. In this paper, an efficient control method for ILF across tile boundaries is proposed for multi-core HEVC hardware decoders. The proposed method does not require additional in-loop filters for ILF across the tile boundaries and it allows a decoder core to continue to process the next coding tree unit (CTU) without waiting for other decoders until they finish their ILF processing for the neighboring CTUs in other tiles. From experiments, we show the effectiveness of our ILF control method via a quad-core HEVC decoder for 4 K-UHD video implemented on a prototyping FPGA board.

A Deeply Pipelined CABAC Decoder for HEVC Supporting Level 6.2 High-Tier Applications

High Efficiency Video Coding (HEVC) is the latest video coding standard that specifies video resolutions up to 8K ultra-high definition (UHD) at 120 frames/s to support the next decade of video applications. This results in high-throughput requirements for the context-adaptive binary arithmetic coding (CABAC) entropy decoder, which was already a well-known bottleneck in H.264/AVC. To address the throughput challenges, several modifications were made to CABAC during the standardization of HEVC. This paper leverages these improvements in the design of a high-throughput HEVC CABAC decoder. It also supports the high-level parallel processing tools introduced by HEVC, including tile and wavefront parallel processing. The proposed design uses a deeply pipelined architecture to achieve a high clock rate. Additional techniques such as the state prefetch logic, latched-based context memory, and separate finite state machines are applied to minimize stall cycles, while multibypass-bin decoding is used to further increase the throughput. The design is implemented in an International Business Machines 45-nm silicon on insulator process. After place and route, its operating frequency reaches 1.6 GHz. The corresponding throughputs achieve up to 1696 and 2314 Mbin/s under common and theoretical worst-case test conditions, respectively. The results show that the design is sufficient to decode in real-time high-tier video bitstreams at level 6.2 (8K UHD at 120 frames/s), or main-tier bitstreams at level 5.1 (4K UHD at 60 frames/s) for applications requiring subframe latency, such as video conferencing.

Performance Analysis of HEVC Parallelization Methods for High-Resolution Videos

Several parallelization methods that can be applied to High Efficiency Video Coding (HEVC) decoders are evaluated. The market requirements of high-resolution videos, such as Full HD and UHD, have been increasing. To satisfy the market requirements, several parallelization methods for HEVC decoders have been studied. Understanding these parallelization methods and objective comparisons of these methods are crucial to the real-time decoding of high-resolution videos. This paper introduces the parallelization methods that can be used in HEVC decoders and evaluates the parallelization methods comparatively. The experimental results show that the average speed-up factors of tile-level parallelism, wavefront parallel processing (WPP), frame-level parallelism, and 2D-wavefront parallelism are observed up to 4.59, 4.00, 2.20, and 3.16, respectively.

Accelerating HEVC using heterogeneous platforms

The high efficiency video coding (HEVC) standard achieves double compression efficiency compared with H.264/advanced video coding at the cost of huge computational complexity. Parallelizing HEVC encoding is an efficient way of fulfilling this computational requirement. The parallelization algorithms considered in HEVC, such as Tiles or wavefront parallel processing (WPP), rely on creating picture partitions that can be processed concurrently in a multi-core architecture. However, this paper focuses on the design of a heterogeneous parallel architecture composed of a graphic processing unit (GPU) plus a multi-core central processing unit (CPU) to take advantage of these techniques. Experimental results indicate that our approach outperforms WPP in terms of speed-up and reduces the delay introduced by alternative techniques such as the group of pictures-based processing pattern. Moreover, the proposed algorithms obtain speed-up values of over $$4 \times $$4× on an Intel quad-core CPU and an NVIDIA GPU with negligible quality losses.

Parallel HEVC Decoding on Multi- and Many-core Architectures

The Joint Collaborative Team on Video Decoding is developing a new standard named High Efficiency Video Coding (HEVC) that aims at reducing the bitrate of H.264/AVC by another 50 %. In order to fulfill the computational demands of the new standard, in particular for high resolutions and at low power budgets, exploiting parallelism is no longer an option but a requirement. Therefore, HEVC includes several coding tools that allows to divide each picture into several partitions that can be processed in parallel, without degrading the quality nor the bitrate. In this paper we adapt one of these approaches, the Wavefront Parallel Processing (WPP) coding, and show how it can be implemented on multi- and many-core processors. Our approach, named Overlapped Wavefront (OWF), processes several partitions as well as several pictures in parallel. This has the advantage that the amount of (thread-level) parallelism stays constant during execution. In addition, performance and power results are provided for three platforms: a server Intel CPU with 8 cores, a laptop Intel CPU with 4 cores, and a TILE-Gx36 with 36 cores from Tilera. The results show that our parallel HEVC decoder is capable of achieving an average frame rate of 116 fps for 4k resolution on a standard multicore CPU. The results also demonstrate that exploiting more parallelism by increasing the number of cores can improve the energy efficiency measured in terms of Joules per frame substantially.

Parallel Scalability and Efficiency of HEVC Parallelization Approaches

Unlike H.264/advanced video coding, where parallelism was an afterthought, High Efficiency Video Coding currently contains several proposals aimed at making it more parallel-friendly. A performance comparison of the different proposals, however, has not yet been performed. In this paper, we will fill this gap by presenting efficient implementations of the most promising parallelization proposals, namely tiles and wavefront parallel processing (WPP). In addition, we present a novel approach called overlapped wavefront (OWF), which achieves higher performance and efficiency than tiles and WPP. Experiments conducted on a 12-core system running at 3.33 GHz show that our implementations achieve average speedups, for 4k sequences, of 8.7, 9.3, and 10.7 for WPP, tiles, and OWF, respectively.