All-zero Block Research Articles

Latitude-Redundancy-Aware All-Zero Block Detection for Fast 360-Degree Video Coding.

The sphere-to-plane projection of 360-degree video introduces substantial stretched redundant data, which is discarded when reprojected to the 3D sphere for display. Consequently, encoding and transmitting such redundant data is unnecessary. Highly redundant blocks can be referred to as all-zero blocks (AZBs). Detecting these AZBs in advance can reduce computational and transmission resource consumption. However, this cannot be achieved by existing AZB detection techniques due to the unawareness of the stretching redundancy. In this paper, we first derive a latitude-adaptive redundancy detection (LARD) approach to adaptively detect coefficients carrying redundancy in transformed blocks by modeling the dependency between valid frequency range and the stretching degree based on spectrum analysis. Utilizing LARD, a latitude-redundancy-aware AZB detection scheme tailored for fast 360-degree video coding (LRAS) is proposed to accelerate the encoding process. LRAS consists of three sequential stages: latitude-adaptive AZB (L-AZB) detection, latitude-adaptive genuine-AZB (LG-AZB) detection and latitude-adaptive pseudo-AZB (LP-AZB) detection. Specifically, L-AZB refers to the AZB introduced by projection. LARD is used to detect L-AZB directly. LG-AZB refers to the AZB after hard-decision quantization and zeroing redundant coefficients. A novel latitude-adaptive sum of absolute difference estimation model is built to derive the threshold for LG-AZB detection. LP-AZB refers to the AZB in terms of rate-distortion optimization considering redundancy. A latitude-adaptive rate-distortion model is established for LP-AZB detection. Experimental results show that LRAS can achieve an average total encoding time reduction of 25.85% and 20.38% under low-delay and random access configurations compared to the original HEVC encoder, with only 0.16% and 0.13% BDBR increases and 0.01dB BDPSNR loss, respectively. The transform and quantization time savings are 60.13% and 59.94% on average.

A Probability-Based Zero-Block Early Termination Algorithm for QSHVC

To seamlessly adapt to time-varying network bandwidths, the Quality Scalable High-Efficiency Video Coding (QSHVC) is developed. However, its coding process is overly complex, and this seriously limits its wide applications in real-time environments. Therefore, it is of great significance to study fast coding algorithms for QSHVC. In this paper, we propose a novel probability-based zero-block early termination algorithm for QSHVC. First, we observed that the generated residual coefficients follow the Laplace distribution if a CU is accurately predicted. According to this observation, we derive the sum of squared differences based All-Zero Block (AZB) decision condition. Second, we develop the Hadamard Transform (HT)-based zero-valued quantized coefficient decision condition to obtain zero-valued quantized coefficients and the corresponding Partial-Zero Block (PZB). Third, the probability of each coding mode and coding depth being chosen as the best ones are combined with both AZBs and PZBs to derive the probability-based early termination condition. The experimental results show that the proposed algorithm can improve the average coding speed by 79.85% with a 0.34% decrease in BDBR.

DFS: Joint data formatting and sparsification for efficient communication in Distributed Machine Learning

Efficient communication is crucial to Distributed Machine Learning (DML). In this work, we propose an approach jointing Data Formatting and Sparsification (DFS) to optimize the communication in DML systems based on the parameter server framework. By doing so, we can reduce the time to transmit (aggregated) gradients between the parameter server and workers, and consequently the time to complete training jobs. More specifically, in DFS, every worker first tries to derive as many blocks with all-zero gradients as possible via sparsification, and transmits gradients block by block in a streaming fashion. By skipping blocks with all-zero gradients, we can reduce the communication cost for gradient transmission. Different from previous works on optimizing the communication in DML systems, DFS has three distinct features: (i). it dynamically determines the gradient block size; (ii). it takes into consideration both the data transfer from workers to the parameter server and that from the parameter server to workers; and (iii). it jointly optimizes the data formatting and sparsification. In other words, it performs sparsification in the way that helps form more (or larger) all-zero blocks and save more communication cost. By implementing DFS on a real testbed, we find that it can reduce the time to train a ResNet-18 model by 74.12%. Through extensive simulations, we demonstrate that DFS outperforms the state-of-the-art technique, i.e., OmniReduce (Fei et al., 2021), by up to 87.17% in terms of reducing communication cost in DML systems.

Probability-Based Fast Intra Prediction Algorithm for Spatial SHVC

Due to multi-layer encoding and Inter-layer prediction, Scalable High Efficiency Video Coding (SHVC) has extremely high coding complexity. It is very crucial to improve its coding speed so as to promote widespread and cost-effective SHVC applications. In this paper, we propose a new probability-based fast Intra prediction algorithm for spatial SHVC. More specifically, first, we integrate depth probabilities with textural based all-zero blocks and all-nonzero blocks through Lagrange Interpolation Polynomial (LIP) to derive thresholds to early skip unlikely depths and early terminate depth selection. Second, to early skip Intra mode prediction, we combine mode probabilities with Jarque-Bera test through LIP to test the residual coefficients of Inter-layer reference (ILR) mode. Third, we adopt the difference of Hadamard Costs (HCs) of horizontal, vertical and diagonal directional modes to predict candidate directional modes (DMs), and then exploit both the percentages of gradient amplitudes and HCs for DM early termination. Experimental results demonstrate that the proposed algorithm can achieve a speed up gain of more than 61% with 0.03% decrease in Bjøntegaard Delta Bit Rate on average.

Multi-stage all-zero block detection for HEVC coding using machine learning

Compared with deadzone hard-decision quantization (HDQ), rate-distortion optimized quantization (RDOQ) in HEVC brings non-negligible coding gain, however consumes considerable computations caused by exhaustive search over multiple candidates to determine optimal output level. Benefiting from efficient prediction in HEVC, transform blocks are frequently quantized to all zero, especially in small-size blocks. It is worthwhile to detect all zero block (AZB) for transform blocks to bypass subsequent computation-intensive RDOQ. Traditional thresholding based AZB detection algorithms are well-suited for deadzone quantized blocks, however miss partial optimal results in RDOQ and suffer from more or less accuracy degradation in RDOQ. This paper proposes a novel multi-stage AZB detection algorithm for RDOQ blocks with good tradeoff between complexity and accuracy. At the first stage, genuine all zero blocks (G_AZB) which are quantized to all zero both in HDQ and RDOQ are prejudged by comparison with conservative threshold determined by mathematical derivation for deadzone HDQ. At the second stage, an adaptive threshold model is built using adaptive deadzone offset by simulating the behavior patterns existing in RDOQ, aiming to further detect the pseudo AZB (P_AZB) which are quantized to all zero in RDOQ however not all zero in HDQ. At the final stage, machine learning based detection is proposed to classify the remaining “cunning” all zero blocks using eight distinguished RDO-related features, by which subtle working mechanism in RDOQ is leveraged. The experimental results demonstrate that the proposed algorithm achieves up to 7.471% total coding computation saving with 0.064% BD-RATE increment compared with RDOQ on average. Moreover, the average FNR and FPR detection accuracies are 6.3% and 6.5% respectively.

Performance engineering for HEVC transform and quantization kernel on GPUs

Continuous growth of video traffic and video services, especially in the field of high resolution and high-quality video content, places heavy demands on video coding and its implementations. High Efficiency Video Coding (HEVC) standard doubles the compression efficiency of its predecessor H.264/AVC at the cost of high computational complexity. To address those computing issues high-performance video processing takes advantage of heterogeneous multiprocessor platforms. In this paper, we present a highly performance-optimized HEVC transform and quantization kernel with all-zero-block (AZB) identification designed for execution on a Graphics Processor Unit (GPU). Performance optimization strategy involved all three aspects of parallel design, exposing as much of the application’s intrinsic parallelism as possible, exploitation of high throughput memory and efficient instruction usage. It combines efficient mapping of transform blocks to thread-blocks and efficient vectorized access patterns to shared memory for all transform sizes supported in the standard. Two different GPUs of the same architecture were used to evaluate proposed implementation. Achieved processing times are 6.03 and 23.94 ms for DCI 4K and 8K Full Format, respectively. Speedup factors compared to CPU, cuBLAS and AVX2 implementations are up to 80, 19 and 4 times respectively. Proposed implementation outperforms previous work 1.22 times.

Low Complexity Quantization in High Efficiency Video Coding

The rate-distortion optimized quantization (RDOQ) provides an excellent trade-off between rate and distortion in High Efficiency Video Coding (HEVC), leading to inspiring improvement in terms of rate-distortion performance. However, its heavy use imposes high complexity on the encoder in real-world video compression applications. In this paper, we provide a comprehensive review on low complexity quantization techniques in HEVC, including both fast RDOQ and all-zero block detection. In particular, the fast RDOQ relies on rate and distortion models for rate-distortion cost estimation, such that the most appropriate quantized coefficient is selected in a low complexity way. All-zero block detection infers the all-zero block by skipping transform and quantization, in an effort to further reduce the complexity. The relationship between the two techniques is also discussed, and moreover, we also envision the future design of low complexity quantization in the upcoming Versatile Video Coding (VVC) standard.

Hybrid All Zero Soft Quantized Block Detection for HEVC.

Transform and quantization account for a considerable amount of computation time in video encoding process. However, there are a large number of discrete cosine transform coefficients which are finally quantized into zeros. In essence, blocks with all zero quantized coefficients do not transmit any information, but still occupy substantial unnecessary computational resources. As such, detecting all-zero block (AZB) before transform and quantization has been recognized to be an efficient approach to speed up the encoding process. Instead of considering the hard-decision quantization (HDQ) only, in this paper, we incorporate the properties of soft-decision quantization into the AZB detection. In particular, we categorize the AZB blocks into genuine AZBs (G-AZB) and pseudo AZBs (P-AZBs) to distinguish their origins. For G-AZBs directly generated from HDQ, the sum of absolute transformed difference-based approach is adopted for early termination. Regarding the classification of P-AZBs which are generated in the sense of rate-distortion optimization, the rate-distortion models established based on transform coefficients together with the adaptive searching of the maximum transform coefficient are jointly employed for the discrimination. Experimental results show that our algorithm can achieve up to 24.16% transform and quantization time-savings with less than 0.06% RD performance loss. The total encoder time saving is about 5.18% on average with the maximum value up to 9.12%. Moreover, the detection accuracy of larger TU sizes, such as and can reach to 95% on average.

All zero block detection for HEVC based on the quantization level of the maximum transform coefficient

Transform/quantization (T/Q) is one of the high complex modules in high efficiency video coding (HEVC). In this paper, all zero block (AZB) detection algorithm for HEVC is proposed to reduce the complexity of T/Q. The proposed AZB detection algorithm is based on the quantization level of the maximum transform coefficient, which is suitable for both uniform quantizer (UQ) and rate-distortion optimized quantizer (RDOQ). Experimental results show that 46% complexity of T/Q for UQ and 42% for RDOQ can be reduced with negligible loss of video quality and compression efficiency, outperforming the state-of-the-art methods.

An efficient all-zero block detection algorithm for high efficiency video coding with RDOQ

Rate distortion optimized HEVC video coding is challenged by the extremely high complexity caused by hierarchical prediction structure and advanced coding tools. All-zero-block (AZB) detection is an efficient tool to decrease the complexity of mode decision in HEVC coders in the sense of rate distortion optimization (RDO). Traditional AZB detection schemes were usually designed by employing hard-decision quantization (HDQ), and the AZB detection thresholds were generally derived according to the individual block’s characteristic parameter such as SAD or SATD. However, soft-decision quantization (SDQ) is preferably employed in HEVC encoders for coding performance improvement, and the AZB detection thresholds are supposed to be determined according to the ensemble’s statistical characteristics in the sense of statistical inference. This paper proposes an adaptive AZB detection algorithm well-suited for SDQ, more specifically RD optimized quantization (RDOQ). Inspired by Bayesian inference, a more accurate coefficient-level zero-quantized threshold model in the RDO sense is proposed by fully simulating RDOQ to aid in the following AZB detection. In addition, a local parameter depicting the individual block’s characteristics regarding the inter-coefficient distribution is jointly combined with SATD to derive an adaptive threshold model for AZB detection. The experimental results demonstrate that the proposed algorithm detects 91.9% 4×4 AZB with smaller than 1.9% FAR and 8.1% FRR false alarm rate on average, at the cost of negligible rate-distortion performance loss. This work is well-suited for fast RD optimized HEVC coding.

Fast mode decision and early termination based on perceptual visual quality for HEVC encoders

This paper presents an early termination and fast mode decision for the HEVC encoder with a perceptual visual quality. For early termination of residual quad-tree, if the predicted errors for a transform block are not visible in terms of perceptual quality, the block is coded as the all-zero block and the transforms for the block and smaller blocks are skipped in RDO process. In addition, when a coding mode in the RDO-based mode decision stage is satisfactory in terms of visual quality, the coding mode is selected and the remaining coding modes requiring more bits are skipped. The proposed fast encoding algorithm is designed by pruning many coding modes when the coding errors with a coding mode having a small number of bits are not perceived by the human visual system. In addition, the proposed perceptual quality-based fast encoding algorithms can work with the existing objective quality-based fast decision algorithms. We found that the proposed algorithm can achieve 1.77 times acceleration of HM-16.8 with minimal perceptual visual degradation and objective quality distortion.

An All-Zero Block Detection Scheme for Low-Complexity HEVC Encoders

In this paper, an all-zero block detection scheme is proposed prior to DCT to reduce the encoding complexity for high efficiency video coding (HEVC). Since many coding blocks tend to have all zero coefficients after DCT and quantization, it is worthwhile to detect all-zero-quantized blocks for input residual blocks before DCT so that subsequent transform and quantization can be skipped. Unlike previous coding standards, HEVC adopts large transform sizes such as 16 × 16 and 8 × 8. It becomes more difficult to accurately detect all-zero blocks in HEVC because the large transform blocks contains more variety of content characteristics than smaller ones, thus making it ineffective the existing all-zero block (AZB) detection schemes for large transform blocks in HEVC. In this paper, a novel AZB detection scheme is proposed for the case that Hadamard transform is used as a distortion metric for RDO in HEVC. Statistical upper bounds to be all-zero blocks are derived using the relationship between Walsh Hadamard and DCT transform kernels. Then, a small number of quantized coefficients in a upper left corner of a transform block, which are obtained using the relations between Hadamard transform and DCT, are examined for AZB detection. For 32 × 32 blocks, DC coefficients of 8 × 8 sub-blocks are further examined for AZB detection. The experimental results demonstrate that the proposed scheme detects 87.79% of actual AZBs with 2.87% false alarm rate in average, outperforming the state-of-the-art method. Computational complexity to detect AZB is almost negligible compared to the conventional method.

Early DIRECT mode decision based on all‐zero block and rate distortion cost for multiview video coding

The exhaustive variable-block-size mode decision can efficiently remove the redundancies among the multiview videos, while it also leads to significant increase of computational complexity in the multiview video coding (MVC) encoder, and the high encoding complexity becomes a bottleneck for the MVC encoder to achieve real-time multimedia applications. To address this bottleneck, many fast mode decision methods have been proposed. However, most of them are only suitable for optimising the encoding complexity of the odd views of the MVC encoder. In this study, based on the property of the all-zero block and rate distortion (RD) cost of the DIRECT mode as well as the correlations between the current macroblock (MB) and its spatial–temporal nearby MBs, an early DIRECT mode decision method is proposed for reducing the encoding complexity of the MVC. Experimental results show that the proposed method achieves 48.25 and 55.64% on average encoding time saving for the even and odd views, respectively, whereas the RD performance degradation is quite acceptable. In summary, the proposed method efficiently reduces the encoding complexity for the MVC encoder.

High-efficient video compression for social multimedia distribution

To reduce data transmission bandwidth and storage space, the compression of original video is a prerequisite for social multimedia content uploading and distribution. The paper proposes an efficient early detection algorithm of all-zero blocks to eliminate redundant computations in H.264/AVC video encoding. First, for DCT-like integer transforms, a general sufficient condition of early detection of all-zero blocks is given based on the sum of absolute differences of the prediction error block, which is a direct derivation of Sousa's method. Then, by analyzing the dynamic ranges of different frequency components, we further refine the sufficient condition of early detection of all-zero blocks for DCT-like integer transforms and apply it to $$4\times 4$$4×4 and $$8\times 8$$8×8 DCT-like integer transforms in H.264/AVC. Experimental results show that the proposed algorithm for early detection of all-zero blocks outperforms Sousa's method in complexity reduction, where computations in transform, quantization, inverse quantization and inverse transform are significantly reduced. In addition, the proposed general sufficient condition of early detection of all-zero blocks for H.264/AVC can also be easily extended to other forms of DCT-like integer transforms. As a result, better audience experience and more efficient analysis can be provided for social multimedia applications.

Early MERGE Mode Decision Based on Motion Estimation and Hierarchical Depth Correlation for HEVC

The high efficiency video coding (HEVC) is the latest video coding standard, which adopts the quadtree structure based coding tree unit (CTU) to improve the coding efficiency. In the HEVC encoding process, the CTU is recursively split into the 8×8 size coding units (CUs) from the 64×64 size CU. Along with the increased number of the sizes of the CUs, the number of coding modes has been greatly increased, which results in high computational complexity in the HEVC encoder. In this paper, we propose an early MERGE mode decision algorithm to reduce the computational complexity of the HEVC encoder. Firstly, based on the all-zero block (AZB) and the motion estimation (ME) information of the INTER 2N×2N mode, an early MERGE mode decision is proposed for the root CUs (i.e., 64×64 size CUs). Then, an early MERGE mode decision is proposed for the children CUs (i.e., 32×32, 16×16, and 8×8 size CUs) by considering the mode selection correlation between the root CU and the children CUs. To maximize the computational complexity reduction, when the root CUs are encoded in the non-MERGE modes, the AZB and the ME information are also used for early termination of the children CUs. Experimental results demonstrate that compared to the state-of-the-art published method, the proposed algorithm can achieve about 35% encoding time on average saving while the rate distortion performance degradation is negligible.

All-zero block detection algorithm in H.264 based on radial basis function network

In this paper,a kind of algorithm for all-zero block detection based on Radial Basis Function(RBF) Neural Network(NN) was proposed to improve the accuracy of all-zero block detection algorithm.By analyzing the H.264 encoder features,six effective features were selected,including Sum of Absolute Difference(SAD),Sum of Absolute Transformed Difference(SATD),block type,Rate Distortion Optimization(RDO) cost,Quantization Parameter(QP) and the situation of reference block.Considering the SATD should be used in the Hadamard Transform(HT),to get the relationship of QP and RBF network width parameter through the least square method,the algorithm used two classifiers to separate all-zero blocks from non-all-zero blocks based on the encoding situation of the reference block.This algorithm could improve coding speed over 50% on average while keeping bit rate and video quality almost unchanged.The experimental results show that the proposed algorithm can improve all-zero block detection accuracy effectively and coding efficiency based on NN.

Zero‐quantised discrete cosine transform coefficients prediction technique for intra‐frame video encoding

One promising solution to reduce the computational complexity of discrete cosine transform (DCT) is to identify the redundant computations and to get rid of them. In this study, the authors present a new method to predict zero-quantised DCT coefficients for efficient implementation of intra-frame video encoding by identifying such redundant computations. Traditional methods use the Gaussian statistical model of residual pixels to predict all-zero or partial-zero blocks. The proposed method is based on two key ideas. At first, the bounds of DCT coefficients are derived from the intermediate signals of the Loeffler DCT algorithm instead of calculating the sum of absolute difference (SAD) of residual pixels. The sufficiency conditions are then suitably chosen to predict the zero-quantised coefficients to reduce the arithmetic complexity without degrading the video quality. Simulation results are found to validate the analytical model and show that the proposed prediction eliminates more redundant computations than the existing methods. Moreover, the authors have derived a pipelined VLSI architecture of the proposed prediction scheme which offers a saving of more than 63 and 91% of multiplications of the second stage of one-dimensional DCT for high and low bit-rate intra-video encoding, respectively.

CBCR: CBC MAC with rotating transformations

Message authentication codes (MACs) are widely used to protect data integrity and data origin authentication in communications. In this paper, we propose a new block-cipher-based MAC algorithm, CBCR, with provable security for arbitrary-length messages. CBCR adopts rotating operations in the end of cipher- block-chaining structure, and it appends fixed-length prefixes to all messages, which implies flexible usages in practice. For example, these fixed-length prefixes can be filled with some security parameters, message redun- dancies, etc. We also propose CBCR0, a special case of CBCR that sets its prefix to be an all-zero block. Compared with CMAC, a recommended standard by the US National Institute of Standards and Technology (NIST), CBCR0 enjoys all the advantages that CMAC has, but requires less memory; so, in practical appli- cations, CBCR0 behaves as well as CMAC does, and it is even more suitable for environments with limited memory resources, e.g. smart cards.

An All-Zero Block Mode Decision Algorithm for H.264/AVC Optimization

The H.264/AVC standard achieves significantly high coding efficiency if multiple block size Motion Estimation is adopted. However, the complexity of Motion Estimation and DCT is dramatically increased as a result. In previous work we propose an early mode decision algorithm to control the complexity, based on all-zero-blocks detection in 16×16 size. In this paper, we improve the algorithm. Firstly, we propose to detect all-zero blocks in 16×16, 8×8 and 4×4 sizes to simplify the course of mode decision. Secondly, we define the thresholds which are used to terminate motion estimation and mode decision in advance for these sizes. Last, we present the whole proposed algorithm. Experiments show that about 77% encoding time and 85% motion estimation time can be saved on average, which is better than state-of-the-art approaches.

Fast Mode Decision Using All-Zero Block Detection for Fidelity and Spatial Scalable Video Coding

In scalable video coding (SVC) as an extension of H.264/advanced video coding (AVC), a computationally expensive exhaustive mode decision is employed to select the best coding mode for each macroblock (MB). In order to reduce computational complexity, we propose a fast mode decision algorithm for SVC which uses an all-zero block (AZB) detection. Based on the empirical analysis of the inter-layer correlation of the AZB, the MB at the enhancement layer (EL) is predicted whether to be the AZB. Then, only predicted MBs are examined by the AZB detection algorithm. Since the mode decision can be terminated by detecting the AZB, we determine the processing order of the candidate modes at the EL in order to terminate the mode decision in the early stage. The proposed algorithm can be combined with other conventional fast mode decision methods in order to further reduce the computational complexity of those methods. Experimental results show that the proposed algorithm can significantly speed up the encoding process, especially for low bit-rate video sequences, without deteriorating the coding efficiency of SVC.