Sum Of Absolute Transformed Difference Research Articles

Research on VVC Intra-Frame Bit Allocation Scheme Based on Significance Detection

This research is based on an intra-frame rate control algorithm based on the Versatile Video Coding (VVC) standard, considering that there is the phenomenon of over-allocating the bitrate of the end coding tree units (CTUs) in the bit allocation process, while the front CTUs are not effectively compressed. Fusing a Canny-based edge detection algorithm, a color contrast-based saliency detection algorithm, a Sum of Absolute Transformed Differences (SATD) based CTU coding complexity measure, and a Partial Least Squares (PLS) regression model, this paper proposes a CTU-level bit allocation improvement scheme for intra-mode code rate control of the VVC standard. First, natural images are selected to produce a lightweight dataset. Second, different metrics are utilized to obtain the significance and complexity values of each coding unit, the relatively important coding units in the whole frame are selected, which are adjusted with different weights, and the optimal adjustment multiplicity is supplemented into the dataset. Finally, the PLS regression model was used to obtain regression equations to refine the weights for adjusting the bit allocation. The proposed bit allocation scheme improves the average rate control accuracy by 0.453%, Y-PSNR by 0.05 dB, BD-rate savings by 0.33%, and BD-PSNR by 0.03 dB compared to the VVC standard rate control algorithm.

Precise Encoding Complexity Control for Versatile Video Coding

Complexity reduction is a commonly used method to deal with complicated video coding standards, such as High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC). But the unstable performance under different video contents and Quantization Parameters (QPs) makes it difficult to precisely specify the target encoding time of every single sequence, which limits the practical use of the encoder. Inspired by rate control, in this paper, encoding time budget is regarded as a resource. We incorporate the hierarchical Group of Picture (GOP) encoding structure, the non-accelerated proportion, and block content variation within the frame, and design a top-down allocation and bottom-up feedback scheme, to achieve precise control of encoding complexity by controlling the maximum depth of QuadTree with nested Multi-type Tree (QTMT). In the scheme, Temporal ID (Tid)-based, Sum of Absolute Transformed Difference (SATD)-based methods and Linear (L) Model are designed to facilitate weighted allocation and feedback. The relationship between Planar Cost and encoding time is exploited as a Time-Cost (T-C) model, which guides the selection of Largest Coding Unit (LCU) encoding strategies in I-frames. For B-frames, we investigate a switching suppression assisted status-based method, to efficiently decide the encoding strategy of each LCU. Through the collaboration of the proposed technologies in the scheme, given any encoding time target achievable, encoding strategies will be automatically switched to help the actual encoding time gradually approach the target. To the best of our knowledge, this work is the first work on VVC-based encoding complexity control. The proposed scheme supports directly specifying the target encoding time or target Frame Per Second (FPS), and accurately realizing it within one pass. According to experimental results, under the target encoding time ratio of 80%, 60% and 40%, the average encoding time error is kept under 0.24%, 0.03% and 0.02% by our method, outperforming state-of-the-art works base on HEVC. These results prove the advantage of the proposed scheme.

Low complexity mode selection for H.266/VVC intra coding

The newest video compression standard, called Versatile Video Coding (H.266/VVC), outperforms its predecessor High Efficiency Video Coding (HEVC) by up to 50% in the matter of coding performance. The achievement results from the integrating of several sophisticated tools such as 32 extended angular modes for intra coding, advanced partitioning structure, and additional transform selection. However, these tools negatively deliver the computational cost of the VVC Test Model (VTM) in all-intra encoding configuration. The existing fast intra coding algorithms achieve a significant saving in computational time with a comparable rate distortion (RD) performance of VVC plus additional computational complexity. In this paper, we study the original strategy of VVC intra mode decision and analyze the correlation among the sum of absolute transformed differences (SATD) costs of the rough mode decision (RMD) and the RD costs of the rate distortion optimization (RDO) processes. Based on the useful correlations with an effective threshold formulation, we propose a fast intra prediction mode selection for the most time-consuming RDO process by selecting a small candidate set based on the SATD costs and calculating the time-consuming RD costs for only intra prediction modes of the small candidate set without a significant overhead. According to the experimental results, our proposed system can reduce up to 27.38% and on average 21.07% of the computation complexity of VTM version 5.0 with an insignificant quality degradation.

An efficient highly parallelized ReRAM-based architecture for motion estimation of HEVC

Motion estimation (ME) is a high efficiency video coding (HEVC) process for determining motion vectors that describe the blocks transformation direction from one adjacent frame to a future frame in a video sequence. ME is a memory and computation consuming process which accounts for more than 50% of the total running time of HEVC. To conquer the memory and computation challenges, this paper presents ReME, a highly paralleled processing-in-memory (PIM) architecture for the ME process based on resistive random access memory (ReRAM). In ReME, the space of ReRAM is mainly separated into storage engine and ME processing engine. The storage engine is used as conventional memory to store video frames and intermediate data, while the computation operations of ME are performed in ME processing engines. Each ME processing engine in ReME consists of Sum of Absolute Differences (SAD) modules, interpolation modules, and Sum of Absolute Transformed Difference (SATD) modules that transfer ME functions into ReRAM-based logic analog computation units. ReME further cooperates these basic computation units to perform ME processes in a highly parallel manner. Simulation results show that the proposed ReME accelerator significantly outperforms other implementations with time consuming and energy saving.

Exploring Optimized Hadamard Methods to Design Energy-Efficient SATD Architectures

State-of-the-art video coding tools are submitted to severe performance and energy consumption requirements resulting from high complexity of video standards and from limited energy budgets of portable mobile devices. While providing most of the compression gains, inter frame and intra frame prediction techniques are the most demanding steps, since they compare a huge number of blocks. In such a process, the similarity metric employed affects both the quality of compression and the calculation effort. In this paper we propose the use of Hadamardbased Sum of Absolute Transformed Differences (SATD), in replacement of the traditionally used Sum of Absolute Differences (SAD), as a means of improving the efficiency of video coding. To allow that we explore two Hadamard Transform methods to design efficient SATD architectures, one using the Fast Hadamard Transform (FHT) butterfly and another one using the so-called Transform-Exempted (TE) SATD algorithm. Those methods were combined with architectural decisions (full parallelism, full parallelism with pipelining or multi-cycling) to build a total of six Hadamard-based SATD architectures that were synthesized for a commercial 45nm standard cell library for two operating frequencies. The architectures were simulated with pixel block data to obtain realistic dynamic power and energy estimates. The TE-SATD architectures achieved the lowest energy results: down to 13.13 pJ/SATD in the case of parallel architecture with pipeline. However, considering also the area results when evaluating energy, the best results are given by both methods using multi-cycling (transpose buffer): nearly 20.75 pJ/SATD with up to 63.54% smaller area compared with fully parallel architectures.

Computationally Efficient Intra and Inter Mode Decision in H.264/AVC

Abstract The variable block size mode decision employed in H.264/AVC improves its coding efficiency, which also increases the computational complexity when rate distortion optimization (RDO) is used. To reduce the computational complexity of H.264/AVC a fast intra and inter mode decision technique is implemented. The fast intra mode decision is achieved by eliminating the less likely prediction modes from the rate distortion computation, for which a method based on directions of prediction and the Sum of Absolute Transformed Difference (SATD) is used for mode selection. The fast inter mode decision algorithm employs early skip mode detection, adaptive thresholds and the correlation of adjacent macroblocks to reduce the computational complexity. The experimental results show that the proposed method can reduce the coding time by 50.73% and thereby improve the real time performance, with slight video quality degradation of 0.13dB and bit-rate increment of 1.13%.

Progressive Spatial Recurrent Neural Network for Intra Prediction

Intra prediction is an important component of modern video codecs, which is able to efficiently squeeze out the spatial redundancy in video frames. With preceding pixels as the context, traditional intra prediction schemes generate linear predictions based on several predefined directions (i.e. modes) for blocks to be encoded. However, these modes are relatively simple and their predictions may fail when facing blocks with complex textures, which leads to additional bits encoding the residue. In this paper, we design a Progressive Spatial Recurrent Neural Network (PS-RNN) that learns to conduct intra prediction. Specifically, our PS-RNN consists of three spatial recurrent units and progressively generates predictions by passing information along from preceding contents to blocks to be encoded. To make our network generate predictions considering both distortion and bit-rate, we propose to use Sum of Absolute Transformed Difference (SATD) as the loss function to train PS-RNN since SATD is able to measure rate-distortion cost of encoding a residue block. Moreover, our method supports variable-block-size for intra prediction, which is more practical in real coding conditions. The proposed intra prediction scheme achieves on average 2.5% bit-rate reduction on variable-block-size settings under the same reconstruction quality compared with HEVC.

Energy-Efficient Hadamard-Based SATD Hardware Architectures Through Calculation Reuse

The Hadamard-based Sum of Absolute Transformed Differences (SATD) is a distortion metric that correlates better with other video encoding steps than the commonly used Sum of Absolute Differences (SAD) and thus potentially improves coding efficiency. However, due to the Hadamard Transform (HT), it demands a large share of total encoding time. Meanwhile, the widespread use of embedded encoders made computational complexity and energy efficiency two relevant issues, besides the usual trade-off between the rate and distortion. Hence, the adoption of SATD during video coding requires the design of energy-efficient hardware accelerators. In this paper, we show that the SATD improves coding efficiency by up to −2.99% in BD-Rate when used in motion estimation within the HEVC Model (HM). Subsequently, as the main contribution, we combined two properties featured by the SATD to devise a novel hardware architecture based on calculation reuse. These properties allow for rearranging the architecture components to reduce the hardware size, save operations, and thus, reduce energy. The synthesis and simulation results indicate that the proposed architecture can lead to area and energy savings of about 28% and 32%, respectively, compared to the state-of-the-art architecture based on reuse solely.

Early Termination of Intra Mode Decision Based on Most Probable Mode and SATD for H.264/AVC encoding

Intra coding in H.264/AVC can significantly improve the coding efficiency but at the cost of high computational complexity due to the use of rich prediction modes and rate-distortion optimization technique. In this paper, an early termination algorithm of intra mode decision is presented to address the complexity issue in 4×4 intra prediction. The proposed algorithm is motivated by two facts: the most probable mode defined based on the spatial similarity has a high possibility to be the best mode, and a good prediction usually has a small residual block which is measured by the sum of absolute transformed difference (SATD). The most probable mode and two modes with the smallest SATD values are investigated to select the candidate modes by using the proposed early termination rules. Experimental results show that the proposed algorithm effectively reduces the complexity of 4×4 intra prediction while maintaining almost the same coding performance on peak signal-to-noise ratio and bit rate compared with the full search algorithm.

Complexity Control in the HEVC Intracoding for Industrial Video Applications

A large number of industrial video applications are expected to work in real-time and power-constrained scenarios. For such applications, the coding complexity has a great impact on their performance. In this paper, we propose a complexity control method in the high-efficiency video coding intracoding to facilitate these video applications. The proposed method is performed on the coding tree unit (CTU) level, which consists of three steps, namely complexity estimation, complexity allocation, and prediction unit (PU) adaption. In the first step, a complexity estimation model is proposed to estimate the coding complexity of each CTU based on the sum of absolute transformed difference. Then, the complexity budget is allocated to each CTU proportionally to its estimated coding complexity. In PU adaption, only a subset of PU sizes are selected for the CTU according to its allocated complexity and prediction performance. A feedback-based error elimination scheme removes the complexity error during the encoding process. Experimental results show that the proposed method is able to adjust the complexity ratio from 100% to 20%. Meanwhile, the rate-distortion performance and complexity control accuracy of the proposed method are superior to those of the state-of-the-art methods.

Algorithm optimization and hardware implementation for Merge mode in HEVC

Merge mode is a new tool for improving inter-frame coding efficiency in high-efficiency video coding. This tool can save the bitrate for the motion vector by sharing this vector with neighboring blocks. Merge is a process that selects a candidate motion vector by calculating the cost of rate-distortion. However, this process requires a large number of complex computations and memory access, thereby resulting in the low efficiency of hardware implementation. This paper proposes a new Merge candidate decision scheme that determines the most favorable Merge candidate from a full list of candidates by comparing the sum of absolute transformed difference with the weighted header bit instead of performing a complex calculation for sum of squared difference and entropy coding process in HM16.7. The simulation results show that the performance of the proposed algorithm is close to that of HM16.7 and increases the BD-rate only by 0.22–1.21%. The multilevel pipelines architecture is also exploited in the hardware design. The weighted header bit operation is performed by using the look-up table, which reduces both the complexity and encoding clock cycle. The designed system is implemented with a register transfer level code. The synthesis results from the Design Compiler show that compared with other architecture, the proposed architecture offers great advantages in resource utilization and can process 1920 × 1080 at 353 frame/s for P-slices with a clock frequency of 1057 MHz and logic gate count of 285.2 K.

A fast intra mode decision algorithm combining neighboring information for H.264/AVC high profile

Intra coding in H.264/AVC can significantly improve the coding efficiency but at the cost of high computational complexity due to the use of rich prediction modes and rate-distortion optimization technique. To reduce the complexity, this paper proposes a fast mode decision algorithm for intra prediction in H.264/AVC high profile. The best prediction mode not only relies on the content of current block but also depends on the neighboring blocks because of the spatial continuity. Based on the reconstructed neighboring pixels, different prediction mode leads to different residual block which is the difference between the original block and predicted block. Since small residual block is advantageous to get good coding including the small distortion and few bit numbers, the feature from the residual block will be used to help do mode decision. Besides the neighboring pixels, the best modes of encoded neighboring blocks are also used to select the candidate modes. The proposed algorithm uses the sum of absolute transformed difference (SATD) to measure the residual block and uses the most probable mode to indicate the influence of the prediction modes of the neighboring blocks. Based on the statistic on mode decision accuracy, the most probable mode and the modes with the smallest residual block are considered as the candidate modes. Meanwhile, the early termination rules which can directly select only one mode are given in the proposed algorithm. Experimental results show that the proposed algorithm effectively reduces the complexity of intra prediction with slight coding performance degradation compared with the full search algorithm.

Fast Algorithm for Intra Prediction of HEVC Using Adaptive Decision Trees

High Efficiency Video Coding (HEVC) Standard, as the latest coding standard, introduces satisfying compression structures with respect to its predecessor Advanced Video Coding (H.264/AVC). The new coding standard can offer improved encoding performance compared with H.264/AVC. However, it also leads to enormous computational complexity that makes it considerably difficult to be implemented in real time application. In this paper, based on machine learning, a fast partitioning method is proposed, which can search for the best splitting structures for Intra-Prediction. In view of the video texture characteristics, we choose the entropy of Gray-Scale Difference Statistics (GDS) and the minimum of Sum of Absolute Transformed Difference (SATD) as two important features, which can make a balance between the computation complexity and classification performance. According to the selected features, adaptive decision trees can be built for the Coding Units (CU) with different size by offline training. Furthermore, by this way, the partition of CUs can be resolved as a binary classification problem. Experimental results have shown that the proposed algorithm can save over 34% encoding time on average, with a negligible Bjontegaard Delta (BD)-rate increase.

An AR based fast mode decision for H.265/HEVC intra coding

As a critical technique to raise coding efficiency in H.265/HEVC (H.265/High Efficiency Video Coding), the number of prediction modes for intra coding is much larger than that in H.264/AVC. However, the large number of prediction modes also leads to high complexity. In this paper, an AR (Auto Regression) based fast mode decision method for H.265/HEVC intra coding is proposed, including an AR based fast mode selection and an MPM (Most Probable Mode) joint fast mode selection method. Considering the correlation between SATD (Sum of Absolute Transformed Differences) cost and RD (Rate-Distortion) cost, we establish a selective function controlled by an adaptive parameter which is acquired by an AR model. According to the distribution of the SATD cost, the selective function can decide which modes can be discarded and which modes can be used for the FMD (Fine Mode Decision). In addition, in order to further reduce the number of candidate modes for FMD, an MPM joint fast mode selection method is presented based on the relationship between MPMs and candidate modes. Experimental results show that just in PU layer, the proposed AR based fast intra mode decision method can save encoding time by 32 % with negligible coding loss in all-intra main configuration compared to the intra mode decision of HM13.0.

Immature green citrus detection based on colour feature and sum of absolute transformed difference (SATD) using colour images in the citrus grove

Recognition and detection of green immature citrus fruit more accurately and efficiently in groves under natural illumination conditions provides a promising benefit for growers to plan application of nutrients during the fruit maturing stages and estimate their yield and profit prior to the harvesting period. The goal of this study was to develop a robust and fast algorithm to detect and count immature green citrus fruit in individual trees from colour images acquired with different fruit sizes and under various illumination conditions. Adaptive Red and Blue chromatic map (ARB) was created and combined with the Hue image extracted after histogram equalization (HEH). The sum of absolute transformed difference (SATD), a block-matching method, was applied to detect potential fruit pixels. After OR operation of the results obtained from colour and SATD analysis which kept as many fruit pixels as possible, a kernel support vector machine (SVM) classifier was built with same learning sets used for different classification stages to remove false positives based on five selected texture features. The algorithm was evaluated with a set of testing images, and achieved more than 83% recognition accuracy. The proposed method can provide a more efficient way for green citrus identification in a grove using colour images.

A Fast Intra Prediction for H.264/AVC Based on SATD and Prediction Direction

To improve the coding efficiency of H.264/AVC, the encoder computes Rate Distortion (RD) costs of all possible intra coding modes to decide the optimum mode, this leads to high computational complexity. A fast intra prediction mode decision method to reduce the computational complexity is presented in this paper. The method employs the prediction directions and the Sum of Absolute Transformed Difference (SATD) of the reconstructed surrounding pixels and the block content to remove the less likely prediction modes from the Rate Distortion Optimization (RDO) computation. A block type skip method is also presented to further improve the performance. The experimental results show that the proposed method can reduce the coding time effectively with slight video quality degradation and bit rate increment.

A computation and energy reduction technique for HEVC intra mode decision

High Efficiency Video Coding (HEVC) intra mode decision algorithm has very high computational complexity. Therefore, in this paper, a computation and energy reduction technique is proposed for reducing the amount of computations performed by Sum of Absolute Transformed Difference (SATD) calculations in HEVC intra mode decision, and therefore reducing the energy consumption of HEVC SATD calculation hardware without any PSNR loss and bit rate increase. The proposed technique reduced the energy consumption of HEVC SATD calculation hardware up to 64.6%. Therefore, it can be used in portable consumer electronics products that require a real-time HEVC encoder.

Fast intracoding unit size decision algorithm for high-efficiency video coding

In order to reduce the computational complexity for the high-efficiency video coding standard, an algorithm is proposed. First, we create a histogram based on the distribution of gray values in each coding tree unit. By using a self-correlation function of the gray histogram, we can skip some of the depth levels that are unnecessary. Second, the relationship between the sum of absolute transformed difference (SATD) and coding unit (CU) is used to decide an early termination for the split of the current CU, if the minimum SATD of this CU is smaller than a threshold. Experimental results show that the proposed algorithm can save an average of 38.39% of the encoding time with negligible degradation of coding efficiency compared with the reference model HM10.1.

All-zero block detection algorithm in H.264 based on radial basis function network

In this paper,a kind of algorithm for all-zero block detection based on Radial Basis Function(RBF) Neural Network(NN) was proposed to improve the accuracy of all-zero block detection algorithm.By analyzing the H.264 encoder features,six effective features were selected,including Sum of Absolute Difference(SAD),Sum of Absolute Transformed Difference(SATD),block type,Rate Distortion Optimization(RDO) cost,Quantization Parameter(QP) and the situation of reference block.Considering the SATD should be used in the Hadamard Transform(HT),to get the relationship of QP and RBF network width parameter through the least square method,the algorithm used two classifiers to separate all-zero blocks from non-all-zero blocks based on the encoding situation of the reference block.This algorithm could improve coding speed over 50% on average while keeping bit rate and video quality almost unchanged.The experimental results show that the proposed algorithm can improve all-zero block detection accuracy effectively and coding efficiency based on NN.

An efficient mode decision algorithm for H.264/AVC intra prediction

Rate distortion optimization technique is adopted by H.264/AVC to select the best intra and inter prediction modes. It achieves remarkable improvement in compression performance, but the computational complexity of coding increases greatly. In order to reduce the computational complexity as much as possible while guaranteeing the video encoding quality and compression efficiency, this paper proposes a fast mode decision method based on the texture direction information of intra prediction modes and the encoding macroblocks. For intra luminance prediction, the proposed algorithm utilizes the smoothness of the encoding macroblock to select the suitable intra prediction block sizes, and then uses the texture direction difference to filter out low possibility prediction modes. The calculation expressions of texture direction difference can be derived by extracting texture direction features from intra prediction modes. For intra chrominance prediction, the candidate prediction modes are determined by a combination of texture direction difference and the sum of absolute transformed difference, which doesn't significantly degrade peak-signal-noise-rate or increase bit rate. Based on the processing, the number of rate distortion cost calculations decreases dramatically, which indicates a significant reduction of computation cost for intra prediction. Compared with JM11.0 reference software, the proposed algorithm can cut down about 76.79 % total intra-frame coding time at the expense of only about 0.08 dB peak-signal-noise-rate degradation and 2.07 % bit rate increase. It proves that the proposed algorithm achieves a tradeoff between the rate distortion performance and the computational complexity.