Sample Adaptive Offset Filter Research Articles

Hardware Efficient Integrated In-loop Filter for HEVC Encoder

The deblocking filter (DF) and the sample adaptive offset (SAO) filter, which aids in enhancing the subjective quality of the image, make up the in-loop filter of the high-efficiency video coding (HEVC) encoder and decoder. The in-loop filter significantly increases the computational load on the HEVC encoder. It is challenging to design an in-loop filter on hardware that can handle intensive computations while using the least amount of on-chip memory, taking external memory traffic and dependencies simultaneously delivering high throughput to support Ultra HD video applications. The proposed design employs the following strategies to address these issues. This work proposes an address generation technique for pipelined horizontal and vertical filtering in DF, that avoids a transpose buffer which otherwise is required. This enables easy pipelining and parallelization thus improving throughput while reducing the on-chip memory utilization. A simplified SAO filter with parallel-pipelined processing is included in the design. These features enable the design to support ultra-HD 7680 × 4320 @ 40 fps video applications. The proposed hardware architecture has a total gate count of 7.73 K LUTs and 2.8 K slice registers, and it is implemented on a 28 nm field programmable gate array (FPGA) platform.

Adaptive and Efficient Hybrid In-loop Filter Based on Enhanced Generative Adversarial Networks with Sample Adaptive Offset Filter for HEVC/H-265

In this manuscript, an Adaptive and Efficient Hybrid In-loop Filter based on Enhanced Generative Adversarial Network Deblocking Filter (EGANDF) with Sample Adaptive Offset filter (EGANDF-SAO-HEVC) is proposed for High Efficiency Video Coding (HEVC)/H-265. In this, the proposed hybrid in-loop filter involves EGANDF and Sample Adaptive Offset (SAO) filter that lessens the blocking artifacts caused by block-wise processing for coding unit (CU), which is mainly used for improving the video quality. Initially, EGANDF is proposed for HEVC/H-265 for removing blocking artifacts along low computation. Here, the output of EGANDF is given to the SAO filter for reducing ringing artifacts by diminishing high-frequency components during quantization. Thus, the proposed method efficiently reduces artifacts for improving video quality performance. The proposed EGANDF-SAO-HEVC method is implemented in the working platform of HEVC reference software with MATLAB. Finally, the proposed EGANDF-SAO-HEVC model has attained 27.26%, 29.65%, 12.45% higher accuracy, 33.56%, 31.8%, 28.7% higher sensitivity, 34.7%, 33.5%, 32.6% higher specificity, 46.92%, 35.7%, 41.3% lower MSE, 25.7%, 29.7%, 35.6% higher PSNR, and 25.6%, 28.9%, 13.6% higher SSIM for using basketball video sequence when compared to the existing methods.

Gated fusion network for SAO filter and inter frame prediction in Versatile Video Coding

In order to achieve higher coding efficiency, the Versatile Video Coding (VVC) standard includes several new components at the expense of an increase in decoder computational complexity. These technologies often create ringing and contouring effects on the reconstructed frames at a low bit rate and introduce blurring and distortion. To smooth those visual artifacts, the H.266/VVC framework supports four post-processing filter operations. The state-of-the-art CNN-based in-loop filters prefer to deploy multiple networks for various quantization parameters and frame resolutions, which increases training resources and subsequently becomes overhead at decoder frame reconstruction. This paper presents a single deep-learning-based model for sample adaptive off-set (SAO) non-linear filtering operation on the decoder side, uses feature correlation among adjacent frames, and substantiates the merits of intra–inter frame quality enhancement. We introduced a variable filter size dual multi-scale convolutional neural network (D-MSCNN) to attenuate the compression artifact and incorporated strided deconvolution to restore the high-frequency details on the distorted frame. Our model follows sequential training across all QP values and updates the model weights. Using data augmentation, weight fusion, and residual learning, we demonstrated that our model could be trained effectively by transferring the convolution prior feature indices to the decoder to produce a dense output map. The Objective measurements demonstrate that the proposed method outperforms the baseline VVC method in PSNR, MS-SSIM, and VMAF metrics and achieves an average of 5.16% bit rate saving on different test sequence categories.

Offset-Based In-Loop Filtering With a Deep Network in HEVC

With the great flexibility and performance of deep learning technology, there have been many attempts to replace existing functions inside video codecs such as High-Efficiency Video Coding (HEVC) with deep-learning-based solutions. One of the most researched approaches is adopting a deep network as an image restoration filter to recover distorted compressed frames. In this paper, instead, we introduce a novel idea for using a deep network, in which it chooses and transmits the side information according to the type of errors and contents, inspired by the sample adaptive offset filter in HEVC. A part of the network computes the optimal offset values while another part estimates the type of error and contents simultaneously. The combination of two subnetworks can address the estimation of highly nonlinear and complicated errors compared to conventional deep- learning-based schemes. Experimental results show that the proposed system yields an average bit-rate saving of 4.2% and 2.8% for the low-delay P and random access modes, respectively, compared to the conventional HEVC. Moreover, the performance improvement is up to 6.3% and 3.9% for higher-resolution sequences.

High-throughput sample adaptive offset hardware architecture for high-efficiency video coding

A high-throughput hardware architecture for a sample adaptive offset (SAO) filter in the high-efficiency video coding video coding standard is presented. First, an implementation-friendly and simplified bitrate estimation method of rate-distortion cost calculation is proposed to reduce the computational complexity in the mode decision of SAO. Then, a high-throughput VLSI architecture for SAO is presented based on the proposed bitrate estimation method. Furthermore, multiparallel VLSI architecture for in-loop filters, which integrates both deblocking filter and SAO filter, is proposed. Six parallel strategies are applied in the proposed in-loop filters architecture to improve the system throughput and filtering speed. Experimental results show that the proposed in-loop filters architecture can achieve up to 48% higher throughput in comparison with prior work. The proposed architecture can reach a high-operating clock frequency of 297 MHz with TSMC 65-nm library and meet the real-time requirement of the in-loop filters for 8 K × 4 K video format at 132 fps.

Design and Implementation of Efficient Streaming Deblocking and SAO Filter for HEVC Decoder

This paper aims to design an efficient mixed serial five-stage pipeline processing hardware architecture of deblocking filter (DBF) and sample adaptive offset (SAO) filter for high efficiency video coding decoder. The proposed hardware is designed to increase the throughput and reduce the number of clock cycles by processing the pixels in a stream of ${4 \times 36}$ samples in which edge filters are applied vertically in a parallel fashion for processing of luma/chroma samples. Subsequently these filtered pixels are transposed and reprocessed through vertical filter for horizontal filtering in a pipeline fashion. Finally, the filtered block transposed back to the original orientation and forwarded to a three-stage pipeline SAO filter. The proposed architecture is implemented in field programmable gate array and application specific integrated circuit platform using 90-nm library. Experimental results illustrate that the proposed DBF and SAO architecture decreases the processing cycles (172) required for processing each ${64 \times 64}$ or large coding unit compared with the state-of-the-art literature with the increase of gate count (593.32K) including memory. The results show that the throughput of the proposed filter can successfully decode ultrahigh definition video sequences at 200 frames/s at 341 MHz.

Implementation of in‐loop filter for HEVC decoder on reconfigurable processor

The in-loop filter comprises deblocking filter and sample adaptive offset filter, which is an important module for improving image quality in a high-efficiency video coding (HEVC) decoder. The in-loop filter has a high computational complexity that accounts for ∼20% of the HEVC decoding computing load. Furthermore, it is difficult to implement a high-performing in-loop filter due to its large conditional processing requirement. First, this study presents a novel reconfigurable HEVC in-loop filter implementation on a coarse-grained dynamically reconfigurable processing unit. Next, a repartition scheme is presented that allows the in-loop filter implementation at a coding tree unit along with the other decoding modules in the HEVC decoder, which satisfies requirements of low latency applications. Finally, a hierarchised-pipeline and synchronised-parallel technique is used to improve performance by eliminating data hazards in pipeline techniques and synchronisation problems in parallel techniques. Implementation results show that the presented HEVC in-loop filter performs up to 1920 × 1080@52 frames per second at 250 MHz. The throughput is 67.5 × 9 × more than solutions based on digital signal processor and general-purpose processor, respectively.

Programmable Low-Power Multicore Coprocessor Architecture for HEVC/H.265 In-Loop Filtering

The High Efficiency Video Coding (HEVC) in-loop filtering is designed to reduce coding artifacts caused by image transforms and quantizations. HEVC in-loop filtering is divided to the deblocking filter and the sample adaptive offset filter, and these two filters take about 20% of total decoding time. This paper presents a very low-power (39 mW) programmable coprocessor architecture to HEVC in-loop filtering, targeting especially embedded devices. The solution consists of three identical tiny application specific instruction set processor cores that are able to process 30 Full-HD intra-luma frames/s when the operating frequency is 350 MHz. The cores are fully programmable by C-language, which allows easy software modifications and updates. Although the cores have been designed for in-loop filtering, they are also capable of signal processing tasks that demand high performance. In terms of energy efficiency, the proposed architecture falls clearly between application-specified integrated circuits and conventional embedded processors, and thus forms a new-generation solution for HEVC in-loop filtering.

Low-latency SAO Architecture and its SIMD Optimization for HEVC Decoder

This paper proposes a low-latency Sample Adaptive Offset filter (SAO) architecture and its Single Instruction Multiple Data (SIMD) optimization scheme to achieve fast High Efficiency Video Coding (HEVC) decoding in a multi-core environment. According to the HEVC standard and its Test Model (HM), SAO operation is performed only at the picture level. Most realtime decoders, however, execute their sub-modules on a Coding Tree Unit (CTU) basis to reduce the latency and memory bandwidth. The proposed low-latency SAO architecture has the following advantages over picture-based SAO: 1) significantly less memory requirements, and 2) low-latency property enabling efficient pipelined multi-core decoding. In addition, SIMD optimization of SAO filtering can reduce the SAO filtering time significantly. The simulation results showed that the proposed low-latency SAO architecture with significantly less memory usage, produces a similar decoding time as a picture-based SAO in single-core decoding. Furthermore, the SIMD optimization scheme reduces the SAO filtering time by approximately 509% and increases the total decoding speed by approximately 7% compared to the existing look-up table approach of HM.