Adaptive Interpolation Filter Research Articles

Frequency-Based Adaptive Interpolation Filter in Intra Prediction

This paper proposes a method to improve the fractional interpolation of reference samples in the Versatile Video Coding (VVC) intra prediction. The proposed method uses additional interpolation filters which use more integer-positioned reference samples for prediction according to the frequency information of the reference samples. In VVC, a 4-tap Discrete Cosine Transform-based interpolation filter (DCT-IF) and 4-tap Smoothing interpolation filter (SIF) are alternatively performed on the block size and block directional prediction mode for reference sample interpolation. This paper uses four alternative interpolation filters such as 8-tap/4-tap DCT-IFs, and 4-tap/8-tap SIFs and an interpolation filter selection method using a high-frequency ratio calculated from one-dimensional (1D) transform of the reference samples are proposed. The proposed frequency-based Adaptive Filter allows to achieve the overall Bjøntegaard Delta (BD) rate gains of −0.16%, −0.13%, and −0.09% for Y, Cb, and Cr components, respectively, compared with VVC.

High speed reconstruction system for indoor environments based on an adaptive interpolation filter

High speed reconstruction system for indoor environments based on an adaptive interpolation filter

An Adaptive Surface Interpolation Filter Using Cloth Simulation and Relief Amplitude for Airborne Laser Scanning Data

Separating point clouds into ground and nonground points is an essential step in the processing of airborne laser scanning (ALS) data for various applications. Interpolation-based filtering algorithms have been commonly used for filtering ALS point cloud data. However, most conventional interpolation-based algorithms have exhibited a drawback in terms of retaining abrupt terrain characteristics, resulting in poor algorithmic precision in these regions. To overcome this drawback, this paper proposes an improved adaptive surface interpolation filter with a multilevel hierarchy by using a cloth simulation and relief amplitude. This method uses three hierarchy levels of provisional digital elevation model (DEM) raster surfaces with thin plate spline (TPS) interpolation to separate ground points from unclassified points based on adaptive residual thresholds. A cloth simulation algorithm is adopted to generate sufficient effective initial ground seeds for constructing topographic surfaces with high quality. Residual thresholds are adaptively constructed by the relief amplitude of the examined area to capture complex landscape characteristics during the classification process. Fifteen samples from the International Society for Photogrammetry and Remote Sensing (ISPRS) commission are used to assess the performance of the proposed algorithm. The experimental results indicate that the proposed method can produce satisfying results in both flat areas and steep areas. In a comparison with other approaches, this method demonstrates its superior performance in terms of filtering results with the lowest omission error rate; in particular, the proposed approach retains discontinuous terrain features with steep slopes and terraces.

Guided adaptive interpolation filter

Edge-aware smoothing has proved to be a fundamental technique for various image processing and computer vision tasks. In this study, the authors introduce a local, non-iterative, and effective edge-preserving filter namely guided adaptive interpolation filter (GAIF). GAIF can be used as a post-processing step after any smoothing filter to improve its edge preservation performance without reformulation. GAIF has an computation complexity, where N is the total number of pixels in the image. To further increase the efficiency of GAIF at edge-preservation, two techniques are introduced and demonstrated. GAIF efficiency is demonstrated and compared to state-of-the-art techniques on a number of tasks including image smoothing, flash/no-flash image denoising/fusion, single image dehazing, and image details enhancement.

Adaptive switching interpolation filter for restoring impulse corrupted digital images

The study introduces an adaptive switching interpolation filter (ASIF) for restoring images contaminated with salt & pepper impulse noise. The new method restores noisy pixels by applying a new interpolation scheme that combines Shepard's inverse distance weighting and Simpson's natural neighbour interpolation techniques. The new interpolation scheme is designed based on the percentage of the overlapping area and the distance between the sites created by noisy pixels with the sites of uncorrupted pixels in the Voronoi tessellation. Since natural neighbour-based techniques do not support extrapolation, the proposed algorithm performs Shepard's inverse distance weighting-based restoration when the noisy pixels fall outside the convex hull formed by uncorrupted pixels. As the proposed method combines the advantages of Simpson's natural neighbour and Shepard's inverse distance weighting interpolation methods, it better preserves the natural intensity variations in images and provides smoother approximation than other methods used in the comparative study. Visual and quantitative experimental analysis performed on various images with peak signal to noise ratio, mean structural similarity index measure, image enhancement factor, and feature-similarity index measure demarcates the improved capability of ASIF over other comparative filters.

Content adaptive interpolation filters based on HEVC framework

Motion compensation is the key technique to reduce temporal redundancy in video coding. Interpolation filters are adopted to generate the inter frame prediction for motion compensation with fractional pixel accuracy. In existing video coding standards such as H.264/AVC and HEVC, a set of predefined interpolation filters is adopted in motion compensation. However, predefined interpolation filters cannot adapt to the video content, which may compromise the coding efficiency. In this paper, a content adaptive interpolation scheme is proposed for motion compensation. In the proposed scheme, a set of adaptive interpolation filters is derived for each frame as additional interpolation filters to minimize the inter prediction difference. Rate-distortion optimization is employed to choose between the predefined interpolation filters and the derived adaptive interpolation filters to achieve the best coding performance at the low bit rates. The proposed scheme is implemented into the HM 12.1 software. Experimental results show that the proposed scheme achieves 5.13 percent, 3.42 percent and 4.07 percent bit rate saving on average compared with HEVC under the “low delay P”, the “low delay B” and the “random access” configurations respectively.

Fast digital zooming system using directionally adaptive image interpolation and restoration.

This paper presents a fast digital zooming system for mobile consumer cameras using directionally adaptive image interpolation and restoration methods. The proposed interpolation algorithm performs edge refinement along the initially estimated edge orientation using directionally steerable filters. Either the directionally weighted linear or adaptive cubic-spline interpolation filter is then selectively used according to the refined edge orientation for removing jagged artifacts in the slanted edge region. A novel image restoration algorithm is also presented for removing blurring artifacts caused by the linear or cubic-spline interpolation using the directionally adaptive truncated constrained least squares (TCLS) filter. Both proposed steerable filter-based interpolation and the TCLS-based restoration filters have a finite impulse response (FIR) structure for real time processing in an image signal processing (ISP) chain. Experimental results show that the proposed digital zooming system provides high-quality magnified images with FIR filter-based fast computational structure.

Block Adaptive Interpolation Filter Using Trained Dictionary for Sub-Pixel Motion Compensation

Adaptive interpolation filtering for sub-pel motion compensation is one of key techniques of ITU-T key technology area (KTA) codec. However, the adaptive interpolation filtering has a limitation in coding efficiency because of its frame-based update strategy of filter coefficients. Although switched interpolation filter with offset is presented as a sort of block-adaptive filtering for KTA codec, its coding efficiency is generally lower than that of the best adaptive interpolation filter. In order to overcome such a problem, this paper presents an advanced block-adaptive interpolation filtering using well-trained dictionaries which store optimized filter coefficients. We derive those filter coefficients by using learning-based super-resolution. The proposed block-adaptive interpolation filtering for quarter-pel motion compensation consists of two steps: up-scaling of half-pel accuracy and subsequent up-scaling of quarter-pel accuracy. The dictionary optimized for each step is employed to produce the precise up-scaled pixels. Simulation results show that the proposed algorithm improves higher coding efficiency than the previous adaptive interpolation filters for KTA.

A Single-Pass-Based Localized Adaptive Interpolation Filter for Video Coding

Recently, an efficient coding tool named adaptive interpolation filtering (AIF) has been proposed to hybrid video coding scheme. By introducing Wiener filter into the fractional-pixel interpolation procedure, AIF can reduce the inter-prediction error and improve coding efficiency significantly. However, the training-based Wiener filter mechanism brings AIF an inherent multi-pass encoding structure, which imposes big burdens on the encoder in terms of huge computational complexity and memory access. In this paper, we propose a single-pass-based localized adaptive interpolation filtering (SPL-AIF) algorithm for video coding, which can reduce the complexity of AIF dramatically without sacrifice of its outstanding coding performance. The proposed SPL-AIF algorithm is based on the observation that there is a high correlation among optimal interpolation filters of consecutive frames, and different regions in a frame often possess different statistical characteristics. Accordingly, the proposed algorithm can be designed including two major parts. First, a competitive filter set which includes the optimal interpolation filters of several previous frames as well as the fixed H.264/AVC interpolation filters is built up for the coding of the current frame. Then a rate-distortion optimization criterion is used to select the best one at macroblock (MB) level. In order to reduce overhead, a predictive coding method is used to compress the filter signaling flag for each MB. Experimental results show that, by using the proposed algorithm, the encoding complexity can be reduced significantly while the average coding gain in Bjontegaard distortion bit-rate reduction can be improved about 1% compared with the multi-pass AIF. The proposed method has been adopted into the Video Coding Expert Group Key Technology Area software.

Parametric OBMC for Pixel-Adaptive Temporal Prediction on Irregular Motion Sampling Grids

This paper adapts overlapped block motion compensation (OBMC) to suit variable block-size motion partitioning. The motion vectors (MVs) for various partitions are formalized as motion samples taken on an irregular grid. From this viewpoint, determining OBMC weights to associate with these samples becomes an under-determined problem since a distinct solution has to be sought for each prediction pixel. In this paper, we tackle this problem by expressing the optimal weights in closed form based on parametric signal assumptions. In particular, the computation of this solution requires only the geometric relations between the prediction pixel and its nearby block centers, leading to a generic framework capable of reconstructing temporal predictors from any irregularly sampled MVs. A modified implementation is also proposed to address the MV location uncertainty and to reduce computational complexity. Experimental results demonstrate that our scheme performs better than similar previous works, and when compared to the recently proposed Quadtree-based adaptive loop filter and enhanced adaptive interpolation filter, show a comparable gain. Furthermore, the combination of it with either of them gives a combined effect that is almost the sum of their separate improvements.

Enhanced Adaptive Loop Filter for Motion Compensated Frame

We propose an adaptive loop filter to remove the redundancy between current and motion compensated frames so that the residual signal is minimized, thus coding efficiency increases. The loop filter coefficients and offset are optimized for each frame or a set of blocks to minimize the total energy of the residual signal resulting from motion estimation and compensation. The optimized loop filter with offset is applied for the set of blocks where the filtering process gives coding gain based upon rate-distortion cost. The proposed loop filter is used for the motion compensated frame whereas the conventional adaptive interpolation filter (AIF) is applied to the reference frames to interpolate the subpixel values. Another conventional scheme adaptive loop filter (ALF), is used after deblocking filtering to enhance quality of reconstructed frames, not to minimize energy of residual signal. The proposed loop filter can be used in combination with the AIF and ALF. Experimental results show that proposed algorithm provides the averaged bit reduction of 8% compared to conventional H.264/AVC scheme. When the proposed scheme is combined with AIF and ALF, the coding gain increases even further.

Adaptive pre-interpolation filter for high efficiency video coding

The proposed interpolation filter comprises two concatenating filters, adaptive pre-interpolation filter (APIF) and the normative interpolation filter in H.264/AVC. The former is applied only to the integer pixels in the reference frames; the latter generates all the sub-position samples, supported by the output of APIF. The convolution of APIF and the standard filter minimizes the motion prediction error on a frame basis. APIF preserves the merits of the adaptive interpolation filter (AIF) and the adaptive loop filter (ALF) in the key technical area (KTA) software and at the same time overcomes their drawbacks. The experimental results show that APIF outperforms either AIF or ALF. Compared with the joint use of AIF and ALF, APIF provides comparable performance, but has much lower complexity.

Content Based Coarse to Fine Adaptive Interpolation Filter for High Resolution Video Coding

With the increasing demand of high video quality and large image size, adaptive interpolation filter (AIF) addresses these issues and conquers the time varying effects resulting in increased coding efficiency, comparing with recent H.264 standard. However, currently most AIF algorithms are based on either frame level or macroblock (MB) level, which are not flexible enough for different video contents in a real codec system, and most of them are facing a severe time consuming problem. This paper proposes a content based coarse to fine AIF algorithm, which can adapt to video contents by adding different filters and conditions from coarse to fine. The overall algorithm has been mainly made up by 3 schemes: frequency analysis based frame level skip interpolation, motion vector modeling based region level interpolation, and edge detection based macroblock level interpolation. According to the experiments, AIF are discovered to be more effective in the high frequency frames, therefore, the condition to skip low frequency frames for generating AIF coefficients has been set. Moreover, by utilizing the motion vector information of previous frames the region level based interpolation has been designed, and Laplacian of Gaussian based macroblock level interpolation has been proposed to drive the interpolation process from coarse to fine. Six 720p and six 1080p video sequences which cover most typical video types have been tested for evaluating the proposed algorithm. The experimental results show that the proposed algorithm reduce total encoding time about 41% for 720p and 25% for 1080p sequences averagely, comparing with Key Technology Areas (KTA) Enhanced AIF algorithm, while obtains a BDPSNR gain up to 0.004 and 3.122 BDBR reduction.

3-2 フィルタ係数符号量削減機能を有する領域ベース適応補間フィルタ(第3部門 メディア処理1)

3-2 フィルタ係数符号量削減機能を有する領域ベース適応補間フィルタ(第3部門 メディア処理1)

Parametric Interpolation Filter for HD Video Coding

Recently, adaptive interpolation filter (AIF) for motion-compensated prediction (MCP) has received increasing attention. This letter studies the existing AIF techniques, and points out that making tradeoff between the two conflicting aspects: the accuracy of coefficients and the size of side information, is the major obstacle to improving the performance of the AIF techniques that code the filter coefficients individually. To overcome this obstacle, parametric interpolation filter (PIF) is proposed for MCP, which represents interpolation filters by a function determined by five parameters instead of by individual coefficients. The function is designed based on the fact that high frequency energies of HD video source are mainly distributed along the vertical and horizontal directions; the parameters are calculated to minimize the energy of prediction error. The experimental results show that PIF outperforms the existing AIF techniques and approaches the efficiency of the optimal filter.

Video Coding Using a Simplified Block Structure and Advanced Coding Techniques

This paper describes a new video coding scheme based on a simplified block structure that significantly outperforms the coding efficiency of the ISO/IEC 14496-10 ITU-T H.264 advanced video coding (AVC) standard. Its conceptual design is similar to a typical block-based hybrid coder applying prediction and subsequent prediction error coding. The basic coding unit is an 8 × 8 block for inter, and an 8 × 8 or a 16 × 16 block for intra, instead of the usual 16 × 16 macroblock. No larger block sizes are considered for prediction and transform. Based on this simplified block structure, the coding scheme uses simple and fundamental coding tools with optimized encoding algorithms. In particular, the motion representation is based on a minimum partitioning with blocks sharing motion borders. In addition, compared to AVC, the new and improved coding techniques include: block-based intensity compensation, motion vector competition, adaptive motion vector resolution, adaptive interpolation filters, edge-based intra prediction and enhanced chrominance prediction, intra template matching, larger trans forms and adaptive switchable transforms selection for intra and inter blocks, and nonlinear and frame-adaptive de-noising loop filters. Finally, the entropy coder uses a generic flexible zero tree representation applied to both motion and texture data. Attention has also been given to algorithm designs that facilitate parallelization. Compared to AVC, the new coding scheme offers clear benefits in terms of subjective video quality at the same bit rate. Objective quality improvements are equally significant. At the same quality, an average bit-rate reduction of 31% compared to AVC is reported.

Improved Adaptive Interpolation Filter for H.264/AVC

Motion compensation with adaptive interpolation filters (AIF) was developed to compensate for temporary varieties in the aliasing of video signals and improve the coding efficiency. The AIF achieves better RD performance than common static interpolation filtering by exploiting the statistics in the reference frames' local auto-correlation and the local cross-correlation between the current encoding frame and the reference frames. This paper presents an interpolation filter buffering structure that derives the current encoding frame's interpolation filters from the filters of previous frames. The number of encoding bits for the filter coefficients is reduced by encoding only the differences between the current filter coefficients and the corresponding buffered filter coefficients. Experimental results show that in comparison with the AIF in the current “key technical area” (KTA) reference software, this interpolation filter buffering structure further improves all the test sequences (with up to 2.87% bit rate saving) with negligible computational increase.

A refined motion estimation strategy for adaptive interpolation filter

Adaptively selecting the filter coefficients for interpolating fractional pixels (sub-pel) can significantly improve the quality of reference pictures. However, the complexity of implementation is greatly increased by performing motion estimation (ME) and mode decision (MD) for multiple passes. This paper briefly introduces the adaptive interpolation filter (AIF) technique and proposes a refined ME strategy for realizing the AIF process with low complexity. By using this method, the 2-pass encoding procedure can be avoided while the coding efficiency of the AIF is maintained. The proposed method is implemented and verified on the AVS video software platform.

Video Coding With Low-Complexity Directional Adaptive Interpolation Filters

A novel adaptive interpolation filter structure for video coding with motion-compensated prediction is presented in this letter. The proposed scheme uses an independent directional adaptive interpolation filter for each sub-pixel location. The Wiener interpolation filter coefficients are computed analytically for each inter-coded frame at the encoder side and transmitted to the decoder. Experimental results show that the proposed method achieves up to 1.1 dB coding gain and a 15% average bit-rate reduction for high-resolution video materials compared to the standard nonadaptive interpolation scheme of H.264/AVC, while requiring 36% fewer arithmetic operations for interpolation. The proposed interpolation can be implemented in exactly 16-bit arithmetic, thus it can have important use-cases in mobile multimedia environments where the computational resources are severely constrained.

Efficient implementation of adaptive interpolation filters for low complexity video coding

This paper presents novel methods for implementing adaptive interpolation filtering techniques for video coding on 16-bit integer architectures. The proposed methods offer significant complexity reduction over the state-of-the-art, making them especially valuable for resource constrained mobile multimedia devices, with high coding efficiency.