Kernel-based Interpolation Research Articles

MSEConv: A Unified Warping Framework for Video Frame Interpolation

Within the context of video frame interpolation, complex motion modeling is the task of capturing, in a video sequence, where the moving objects are located in the interpolated frame, and how to maintain the temporal consistency of motion. Existing video frame interpolation methods typically assign either a fixed size of the motion kernel or a refined optical flow to model complex motions. However, they have the limitation of data redundancy and inaccuracy representation of motion. This paper introduces a unified warping framework, named multi-scale expandable deformable convolution (MSEConv), for simultaneously performing complex motion modeling and frame interpolation. In the proposed framework, a deep fully convolutional neural network with global attention is proposed to estimate multiple small-scale kernel weights with different expansion degrees and adaptive weight allocation for each pixel synthesis. Moreover, most of the kernel-based interpolation methods can be treated as the special case of the proposed MSEConv, thus, MSEConv can be easily transferred to other kernel-based frame interpolation methods for performance improvement. To further improve the robustness of motion occlusions, an operation of mask occlusion is introduced. As a consequence, our proposed MSEConv shows strong performance on par or even better than the state-of-the-art kernel-based frame interpolation works on public datasets. Our source code and visual comparable results are available at https://github.com/Pumpkin123709/MSEConv.

Small errors imply large evaluation instabilities

Numerical analysts and scientists working in applications often observe that once they improve their techniques to get a better accuracy, some instability of the evaluation creeps in through the back door. This paper shows for a large class of numerical methods that such a Trade-off Principle between error and evaluation stability is unavoidable. It is an instance of a no free lunch theorem. Here, evaluation is the mathematical map that takes input data to output data. This is independent from the numerical routine that calculates the output. Therefore, evaluation stability is different from computational stability. The setting is confined to recovery of functions from data, but it includes solving differential equations by writing such methods as a recovery of functions under constraints imposed by differential operators and boundary values. The trade-off principle bounds the product of two terms from below. The first is related to errors, and the second turns out to be related to evaluation instability. Under certain conditions satisfied for splines and kernel-based interpolation, both can be minimized. Then the lower bound is attained, and the error term is the inverse of the instability term. As a byproduct, it is shown that Kansa’s Unsymmetric Collocation Method sacrifices accuracy for improved evaluation stability, when compared to symmetric collocation.

Learning with Partition of Unity-based Kriging Estimators

For supervised regression tasks we propose and study a new tool, namely Kriging Estimator based on the Partition of Unity (KEPU) method. Its background belongs to the framework of local kernel-based interpolation methods. Indeed, even if the latter needs to be accurately tailored for gaussian process regression, the KEPU scheme provides a global estimator which is constructed by gluing together the local Kriging predictors via compactly supported weights. The added value of this investigation is twofold. On the one hand, our theoretical studies about the propagation of the uncertainties from the local predictors to the global one offer the opportunity to define the PU method in a stochastic framework and hence to provide confidence intervals for the PU approximant. On the other hand, as confirmed by extensive numerical experiments, when the number of instances grows, such a method enables us to significantly reduce the usually high complexity cost of fitting via gaussian processes.

Learned Kernel-Based Interpolation for Efficient RGBW Remosaicing

Learned Kernel-Based Interpolation for Efficient RGBW Remosaicing

Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolution.

Generating non-existing frames from a consecutive video sequence has been an interesting and challenging problem in the video processing field. Typical kernel-based interpolation methods predict pixels with a single convolution process that convolves source frames with spatially adaptive local kernels, which circumvents the time-consuming, explicit motion estimation in the form of optical flow. However, when scene motion is larger than the pre-defined kernel size, these methods are prone to yield less plausible results. In addition, they cannot directly generate a frame at an arbitrary temporal position because the learned kernels are tied to the midpoint in time between the input frames. In this paper, we try to solve these problems and propose a novel non-flow kernel-based approach that we refer to as enhanced deformable separable convolution (EDSC) to estimate not only adaptive kernels, but also offsets, masks and biases to make the network obtain information from non-local neighborhood. During the learning process, different intermediate time step can be involved as a control variable by means of an extension of coord-conv trick, allowing the estimated components to vary with different input temporal information. This makes our method capable to produce multiple in-between frames. Furthermore, we investigate the relationships between our method and other typical kernel- and flow-based methods. Experimental results show that our method performs favorably against the state-of-the-art methods across a broad range of datasets. Code will be publicly available on URL: https://github.com/Xianhang/EDSC-pytorch.

Graph signal interpolation with positive definite graph basis functions

For the interpolation of graph signals with generalized shifts of a graph basis function (GBF), we introduce the concept of positive definite functions on graphs. This concept merges kernel-based interpolation with spectral theory on graphs and can be regarded as a graph analog of radial basis function interpolation in Euclidean spaces or spherical basis functions. We provide several descriptions of positive definite functions on graphs, the most relevant one is a Bochner-type characterization in terms of positive Fourier coefficients. These descriptions allow us to design GBF's and to study GBF interpolation in more detail. We are able to characterize the native spaces of the interpolants. We provide explicit estimates for the interpolation error and obtain bounds for the numerical stability. As a final application, we show how GBF interpolation can be used to get quadrature formulas on graphs.

Refined error estimates for Green kernel-based interpolation

Positive-definite kernels are probably best known for their application in many problems driven by scattered data interpolation. Fasshauer and Ye introduced constructive theory of reproducing kernels of generalized Sobolev spaces in 2011 to provide insight into the types of functions being well approximated by these kernels on a set of scattered points. In this approach, the reproducing kernel is viewed as the Green kernel of a suitable differential operator with some boundary conditions. Sampling inequalities and the minimum norm property in reproducing kernel Hilbert spaces (RKHSs) bring out the standard error bound; however, this estimate is valid only when the target functions belong to the native spaces of the Green kernels. In this paper we provide Sobolev-type error estimates for cases in which the target functions are smoother than functions in the native space. The results are useful and effective for the error analysis of Green kernel-based interpolation problems.

Efficient Reduced Basis Algorithm (ERBA) for Kernel-Based Approximation

The main purpose of this work is to provide an efficient scheme for constructing kernel-based reduced interpolation models. In the existing literature such problems are mainly addressed via the well-established knot insertion or knot removal schemes. Such iterative strategies are usually quite demanding from a computational point of view and our goal is to study an efficient implementation for data removal approaches, namely efficient reduced basis algorithm (ERBA). Focusing on kernel-based interpolation, the algorithm makes use of two iterative rules for removing data. The former, called ERBA-r, is based on classical residual evaluations. The latter, namely ERBA-p, is independent of the function values and relies on error bounds defined by the power function. In both cases, inspired by the so-called extended Rippa’s algorithm, our ERBA takes advantage of a fast implementation.

Fast approximation by periodic kernel-based lattice-point interpolation with application in uncertainty quantification

This paper deals with the kernel-based approximation of a multivariate periodic function by interpolation at the points of an integration lattice—a setting that, as pointed out by Zeng et al. (Monte Carlo and Quasi-Monte Carlo Methods 2004, Springer, New York, 2006) and Zeng et al. (Constr. Approx. 30: 529–555, 2009), allows fast evaluation by fast Fourier transform, so avoiding the need for a linear solver. The main contribution of the paper is the application to the approximation problem for uncertainty quantification of elliptic partial differential equations, with the diffusion coefficient given by a random field that is periodic in the stochastic variables, in the model proposed recently by Kaarnioja et al. (SIAM J Numer Anal 58(2): 1068–1091, 2020). The paper gives a full error analysis, and full details of the construction of lattices needed to ensure a good (but inevitably not optimal) rate of convergence and an error bound independent of dimension. Numerical experiments support the theory.

Correction to: Kernel-based interpolation at approximate Fekete points

Correction to: Kernel-based interpolation at approximate Fekete points

Kernel-based interpolation at approximate Fekete points

We construct approximate Fekete point sets for kernel-based interpolation by maximising the determinant of a kernel Gram matrix obtained via truncation of an orthonormal expansion of the kernel. Uniform error estimates are proved for kernel interpolants at the resulting points. If the kernel is Gaussian, we show that the approximate Fekete points in one dimension are the solution to a convex optimisation problem and that the interpolants converge with a super-exponential rate. Numerical examples are provided for the Gaussian kernel.

Video Frame Interpolation via Deformable Separable Convolution

Learning to synthesize non-existing frames from the original consecutive video frames is a challenging task. Recent kernel-based interpolation methods predict pixels with a single convolution process to replace the dependency of optical flow. However, when scene motion is larger than the pre-defined kernel size, these methods yield poor results even though they take thousands of neighboring pixels into account. To solve this problem in this paper, we propose to use deformable separable convolution (DSepConv) to adaptively estimate kernels, offsets and masks to allow the network to obtain information with much fewer but more relevant pixels. In addition, we show that the kernel-based methods and conventional flow-based methods are specific instances of the proposed DSepConv. Experimental results demonstrate that our method significantly outperforms the other kernel-based interpolation methods and shows strong performance on par or even better than the state-of-the-art algorithms both qualitatively and quantitatively.

Adaptive Filter with Type-2 Fuzzy System and Optimization-Based Kernel Interpolation for Satellite Image Denoising

Abstract Satellite image denoising is a recent trend in image processing, but faces many challenges due to the environmental factors. Previous works have developed many filters for denoising the hyperspectral satellite images. Accordingly, this work utilizes an adaptive filter with the type 2 fuzzy system and the optimization-based kernel interpolation for the satellite image denoising. Here, the image denoising has been done through three steps, namely noise identification, noise correction and image enhancement. Initially, the type 2 fuzzy system identifies the noisy pixels in the satellite image and converts the image into a binary image, which is passed through the adaptive nonlocal mean filter (ANLMF) for the noise correction. Finally, the kernel-based interpolation scheme carries out the image enhancement, which is done through the proposed chronological Jaya optimization algorithm (chronological JOA) that is developed by modifying Jaya optimization algorithm (JOA) with the chronological idea. The performance of the proposed denoising scheme is analyzed by considering the satellite images from two standard databases, namely Indian pines database and NRSC/ISRO satellite database. Also, the comparative analysis is performed with the state-of-the-art denoising methods using the evaluation metrics, peak signal to noise ratio (PSNR), structural similarity index (SSIM) and second derivative-like measure of enhancement (SDME). From the results, it is exposed that the proposed adaptive filter with the chronological JOA has the improved performance with the PSNR of 22.0408 dB, SDME of 244.133 dB and SSIM of 0.872.

Generation of point sets by convex optimization for interpolation in reproducing kernel Hilbert spaces

We propose algorithms to take point sets for kernel-based interpolation of functions in reproducing kernel Hilbert spaces (RKHSs) by convex optimization. We consider the case of kernels with the Mercer expansion and propose an algorithm by deriving a second-order cone programming (SOCP) problem that yields $n$ points at one sitting for a given integer $n$. In addition, by modifying the SOCP problem slightly, we propose another sequential algorithm that adds an arbitrary number of new points in each step. Numerical experiments show that in several cases the proposed algorithms compete with the $P$-greedy algorithm, which is known to provide nearly optimal points.

Discrete least-squares radial basis functions approximations

We consider discrete least-squares methods using radial basis functions. A general ℓ2-Tikhonov regularization with W2m-penalty is considered. We provide error estimates that are comparable to kernel-based interpolation in cases which the function it is approximating is within and is outside of the native space of the kernel. Our proven theories concern the denseness condition of collocation points and selection of regularization parameters. In particular, the unregularized least-squares method is shown to have W2μ(Ω) convergence for μ > d/2 on smooth domain Ω⊂Rd. For any properly regularized least-squares method, the same convergence estimates hold for a large range of μ ≥ 0. These results are extended to the case of noisy data. Numerical demonstrations are provided to verify the theoretical results. In terms of applications, we also apply the proposed method to solve a heat equation whose initial condition has huge oscillation in the domain.

Kernel-based collocation methods for Zakai equations

We examine an application of the kernel-based interpolation to numerical solutions for Zakai equations in nonlinear filtering, and aim to prove its rigorous convergence. To this end, we find the class of kernels and the structure of collocation points explicitly under which the process of iterative interpolation is stable. This result together with standard argument in error estimation shows that the approximation error is bounded by the order of the square root of the time step and the error that comes from a single step interpolation. Our theorem is well consistent with the results of numerical experiments.

Algorithmic Patterns for $$\mathcal {H}$$-Matrices on Many-Core Processors

In this work, we consider the reformulation of hierarchical ( $$\mathcal {H}$$ ) matrix algorithms for many-core processors with a model implementation on graphics processing units (GPUs). $$\mathcal {H}$$ matrices approximate specific dense matrices, e.g., from discretized integral equations or kernel ridge regression, leading to log-linear time complexity in dense matrix–vector products. The parallelization of $$\mathcal {H}$$ matrix operations on many-core processors is difficult due to the complex nature of the underlying algorithms. While previous algorithmic advances for many-core hardware focused on accelerating existing $$\mathcal {H}$$ matrix CPU implementations by many-core processors, we here aim at totally relying on that processor type. As main contribution, we introduce the necessary parallel algorithmic patterns allowing to map the full $$\mathcal {H}$$ matrix construction and the fast matrix–vector product to many-core hardware. Here, crucial ingredients are space filling curves, parallel tree traversal and batching of linear algebra operations. The resulting model GPU implementation hmglib is the, to the best of the authors knowledge, first entirely GPU-based Open Source $$\mathcal {H}$$ matrix library of this kind. We investigate application examples as present in kernel ridge regression, Gaussian Process Regression and kernel-based interpolation. In this context, an in-depth performance analysis and a comparative performance study against a standard multi-core CPU $$\mathcal {H}$$ matrix library highlights profound speedups of our many-core parallel approach.

A multilevel sparse kernel-based stochastic collocation finite element method for elliptic problems with random coefficients

A new stochastic collocation finite element method is proposed for the numerical solution of elliptic boundary value problems (BVP) with random coefficients, assuming that the randomness is well-approximated by a finite number of random variables with given probability distributions. The proposed method consists of a finite element approximation in physical space, along with a stochastic collocation quadrature approach utilizing the recent Multilevel Sparse Kernel-Based Interpolation (MuSIK) technique (Georgoulis et al., 2013). MuSIK is based on a multilevel sparse grid-type algorithm with the basis functions consisting of directionally anisotropic Gaussian radial basis functions (kernels) placed at directionally-uniform grid-points. We prove that MuSIK is interpolatory at these nodes, and, therefore, can be naturally used to define a quadrature scheme. Numerical examples are also presented, assessing the performance of the new algorithm in the context of high-dimensional stochastic collocation finite element methods.

Superconvergence of kernel-based interpolation

From spline theory it is well-known that univariate cubic spline interpolation, if carried out in its natural Hilbert space W22[a,b] and on point sets with fill distance h, converges only like O(h2) in L2[a,b] if no additional assumptions are made. But superconvergence up to order h4 occurs if more smoothness is assumed and if certain additional boundary conditions are satisfied. This phenomenon was generalized in 1999 to multivariate interpolation in Reproducing Kernel Hilbert Spaces on domains Ω⊂Rd for continuous positive definite Fourier-transformable shift-invariant kernels on Rd. But the sufficient condition for superconvergence given in 1999 still needs further analysis, because the interplay between smoothness and boundary conditions is not clear at all. Furthermore, if only additional smoothness is assumed, superconvergence is numerically observed in the interior of the domain, but a theoretical foundation still is a challenging open problem. This paper first generalizes the “improved error bounds” of 1999 by an abstract theory that includes the Aubin–Nitsche trick and the known superconvergence results for univariate polynomial splines. Then the paper analyzes what is behind the sufficient conditions for superconvergence. They split into conditions on smoothness and localization, and these are investigated independently. If sufficient smoothness is present, but no additional localization conditions are assumed, it is numerically observed that superconvergence always occurs in the interior of the domain, and some supporting arguments are provided. If smoothness and localization interact in the kernel-based case on Rd, weak and strong boundary conditions in terms of pseudodifferential operators occur. A special section on Mercer expansions is added, because Mercer eigenfunctions always satisfy the sufficient conditions for superconvergence. Numerical examples illustrate the theoretical findings.

Multilevel quasi-interpolation on a sparse grid with the Gaussian

Motivated by the recent multilevel sparse kernel-based interpolation (MuSIK) algorithm proposed in Georgoulis et al. (SIAM J. Sci. Comput. 35, 815–832, 2013), we introduce the new quasi-multilevel sparse interpolation with kernels (Q-MuSIK) via the combination technique. The Q-MuSIK scheme achieves better convergence and run time when compared with classical quasi-interpolation. Also, the Q-MuSIK algorithm is generally superior to the MuSIK methods in terms of run time in particular in high-dimensional interpolation problems, since there is no need to solve large algebraic systems. We subsequently propose a fast, low complexity, high-dimensional positive-weight quadrature formula based on Q-MuSIKSapproximation of the integrand. We present the results of numerical experimentation for both quasi-interpolation and quadrature in high dimensions.