Reproducing Kernel Hilbert Space Research Articles

A novel numerical approach to solutions of fractional Bagley-Torvik equation fitted with a fractional integral boundary condition

Abstract In this work, we present a sophisticated operating algorithm, the reproducing kernel Hilbert space method, to investigate the approximate numerical solutions for a specific class of fractional Begley-Torvik equations (FBTE) equipped with fractional integral boundary condition. Such fractional integral boundary condition allows us to understand the non-local behavior of FBTE along with the given domain. The algorithm methodology depends on creating an orthonormal basis based on reproducing kernel function that satisfies the constraint boundary conditions so that the solution is finally formulated in the form of a uniformly convergent series in ϖ 3 [ a , b ] {\varpi }_{3}\left[a,b] . From a numerical point of view, some illustrative examples are provided to determine the appropriateness of algorithm design and the effect of using non-classical boundary conditions on the behavior of solutions approach.

Gaussian random field approximation via Stein's method with applications to wide random neural networks

We derive upper bounds on the Wasserstein distance (W1), with respect to sup-norm, between any continuous Rd valued random field indexed by the n-sphere and the Gaussian, based on Stein's method. We develop a novel Gaussian smoothing technique that allows us to transfer a bound in a smoother metric to the W1 distance. The smoothing is based on covariance functions constructed using powers of Laplacian operators, designed so that the associated Gaussian process has a tractable Cameron-Martin or Reproducing Kernel Hilbert Space. This feature enables us to move beyond one dimensional interval-based index sets that were previously considered in the literature. Specializing our general result, we obtain the first bounds on the Gaussian random field approximation of wide random neural networks of any depth and Lipschitz activation functions at the random field level. Our bounds are explicitly expressed in terms of the widths of the network and moments of the random weights. We also obtain tighter bounds when the activation function has three bounded derivatives.

Recovery of bandlimited graph signals based on the reproducing kernel Hilbert space

Signal recovery on graphs is attracting more and more attentions. Based on the smoothness assumption, the signal recovery problem can be formulated as an unconstrained optimization model. Although the model can be solved analytically, the computational cost is too expensive. In this paper, to reduce the computational complexity, we skillfully construct a kernel function, such that the smoothness item is equivalent to the norm of the signal modulo a constant in the reproducing kernel Hilbert space. Based on this, by using the Representer Theorem of reproducing kernel Hilbert space, the optimization problem for recovering the signal from a given bandlimited space, can be solved in a lower-dimensional subspace corresponding to the known labeled set. Experiments for the recovery of artificial graph signals in GSPBOX and real data in the recommendation system show very encouraging results.

Join multiple Riemannian manifold representation and multi‐kernel non‐redundancy for image clustering

AbstractImage clustering has received significant attention due to the growing importance of image recognition. Researchers have explored Riemannian manifold clustering, which is capable of capturing the non‐linear shapes found in real‐world datasets. However, the complexity of image data poses substantial challenges for modelling and feature extraction. Traditional methods such as covariance matrices and linear subspace have shown promise in image modelling, and they are still in their early stages and suffer from certain limitations. However, these include the uncertainty of representing data using only one Riemannian manifold, limited feature extraction capacity of single kernel functions, and resulting incomplete data representation and redundancy. To overcome these limitations, the authors propose a novel approach called join multiple Riemannian manifold representation and multi‐kernel non‐redundancy for image clustering (MRMNR‐MKC). It combines covariance matrices with linear subspace to represent data and applies multiple kernel functions to map the non‐linear structural data into a reproducing kernel Hilbert space, enabling linear model analysis for image clustering. Additionally, the authors use matrix‐induced regularisation to improve the clustering kernel selection process by reducing redundancy and assigning lower weights to identical kernels. Finally, the authors also conducted numerous experiments to evaluate the performance of our approach, confirming its superiority to state‐of‐the‐art methods on three benchmark datasets.

On Berezin norm and Berezin number inequalities for sum of operators

Abstract The aim of this study is to obtain several inequalities involving the Berezin number and the Berezin norm for various combinations of operators acting on a reproducing kernel Hilbert space. First, we present some bounds regarding the Berezin number associated with W * Q + W * Q ′ {W}^{* }Q+{W}^{* }Q^{\prime} , where W W , Q Q , and Q ′ Q^{\prime} are three bounded linear operators. Next, several Berezin norm and Berezin number inequalities for the sum of n n operators are established.

Stability of convergence rates: kernel interpolation on non-Lipschitz domains

Abstract Error estimates for kernel interpolation in Reproducing Kernel Hilbert Spaces usually assume quite restrictive properties on the shape of the domain, especially in the case of infinitely smooth kernels like the popular Gaussian kernel. In this paper we prove that it is possible to obtain convergence results (in the number of interpolation points) for kernel interpolation for arbitrary domains $\varOmega \subset{\mathbb{R}} ^{d}$, thus allowing for non-Lipschitz domains including e.g., cusps and irregular boundaries. Especially we show that, when going to a smaller domain $\tilde{\varOmega } \subset \varOmega \subset{\mathbb{R}} ^{d}$, the convergence rate does not deteriorate—i.e., the convergence rates are stable with respect to going to a subset. We obtain this by leveraging an analysis of greedy kernel algorithms. The impact of this result is explained on the examples of kernels of finite as well as infinite smoothness. A comparison to approximation in Sobolev spaces is drawn, where the shape of the domain $\varOmega $ has an impact on the approximation properties. Numerical experiments illustrate and confirm the analysis.

Equivalence between constrained optimal smoothing and Bayesian estimation

In this paper, we extend the correspondence between Bayesian estimation and optimal smoothing in a Reproducing Kernel Hilbert Space (RKHS) by adding convex constraints to the problem. Through a sequence of approximating Hilbertian subspaces and a discretised model, we prove that the Maximum a posteriori (MAP) of the posterior distribution is exactly the optimal constrained smoothing function in the RKHS. This paper can be read as a generalisation of the paper Kimeldorf and Wahba (1970), where it is proved that the optimal smoothing solution is the mean of the posterior distribution. Synthetic and real data studies confirm the correspondence established in this paper.

Kernel-algorithms in frame-approximations

With view to applications, we present here general classes of non-orthogonal expansions in Hilbert space. The main purpose of our paper is a new approach to design of algorithms of Kaczmarz type in the framework of operators in Hilbert space. Our work includes a diverse list of optimization problems, new Karhunen-Loève transforms, and Principal Component Analysis (PCA) for digital images. A key feature of our algorithms is our use of recursive systems of projection operators. For this we also make use of specific reproducing kernel Hilbert spaces, kernel factorizations, and finite-dimensional approximations. Our projection algorithms are designed with view to maximum likelihood solutions, minimization of “cost” problems, identification of principal components, and data-dimension reduction.

On the convergence of gradient descent for robust functional linear regression

Functional data analysis offers a set of statistical methods concerned with extracting insights from intrinsically infinite-dimensional data and has attracted considerable amount of attentions in the past few decades. In this paper, we study robust functional linear regression model with a scalar response and a functional predictor in the framework of reproducing kernel Hilbert spaces. A gradient descent algorithm with early stopping is introduced to solve the corresponding empirical risk minimization problem associated with robust loss functions. By appropriately selecting the early stopping rule and the scaling parameter of the robust losses, the convergence of the proposed algorithm is established when the response variable is bounded or satisfies a moment condition. Explicit learning rates with respect to both estimation and prediction error are provided in terms of regularity of the regression function and eigenvalue decay rate of the integral operator induced by the reproducing kernel and covariance function.

Reproducing property of bounded linear operators and kernel regularized least square regressions

We consider bounded linear operators from the view of functional reproducing property. For some bounded linear operators associated with orthogonal polynomials we define an inner product space associated with a kernel constructed with orthogonal polynomials, show that it is a functional reproducing kernel Hilbert space (FRKHS) associated with these bounded linear operators and give decay rate for the best FRKHS approximation with a [Formula: see text]-functional associated with the FRKHS. On this basis, we provide a learning rate for kernel regularized regression whose hypothesis space is the defined FRKHS. As applications, we define some concrete FRKHSs associated with polynomial operators such as the Bernstein–Durrmeyer operators, the de la Vallée Poussin operators on both the unit sphere [Formula: see text] and the unit ball [Formula: see text]. We show that these polynomial operators have reproducing property with respect to the corresponding concrete FRKHSs and show the learning rate for the kernel regularized regression. In short, we provide a way of constructing FRKHS operators with Fourier multipliers and show a learning framework from the view of operator approximation.

Propagating uncertainty through system dynamics in reproducing kernel Hilbert space

We present a data-driven approach to propagate uncertainty in initial conditions through the dynamics of an unknown system in a reproducing kernel Hilbert space (RKHS). The uncertainty in initial conditions is represented through its kernel mean embedding (KME) in the RKHS. For a discrete-time Markovian dynamical system, we utilize the conditional mean embedding (CME) operator to encode the underlying dynamics. Learning in RKHS often incurs prohibitive data storage requirements. To circumvent said limitation, we propose an algorithm to propagate uncertainty via a learned sparse CME operator, and provide theoretical guarantees on the approximation error for the embedded distribution with time. We empirically study our algorithm over illustrative dynamical systems and power systems.

Generalized 𝑞-Fock spaces and structural identities

Using q q -calculus we study a family of reproducing kernel Hilbert spaces which interpolate between the Hardy space and the Fock space. We give characterizations of these spaces in terms of classical operators such as integration and backward-shift operators, and their q q -calculus counterparts. Furthermore, these new spaces allow us to study intertwining operators between classic backward-shift operators and the q-Jackson derivative.

Bergman kernel functions associated to measures supported on totally real submanifolds

Abstract We prove that the Bergman kernel function associated to a smooth measure supported on a piecewise-smooth maximally totally real submanifold 𝐾 in C n \mathbb{C}^{n} is of polynomial growth. For example, this holds in dimension one if 𝐾 is a finite union of transverse Jordan arcs in ℂ. Our bounds are sharp when 𝐾 is smooth. We give an application to the equidistribution of the zeros of random polynomials, which extends a result of Shiffman–Zelditch to the higher-dimensional setting.

Spectral decomposition of [formula omitted] and Poincaré inequality on a compact interval — Application to kernel quadrature

Motivated by uncertainty quantification of complex systems, we aim at finding quadrature formulas of the form ∫abf(x)dμ(x)=∑i=1nwif(xi) where f belongs to H1(μ). Here, μ belongs to a class of continuous probability distributions on [a,b]⊂R and ∑i=1nwiδxi is a discrete probability distribution on [a,b]. We show that H1(μ) is a reproducing kernel Hilbert space with a continuous kernel K, which allows to reformulate the quadrature question as a kernel (or Bayesian) quadrature problem. Although K has not an easy closed form in general, we establish a correspondence between its spectral decomposition and the one associated to Poincaré inequalities, whose common eigenfunctions form a T-system (Karlin and Studden, 1966). The quadrature problem can then be solved in the finite-dimensional proxy space spanned by the first eigenfunctions. The solution is given by a generalized Gaussian quadrature, which we call Poincaré quadrature.We derive several results for the Poincaré quadrature weights and the associated worst-case error. When μ is the uniform distribution, the results are explicit: the Poincaré quadrature is equivalent to the midpoint (rectangle) quadrature rule. Its nodes coincide with the zeros of an eigenfunction and the worst-case error scales as b−a23n−1 for large n. By comparison with known results for H1(0,1), this shows that the Poincaré quadrature is asymptotically optimal. For a general μ, we provide an efficient numerical procedure, based on finite elements and linear programming. Numerical experiments provide useful insights: nodes are nearly evenly spaced, weights are close to the probability density at nodes, and the worst-case error is approximately O(n−1) for large n.

Measure induced Hankel and Toeplitz type operators on weighted Dirichlet spaces

For a complex Borel measure μ \mu on the open unit disk, and for a weighted Dirichlet space H s \mathcal {H}_s with 0 > s > 1 0>s>1 , we characterize the boundedness of the measure induced Hankel type operator H μ , s : H s → H s ¯ H_{\mu ,s}: \mathcal {H}_s \to \overline {\mathcal {H}_s} , extending the results of Xiao [Bull. Austral. Math. Soc. 62 (2000), pp. 135–140] for the classical Hardy space H 2 = H 1 H^2=\mathcal {H}_1 , and of Arcozzi, Rochberg, Sawyer, and Wick [J. Lond. Math. Soc. (2) 83 (2011), pp. 1–18] for the classical Dirichlet space D = H 0 \mathcal {D}= \mathcal {H}_0 . Our approach relies on some recent results about weak products of complete Nevanlinna-Pick reproducing kernel Hilbert spaces. We also include some related results on Hankel measures, Carleson measures, and Toeplitz type operators on weighted Dirichlet spaces H s \mathcal {H}_s , 0 > s > 1 0>s>1 .

Linear canonical Stockwell transform and the associated multiresolution analysis

In this article, we propose a new class of transform called linear canonical Stockwell transform (LCST) and obtain its basic properties along with the inner product relation and reconstruction formula. We characterize the range of the transform and show that its range is the reproducing kernel Hilbert space. We also develop a multiresolution analysis (MRA) associated with the proposed transform together with the construction of the orthonormal basis for .

Error bounds for kernel-based approximations of the Koopman operator

We consider the data-driven approximation of the Koopman operator for stochastic differential equations on reproducing kernel Hilbert spaces (RKHS). Our focus is on the estimation error if the data are collected from long-term ergodic simulations. We derive both an exact expression for the variance of the kernel cross-covariance operator, measured in the Hilbert-Schmidt norm, and probabilistic bounds for the finite-data estimation error. Moreover, we derive a bound on the prediction error of observables in the RKHS using a finite Mercer series expansion. Further, assuming Koopman-invariance of the RKHS, we provide bounds on the full approximation error. Numerical experiments using the Ornstein-Uhlenbeck process illustrate our results.

Kernel embedding of measures and low-rank approximation of integral operators

We describe a natural coisometry from the Hilbert space of all Hilbert-Schmidt operators on a separable reproducing kernel Hilbert space (RKHS)H\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\hbox { (RKHS)}\\, \\mathcal {H}$$\\end{document} and onto the RKHS G\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {G}$$\\end{document} associated with the squared-modulus of the reproducing kernel of H\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {H}$$\\end{document}. Through this coisometry, trace-class integral operators defined by general measures and the reproducing kernel of H\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {H}$$\\end{document} are isometrically represented as potentials in G\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {G}$$\\end{document}, and the quadrature approximation of these operators is equivalent to the approximation of integral functionals on G\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {G}$$\\end{document}. We then discuss the extent to which the approximation of potentials in RKHSs with squared-modulus kernels can be regarded as a differentiable surrogate for the characterisation of low-rank approximation of integral operators.

Kreĭn twin support vector machines for imbalanced data classification

Conventional classification assumes a balanced sample distribution among classes. However, such a premise leads to biased performance over the majority class (with the highest number of instances). The Twin Support Vector Machines (TWSVM) obtained great prominence due to their low computational burden compared to the standard SVM. Besides, traditional machine learning seeks methods whose solution depends on a convex problem or semi-positive definite similarity matrices. Yet, this kind of matrix cannot adequately represent many real-world applications. The above defines the need to use non-negative measures as an indefinite function in a Reproducing Kernel Kreĭn Space (RKKS). This paper proposes a novel approach called Kreĭn Twin Support Vector Machines (KTSVM), which appropriately incorporates indefinite kernels within a TWSVM-based gradient optimization. To code pertinent input patterns within an imbalanced data discrimination, our KTSVM employs an implicit mapping to a RKKS. Also, our approach takes advantage of the TWSVM scheme’s benefits by creating two parallel hyperplanes. This promotes the KTSVM optimization in a gradient-descent framework. Results obtained on synthetic and real-world datasets demonstrate that our solution performs better in terms of imbalanced data classification than state-of-the-art techniques.

Learning nonparametric ordinary differential equations from noisy data

Learning nonparametric systems of Ordinary Differential Equations (ODEs) x˙=f(t,x) from noisy data is an emerging machine learning topic. We use the well-developed theory of Reproducing Kernel Hilbert Spaces (RKHS) to define candidates for f for which the solution of the ODE exists and is unique. Learning f consists of solving a constrained optimization problem in an RKHS. We propose a penalty method that iteratively uses the Representer theorem and Euler approximations to provide a numerical solution. We prove a generalization bound for the L2 distance between x and its estimator. Experiments are provided for the FitzHugh–Nagumo oscillator, the Lorenz system, and for predicting the Amyloid level in the cortex of aging subjects. In all cases, we show competitive results compared with the state-of-the-art.