Reproducing Kernel Hilbert Space Research Articles

Solving an electrically conducting nanofluid over an impermeable stretching cylinder problem with a spectral reproducing kernel method

AbstractIn this paper, a nonlinear mechanical system of ordinary differential equations (ODEs) with multi-point boundary conditions is considered by a novel type of reproducing kernel Hilbert space method (RKHSM). To begin, we define the unknown variables in terms of the reproducing kernel function. The roots of the Shifted Chebyshev polynomials (SCPs) are then utilized to collocate the resulting system. Finally, Newton’s iterative method is employed to find the unknown expansion coefficients. The solutions of this system of equations, which arise from the flow of an electrically conducting nanofluid over an impermeable stretching cylinder, are numerically analyzed, and convergence analysis is discussed to demonstrate the reliability of the presented method (PM). Tables and figures are provided to further discuss the solutions and assess the effectiveness of the method in comparison to other techniques in the literature.

Data‐driven nonlinear state observation for controlled systems: A kernel method and its analysis

AbstractThis work proposes a data‐driven state observation algorithm for nonlinear dynamical systems, when the true state trajectory is not measurable and hence the states information needs to be reconstructed from input and output measurements. Such a reduction is formed by kernel canonical correlation analysis (KCCA), which (i) implicitly maps the available input–output data into a higher‐dimensional feature space, namely the reproducing kernel Hilbert space (RKHS); (ii) finds a projection of the past input–output data and a projection of the future input–output data with maximal correlation; and (iii) identifies the projected inputs and outputs, namely the canonical variates, as the observed states. We adopt a least squares support vector machine (LS‐SVM) formulation for KCCA, which imposes regularization on the vectors that specify the projections and is amenable to convex optimization. We prove theoretically that, based on the statistical consistency of KCCA, the observed states determined by the proposed state observer has a guaranteed correlativity with the actual states (when properly transformed). Furthermore, such observed states, when supplemented with the information of succeeding inputs, can be used to predict the succeeding outputs with guaranteed upper bound on the prediction error. Case studies are performed on two numerical examples and an exothermic continuously stirred tank reactor (CSTR).

Exponentially improved efficient machine learning for quantum many-body states with provable guarantees

Solving the ground state and the ground-state properties of quantum many-body systems is generically a hard task for classical algorithms. For a family of Hamiltonians defined on an m-dimensional space of physical parameters, the ground state and its properties at an arbitrary parameter configuration can be predicted via a machine learning protocol up to a prescribed prediction error ɛ, provided that a sample set (of size N) of the states can be efficiently prepared and measured. In a recent work [Huang , ], a rigorous guarantee for such a generalization was proved. Unfortunately, an exponential scaling for the provable sample complexity, N=mO(1ɛ), was found to be universal for generic gapped Hamiltonians. This result applies to the situation where the dimension of the parameter space is large, while the scaling with the accuracy is not an urgent factor, not entering the realm of more precise learning and prediction. In this work, we consider an alternative relevant scenario, where the effective dimension m is a finite, not necessarily large constant, while the scaling with the prediction error becomes the central concern. By jointly preserving the fundamental properties of density matrices in the learning protocol and utilizing the continuity of quantum states in the parameter range of interest, we rigorously obtain a polynomial sample complexity for predicting quantum many-body states and their properties, with respect to the prediction error ɛ and the number of qubits, n, with N=poly(ɛ−1,n,ln1δ), where poly denotes a polynomial function, and (1−δ) is the probability of success. Moreover, if restricted to learning local quantum-state properties, the number of samples can be further reduced to N=poly(ɛ−1,lnnδ). Numerical demonstrations confirm our findings, and an alternative approach utilizing statistical learning theory with reproducing kernel Hilbert space achieves consistent results. The mere continuity assumption indicates that our results are not restricted to gapped Hamiltonian systems and properties within the same phase. Published by the American Physical Society 2024

Sample path moderate deviations for shot noise processes in the high intensity regime

We study the sample-path moderate deviation principle (MDP) for shot noise processes in the high intensity regime. The shot noise processes have a renewal arrival process, non-stationary noises (with arrival-time dependent distributions) and a general shot response function of the noises. The rate function in the MDP exhibits a memory phenomenon in this asymptotic regime, which is in contrast with that in the conventional time-space scaling regime. To prove the sample-path MDP, we first establish that this is equivalent to establishing the sample-path MDP of another process that is easier to study. We prove its finite-dimensional MDP and then establish the exponential tightness under the Skorohod J1 topology. This results in the sample-path MDP in D under the Skorohod J1 topology with a rate function that is derived from the rate function in the finite-dimensional MDP using the tools of reproducing kernel Hilbert space. In the proofs, because of the non-stationarity of shot noise process, we establish a new exponential maximal inequality and use it to prove exponential tightness and the aforementioned equivalence.

Sparse kernel sufficient dimension reduction

The sufficient dimension reduction (SDR) with sparsity has received much attention for analysing high-dimensional data. We study a nonparametric sparse kernel sufficient dimension reduction (KSDR) based on the reproducing kernel Hilbert space, which extends the methodology of the sparse SDR based on inverse moment methods. We establish the statistical consistency and efficient estimation of the sparse KSDR under the high-dimensional setting where the dimension diverges as the sample size increases. Computationally, we introduce a new nonconvex alternating directional method of multipliers (ADMM) to solve the challenging sparse SDR and propose the nonconvex linearised ADMM to solve the more challenging sparse KSDR. We study the computational guarantees of the proposed ADMMs and show an explicit iteration complexity bound to reach the stationary solution. We demonstrate the finite-sample properties in simulation studies and a real application.

SAR target recognition through adaptive kernel sparse representation model based on local contrast perception

Sparse representation-based methods have shown promising prospects in synthetic aperture radar (SAR) target recognition due to their robustness under complex application conditions. However, these methods are still susceptible to the feature representation of SAR images. In this paper, we propose a new SAR target recognition method using an adaptive kernel sparse representation model based on local contrast perception (LCP). The LCP method combines a specially designed random projection matrix with two-dimensional filtering to capture extensive intensity contrasts between local regions in SAR images. A response map reconstruction technique shows that LCP can obtain discriminative properties between target and background regions and improve feature representation. The LCP features are then transformed into the kernel representation to improve linear separability by defining a reproducing kernel Hilbert space (RKHS) and using a nonlinear kernel function. The kernel representation is utilized to construct a new kernel sparse representation model for classification. Furthermore, neighborhood analysis (NA) is proposed to establish the relationship between the parameter and feature distribution, with an adaptive regularization parameter selection strategy to mitigate the parameter sensitivity. Experiments conducted on the moving and stationary target acquisition and recognition dataset demonstrate the superiority and robustness of the proposed method under different application conditions.

Tropical Reproducing Kernels and Optimization

Tropical Reproducing Kernels and Optimization

Kernel Angle Dependence Measures in Metric Spaces

Measuring and testing dependence between data in separable metric spaces is of great importance in modern statistics. Most existing work relied on the distance between random variables, which inevitably required the moment conditions to guarantee the distance is well-defined. Based on the geometry element “angle”, we develop a novel class of nonlinear dependence measures for data in metric space that can avoid such conditions. Specifically, by making use of the reproducing kernel Hilbert space equipped with Gaussian measure, we introduce kernel angle covariances that can be applied to various types of data, including low dimensional vector, high dimensional vectors, non-Euclidean data like symmetric positive definite matrices, and compositional data. We estimate kernel angle covariances based on U-statistic and establish the corresponding independence tests via gamma approximation. Our kernel angle independence tests, imposing no-moment conditions on kernels, are robust with heavy-tailed random variables. We conduct comprehensive simulation studies and apply our proposed methods to a facial recognition task. Our kernel angle covariances-based tests show remarkable performances in dealing with image data. All the codes and proofs are included in the Supplementary Materials.

A novel numerical approach to solutions of fractional Bagley-Torvik equation fitted with a fractional integral boundary condition

Abstract In this work, we present a sophisticated operating algorithm, the reproducing kernel Hilbert space method, to investigate the approximate numerical solutions for a specific class of fractional Begley-Torvik equations (FBTE) equipped with fractional integral boundary condition. Such fractional integral boundary condition allows us to understand the non-local behavior of FBTE along with the given domain. The algorithm methodology depends on creating an orthonormal basis based on reproducing kernel function that satisfies the constraint boundary conditions so that the solution is finally formulated in the form of a uniformly convergent series in ϖ 3 [ a , b ] {\varpi }_{3}\left[a,b] . From a numerical point of view, some illustrative examples are provided to determine the appropriateness of algorithm design and the effect of using non-classical boundary conditions on the behavior of solutions approach.

Gaussian random field approximation via Stein's method with applications to wide random neural networks

We derive upper bounds on the Wasserstein distance (W1), with respect to sup-norm, between any continuous Rd valued random field indexed by the n-sphere and the Gaussian, based on Stein's method. We develop a novel Gaussian smoothing technique that allows us to transfer a bound in a smoother metric to the W1 distance. The smoothing is based on covariance functions constructed using powers of Laplacian operators, designed so that the associated Gaussian process has a tractable Cameron-Martin or Reproducing Kernel Hilbert Space. This feature enables us to move beyond one dimensional interval-based index sets that were previously considered in the literature. Specializing our general result, we obtain the first bounds on the Gaussian random field approximation of wide random neural networks of any depth and Lipschitz activation functions at the random field level. Our bounds are explicitly expressed in terms of the widths of the network and moments of the random weights. We also obtain tighter bounds when the activation function has three bounded derivatives.

Recovery of bandlimited graph signals based on the reproducing kernel Hilbert space

Signal recovery on graphs is attracting more and more attentions. Based on the smoothness assumption, the signal recovery problem can be formulated as an unconstrained optimization model. Although the model can be solved analytically, the computational cost is too expensive. In this paper, to reduce the computational complexity, we skillfully construct a kernel function, such that the smoothness item is equivalent to the norm of the signal modulo a constant in the reproducing kernel Hilbert space. Based on this, by using the Representer Theorem of reproducing kernel Hilbert space, the optimization problem for recovering the signal from a given bandlimited space, can be solved in a lower-dimensional subspace corresponding to the known labeled set. Experiments for the recovery of artificial graph signals in GSPBOX and real data in the recommendation system show very encouraging results.

Join multiple Riemannian manifold representation and multi‐kernel non‐redundancy for image clustering

AbstractImage clustering has received significant attention due to the growing importance of image recognition. Researchers have explored Riemannian manifold clustering, which is capable of capturing the non‐linear shapes found in real‐world datasets. However, the complexity of image data poses substantial challenges for modelling and feature extraction. Traditional methods such as covariance matrices and linear subspace have shown promise in image modelling, and they are still in their early stages and suffer from certain limitations. However, these include the uncertainty of representing data using only one Riemannian manifold, limited feature extraction capacity of single kernel functions, and resulting incomplete data representation and redundancy. To overcome these limitations, the authors propose a novel approach called join multiple Riemannian manifold representation and multi‐kernel non‐redundancy for image clustering (MRMNR‐MKC). It combines covariance matrices with linear subspace to represent data and applies multiple kernel functions to map the non‐linear structural data into a reproducing kernel Hilbert space, enabling linear model analysis for image clustering. Additionally, the authors use matrix‐induced regularisation to improve the clustering kernel selection process by reducing redundancy and assigning lower weights to identical kernels. Finally, the authors also conducted numerous experiments to evaluate the performance of our approach, confirming its superiority to state‐of‐the‐art methods on three benchmark datasets.

On Berezin norm and Berezin number inequalities for sum of operators

Abstract The aim of this study is to obtain several inequalities involving the Berezin number and the Berezin norm for various combinations of operators acting on a reproducing kernel Hilbert space. First, we present some bounds regarding the Berezin number associated with W * Q + W * Q ′ {W}^{* }Q+{W}^{* }Q^{\prime} , where W W , Q Q , and Q ′ Q^{\prime} are three bounded linear operators. Next, several Berezin norm and Berezin number inequalities for the sum of n n operators are established.

Stability of convergence rates: kernel interpolation on non-Lipschitz domains

Abstract Error estimates for kernel interpolation in Reproducing Kernel Hilbert Spaces usually assume quite restrictive properties on the shape of the domain, especially in the case of infinitely smooth kernels like the popular Gaussian kernel. In this paper we prove that it is possible to obtain convergence results (in the number of interpolation points) for kernel interpolation for arbitrary domains $\varOmega \subset{\mathbb{R}} ^{d}$, thus allowing for non-Lipschitz domains including e.g., cusps and irregular boundaries. Especially we show that, when going to a smaller domain $\tilde{\varOmega } \subset \varOmega \subset{\mathbb{R}} ^{d}$, the convergence rate does not deteriorate—i.e., the convergence rates are stable with respect to going to a subset. We obtain this by leveraging an analysis of greedy kernel algorithms. The impact of this result is explained on the examples of kernels of finite as well as infinite smoothness. A comparison to approximation in Sobolev spaces is drawn, where the shape of the domain $\varOmega $ has an impact on the approximation properties. Numerical experiments illustrate and confirm the analysis.

Equivalence between constrained optimal smoothing and Bayesian estimation

In this paper, we extend the correspondence between Bayesian estimation and optimal smoothing in a Reproducing Kernel Hilbert Space (RKHS) by adding convex constraints to the problem. Through a sequence of approximating Hilbertian subspaces and a discretised model, we prove that the Maximum a posteriori (MAP) of the posterior distribution is exactly the optimal constrained smoothing function in the RKHS. This paper can be read as a generalisation of the paper Kimeldorf and Wahba (1970), where it is proved that the optimal smoothing solution is the mean of the posterior distribution. Synthetic and real data studies confirm the correspondence established in this paper.

Kernel-algorithms in frame-approximations

With view to applications, we present here general classes of non-orthogonal expansions in Hilbert space. The main purpose of our paper is a new approach to design of algorithms of Kaczmarz type in the framework of operators in Hilbert space. Our work includes a diverse list of optimization problems, new Karhunen–Loève transforms, and Principal Component Analysis (PCA) for digital images. A key feature of our algorithms is our use of recursive systems of projection operators. For this we also make use of specific reproducing kernel Hilbert spaces, kernel factorizations, and finite-dimensional approximations. Our projection algorithms are designed with view to maximum likelihood solutions, minimization of “cost” problems, identification of principal components, and data-dimension reduction.

On the convergence of gradient descent for robust functional linear regression

Functional data analysis offers a set of statistical methods concerned with extracting insights from intrinsically infinite-dimensional data and has attracted considerable amount of attentions in the past few decades. In this paper, we study robust functional linear regression model with a scalar response and a functional predictor in the framework of reproducing kernel Hilbert spaces. A gradient descent algorithm with early stopping is introduced to solve the corresponding empirical risk minimization problem associated with robust loss functions. By appropriately selecting the early stopping rule and the scaling parameter of the robust losses, the convergence of the proposed algorithm is established when the response variable is bounded or satisfies a moment condition. Explicit learning rates with respect to both estimation and prediction error are provided in terms of regularity of the regression function and eigenvalue decay rate of the integral operator induced by the reproducing kernel and covariance function.

Reproducing property of bounded linear operators and kernel regularized least square regressions

We consider bounded linear operators from the view of functional reproducing property. For some bounded linear operators associated with orthogonal polynomials we define an inner product space associated with a kernel constructed with orthogonal polynomials, show that it is a functional reproducing kernel Hilbert space (FRKHS) associated with these bounded linear operators and give decay rate for the best FRKHS approximation with a [Formula: see text]-functional associated with the FRKHS. On this basis, we provide a learning rate for kernel regularized regression whose hypothesis space is the defined FRKHS. As applications, we define some concrete FRKHSs associated with polynomial operators such as the Bernstein–Durrmeyer operators, the de la Vallée Poussin operators on both the unit sphere [Formula: see text] and the unit ball [Formula: see text]. We show that these polynomial operators have reproducing property with respect to the corresponding concrete FRKHSs and show the learning rate for the kernel regularized regression. In short, we provide a way of constructing FRKHS operators with Fourier multipliers and show a learning framework from the view of operator approximation.

Propagating uncertainty through system dynamics in reproducing kernel Hilbert space

We present a data-driven approach to propagate uncertainty in initial conditions through the dynamics of an unknown system in a reproducing kernel Hilbert space (RKHS). The uncertainty in initial conditions is represented through its kernel mean embedding (KME) in the RKHS. For a discrete-time Markovian dynamical system, we utilize the conditional mean embedding (CME) operator to encode the underlying dynamics. Learning in RKHS often incurs prohibitive data storage requirements. To circumvent said limitation, we propose an algorithm to propagate uncertainty via a learned sparse CME operator, and provide theoretical guarantees on the approximation error for the embedded distribution with time. We empirically study our algorithm over illustrative dynamical systems and power systems.

Generalized 𝑞-Fock spaces and structural identities

Using q q -calculus we study a family of reproducing kernel Hilbert spaces which interpolate between the Hardy space and the Fock space. We give characterizations of these spaces in terms of classical operators such as integration and backward-shift operators, and their q q -calculus counterparts. Furthermore, these new spaces allow us to study intertwining operators between classic backward-shift operators and the q-Jackson derivative.