Reproducing Kernel Hilbert Space Research Articles

Polynomial-based kernel reproduced gradient descent for stochastic optimization

Stochastic gradient descent (SGD) is widely applied to machine learning tasks. However, the existence of gradient variance degrades the convergence performance of SGD, which cannot be tackled well for tasks with one-time data sampling per sample. In this paper, a kernel reproduced gradient descent algorithm is proposed where the gradient of the risk function is estimated by utilizing the Reproducing Kernel Hilbert Space (RKHS) theory. Meanwhile, to ensure a low storage requirement, a kernel function linearization strategy is used to design the polynomial form of the kernel learning algorithm. It is shown that the error resulting from linearization decreases as the model parameter size increases, and the proposed polynomial-like kernel reproduced gradient descent (PKRGD) algorithm can converge faster than SGD. From experiments on models such as the latest image matching model, LightGlue, it is verified that the reproduced gradients have lower variances, and existing optimizers using the reproduced gradients can further accelerate model training.

An integrated approach for mechanical fault diagnosis using maximum mean square discrepancy representation and CNN-based mixed information fusion

Bearings are an essential component of modern industry and cross-domain diagnosis of bearings holds significant importance. However, in practical applications, issues such as insufficient training data and differences between equipment present challenges. Transfer learning has become one of the effective methods to address these problems. This article proposes a fault diagnosis method that combines maximum mean square discrepancy (MMSD) measurement with the convolutional neural network (CNN) with mixed information (MIXCNN). MIXCNN enhances spatial position discrimination through deep convolution and achieves cross-channel information interaction using traditional convolution. The introduction of residual connections reduces information loss, while increasing network depth focuses on highly distinguishable features. MMSD constructs a metric that comprehensively reflects the mean and variance information of data samples in the reproducing kernel Hilbert space, thereby enhancing domain confusion. Experimental results show that this method achieves high diagnostic accuracy in various transfer tasks, with a maximum accuracy of 99.29%, providing reliable support for bearing fault diagnosis.

On the Berezin Number of Operators on the Reproducing Kernel of Hilbert Space and Related Problems

On the Berezin Number of Operators on the Reproducing Kernel of Hilbert Space and Related Problems

Q\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{q}$$\\end{document}-rational Functions and Interpolation with Complete Nevanlinna–Pick Kernels

In this paper we introduce the concept of matrix-valued q-rational functions. In comparison to the classical case, we give different characterizations with principal emphasis on realizations and discuss algebraic manipulations. We also study the concept of Schur multipliers and complete Nevanlinna–Pick kernels in the context of q-deformed reproducing kernel Hilbert spaces and provide first applications in terms of an interpolation problem using Schur multipliers and complete Nevanlinna–Pick kernels.

Functional quantile regression with missing data in reproducing kernel Hilbert space

We, in this article, focus on functional partially linear quantile regression, where the observations are missing at random, which allows the response or covariates or response and covariates simultaneously missing. Estimation of the unknown function is done based on reproducing kernel method. Under suitable assumptions, we discuss consistency with rates of the estimators, and establish asymptotic normality of the estimator for the parameter. At the same time, we study hypothesis test of the parameter, and prove asymptotic distributions of restricted estimators of the parameter and test statistic under null hypothesis and local alternative hypothesis, respectively. Also, we study variable selection of the linear part of the model. By simulation and real data, finite sample performance of the proposed methods is analyzed.

Approximating Reproducing Kernel Hilbert Space Functions by Bernstein Operators

Motivated by kernel methods in machine learning theory, we study the uniform approximation of functions from reproducing kernel Hilbert spaces by Bernstein operators. Rates of approximation are provided in terms of the function norm in the reproducing kernel Hilbert space. A case study of contracting properties of the Bernstein operators is also presented.

A novel numerical approach for the third order Emden–Fowler type equations

AbstractThis article aims to achieve robust numerical results by applying the Chebyshev reproducing kernel method without homogenizing the initial‐boundary conditions of the Emden–Fowler (E‐F) equation, thereby introducing a new perspective to the literature. A novel numerical approach is presented for solving the initial‐boundary value problem of third‐order E‐F equations using Chebyshev reproducing kernel theory. Unlike previous applications, which were confined to homogeneous initial‐boundary value problems or required homogenization, the proposed method is effective for both homogeneous and nonhomogeneous cases. To handle the initial‐boundary conditions of the E‐F equations, additional basis functions are introduced rather than imposing conditions on the reproducing kernel Hilbert space. The method's effectiveness is demonstrated through five examples, which validate the theoretical analysis. Overall, the results emphasize the method's efficiency.

Machine learning in viscoelastic fluids via energy-based kernel embedding

The ability to measure differences in collected data is of fundamental importance for quantitative science and machine learning, motivating the establishment of metrics grounded in physical principles. In this study, we focus on the development of such metrics for viscoelastic fluid flows governed by a large class of linear and nonlinear stress models. To do this, we introduce energy-compatible families of kernel functions corresponding to a given viscoelastic stress model. Each kernel implicitly embeds flowfield snapshots into a Reproducing Kernel Hilbert Space (RKHS) in which distances and angles are computed and whose squared norm equals the total mechanical energy. Additionally, we present a solution to the preimage problem for these kernels, enabling accurate reconstruction of flowfields from their RKHS representations. Through numerical experiments on an unsteady viscoelastic lid-driven cavity flow, we demonstrate the utility of energy-compatible kernels for extracting energetically-dominant coherent structures in viscoelastic flows across a range of Reynolds and Weissenberg numbers. Specifically, the features extracted by Kernel Principal Component Analysis (KPCA) of flowfield snapshots using energy-compatible kernel functions yield reconstructions with superior accuracy in terms of mechanical energy compared to conventional methods such as ordinary Principal Component Analysis (PCA) with naïvely-defined state vectors or KPCA with ad-hoc choices of kernel functions. Our findings underscore the importance of principled choices of metrics in both scientific and machine learning investigations of complex fluid systems.

Ab initio study of the interaction between the charged system (FH+) and the He atom

We investigated the molecular structure and spectroscopic properties of the charged system (FH+) in interaction with a helium atom using an ab initio quantum chemistry approach in Jacobi coordinates. Our study focused on the ground state X2Π of (FH+) with various theoretical methods including UCCSD, UCCSD(T) and UCCSD(T)-F12, alongside basis sets like aug-cc-pVnZ (n = T, Q, 5, and 6) and cc-pVQZ-F12. We evaluated how these methods and basis sets affect the minimum energies We assessed the impact of the methods, basis sets, and extrapolation approaches on the minimum energies. Potential energy surfaces (PES) were generated using the correlated UCCSD(T)-F12 method with cc-pVQZ-F12 for hydrogen and fluorine, and cc-pVQZ-F12/optri for helium. These surfaces, considering spin-orbit coupling, are degenerate for linear geometries (θ=0° and θ=180°) of the (FH+)-He complex and correlate with the doubly degenerate X2Π state of (FH+). Our results reveal a strong anisotropic character at short and intermediate distances and a quasi-isotropic character upon dissociation into FH+ and He. We compared the spectroscopic parameters of the (FH+)-He complex with existing theoretical data, finding that the linear arrangement exhibits the highest stability, consistent with previous results for similar complexes. This conclusion is supported by the Milliken atomic charge distribution and a three-dimensional map of the Molecular Electrostatic Potential. Additionally, we used the LEVEL program, to calculate the bound rovibronic levels of the (FH+)-He complex for angular momentum values Ω=1/2 and Ω=3/2. Utilizing our ab initio results and a general interpolation approach based on the Reproducing Kernel Hilbert Space method, we generated contour maps illustrating the analytical potential for the (FH+)-He system. These findings will be employed to study the stabilities, geometries, energetics, and structures of the FH+ charged system within helium clusters.

Simultaneous predictive bands for functional time series using minimum entropy sets

Functional Time Series (FTS) are sequences of dependent random elements taking values on some functional space. Most of the research on this domain focuses on producing a predictor able to forecast the next function, having observed a part of the sequence. For this, the Autoregressive Hilbertian process is a suitable framework. Here, we address the problem of constructing simultaneous predictive confidence bands for a stationary FTS. The method is based on an entropy measure for stochastic processes. To construct predictive bands, we use a Reproducing Kernel Hilbert Space (RKHS) to represent the functions and a functional bootstrap procedure that allows us to estimate the prediction law and a Reproducing Kernel Hilbert Spaces (RKHS) to represent the functions, considering then the basis associated to the reproducing kernel. We then classify the points on the projected space according to those that belong to the minimum entropy set (MES) and those that do not. We map the minimum entropy set back to the functional space and construct a band using the regularity property of the RKHS. The proposed methodology is illustrated through artificial and real data sets.

Evaluation of deep learning for predicting rice traits using structural and single-nucleotide genomic variants

BackgroundStructural genomic variants (SVs) are prevalent in plant genomes and have played an important role in evolution and domestication, as they constitute a significant source of genomic and phenotypic variability. Nevertheless, most methods in quantitative genetics focusing on crop improvement, such as genomic prediction, consider only Single Nucleotide Polymorphisms (SNPs). Deep Learning (DL) is a promising strategy for genomic prediction, but its performance using SVs and SNPs as genetic markers remains unknown.ResultsWe used rice to investigate whether combining SVs and SNPs can result in better trait prediction over SNPs alone and examine the potential advantage of Deep Learning (DL) networks over Bayesian Linear models. Specifically, the performances of BayesC (considering additive effects) and a Bayesian Reproducible Kernel Hilbert space (RKHS) regression (considering both additive and non-additive effects) were compared to those of two different DL architectures, the Multilayer Perceptron, and the Convolution Neural Network, to explore their prediction ability by using various marker input strategies. We found that exploiting structural and nucleotide variation slightly improved prediction ability on complex traits in 87% of the cases. DL models outperformed Bayesian models in 75% of the studied cases, considering the four traits and the two validation strategies used. Finally, DL systematically improved prediction ability of binary traits against the Bayesian models.ConclusionsOur study reveals that the use of structural genomic variants can improve trait prediction in rice, independently of the methodology used. Also, our results suggest that Deep Learning (DL) networks can perform better than Bayesian models in the prediction of binary traits, and in quantitative traits when the training and target sets are not closely related. This highlights the potential of DL to enhance crop improvement in specific scenarios and the importance to consider SVs in addition to SNPs in genomic selection.

A reproducing kernel Hilbert space approach to singular local stochastic volatility McKean–Vlasov models

Motivated by the challenges related to the calibration of financial models, we consider the problem of numerically solving a singular McKean–Vlasov equation dXt=σ(t,Xt)XtvtE[vt|Xt]dWt,\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$ d X_{t}= \\sigma (t,X_{t}) X_{t} \\frac{\\sqrt{v}_{t}}{\\sqrt{\\mathbb{E}[v_{t}|X_{t}]}}dW_{t}, $$\\end{document} where W\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$W$\\end{document} is a Brownian motion and v\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$v$\\end{document} is an adapted diffusion process. This equation can be considered as a singular local stochastic volatility model. While such models are quite popular among practitioners, its well-posedness has unfortunately not yet been fully understood and in general is possibly not guaranteed at all. We develop a novel regularisation approach based on the reproducing kernel Hilbert space (RKHS) technique and show that the regularised model is well posed. Furthermore, we prove propagation of chaos. We demonstrate numerically that a thus regularised model is able to perfectly replicate option prices coming from typical local volatility models. Our results are also applicable to more general McKean–Vlasov equations.

Advancing differential equation solutions: A novel Laplace-reproducing kernel Hilbert space method for nonlinear fractional Riccati equations

Fractional ordinary differential equations are crucial for modeling various phenomena in engineering and physics. This study aims to enhance solution methodologies by combining the Laplace transform with the reproducing kernel Hilbert space method (RKHSM). This integration leads to a more effective approach compared to the classical RKHSM. We apply the Laplace-reproducing kernel Hilbert space method (L-RKHSM) to develop novel numerical solutions for nonlinear fractional Riccati differential equations. The L-RKHSM systematically produces both approximate and analytic solutions in series form. We present detailed results for four illustrative examples, showcasing the superior performance of the L-RKHSM over traditional methods. This innovative approach not only advances our understanding of nonlinear fractional ordinary differential equations but also demonstrates its effectiveness through significantly improved outcomes in various applications.

Effect of dimensionality on convergence rates of kernel ridge regression estimator

Despite the curse of dimensionality, kernel ridge regression often exhibits good performance in practical applications, even when the dimension is moderately large. However, it has been shown that kernel ridge regression cannot be free from the curse of dimensionality. Until now, the literature on kernel ridge regression has suggested that the gap between theory and practice in relation to dimensionality has not narrowed. In this study, we first investigate when the influence of dimensionality does not significantly affect the convergence rate of the kernel ridge regression. Specifically, we study the convergence rate of L2 and L∞ risks for the kernel ridge estimator, with a focus on reproducing kernel Hilbert space (RKHS) generated by a product kernel. We show that the univariate optimal convergence rate up to a logarithmic factor in L2 and L∞ risks can be achieved by controlling the size of the RKHS. The result of a numerical study confirms our theoretical findings.

M-estimation for varying coefficient models with a functional response in a reproducing kernel Hilbert space

M-estimation for varying coefficient models with a functional response in a reproducing kernel Hilbert space

The Approximate Integrals by the Reproducing Kernel Method on the Elliptical Region Eₙ In the Space Rⁿ

In this paper, we applied the reproducing kernel method on elliptical region, we establish the formulas of the reproducing kernel from several degrees and generalized those formulas whatever the dimension of space, we obtained all surfaces, points, and their constants for compensation in the cubature formula, tables are included to show our results and examples to show the approximation solution in this method.

Uni-variable cross-spectral densities

We introduce a novel class of planar, scalar sources whose cross-spectral density is a function of a single complex variable. For a typical pair of source points, the modulus and argument of such a variable equal the product of their radial coordinates and the difference of their angular coordinates, respectively. As functions of a single complex variable, the corresponding cross-spectral densities (CSD) are expandable in power series in their convergence domain, and a virtually infinite number of source models can be devised. All such sources are shown to have vortex fields as modes. The closed analytical form of these uni-variable CSDs makes it possible to evaluate in a simple way the significant quantities characterizing the sources and the ones of the fields they radiate. The basic tool leading to the definition of the uni-variable CSDs are the reproducing-kernel Hilbert spaces. As an example, the CSD derived from the so called Szegö kernel is studied in some detail, and its features are derived, together with of those of the radiated field in the far zone.

High-Energy Reaction Dynamics of N3.

The atom-exchange and atomization dissociation dynamics for the N(4S) + N2(1Σg+) reaction are studied using a reproducing kernel Hilbert space (RKHS)-based, global potential energy surface (PES) at the MRCI-F12/aug-cc-pVTZ-F12 level of theory (MRCI, multireference configuration interaction). For the atom exchange reaction (NANB + NC → NANC + NB), computed thermal rates and their temperature dependence from quasi-classical trajectory (QCT) simulations agree to within error bars with the available experiments. Companion QCT simulations using a recently published CASPT2-based PES confirm these findings. For the atomization reaction, leading to three N(4S) atoms, the computed rates from the RKHS-PES overestimate the experimentally reported rates by 1 order of magnitude, whereas those from the permutationally invariant polynomial (PIP)-PES agree favorably, and the T dependence of both computations is consistent with the experiment. These differences can be traced back to the different methods and basis sets used. The lifetime of the metastable N3 molecule is estimated to be ∼200 fs depending on the initial state of the reactants. Finally, neural-network-based exhaustive state-to-distribution models are presented using both PESs for the atom exchange reaction. These models will be instrumental for a broader exploration of the reaction dynamics of air.

Sample path moderate deviations for shot noise processes in the high intensity regime

We study the sample-path moderate deviation principle (MDP) for shot noise processes in the high intensity regime. The shot noise processes have a renewal arrival process, non-stationary noises (with arrival-time dependent distributions) and a general shot response function of the noises. The rate function in the MDP exhibits a memory phenomenon in this asymptotic regime, which is in contrast with that in the conventional time–space scaling regime. To prove the sample-path MDP, we first establish that this is equivalent to establishing the sample-path MDP of another process that is easier to study. We prove its finite-dimensional MDP and then establish the exponential tightness under the Skorohod J1 topology. This results in the sample-path MDP in D under the Skorohod J1 topology with a rate function that is derived from the rate function in the finite-dimensional MDP using the tools of reproducing kernel Hilbert space. In the proofs, because of the non-stationarity of shot noise process, we establish a new exponential maximal inequality and use it to prove exponential tightness and the aforementioned equivalence.

Solving an electrically conducting nanofluid over an impermeable stretching cylinder problem with a spectral reproducing kernel method

In this paper, a nonlinear mechanical system of ordinary differential equations (ODEs) with multi-point boundary conditions is considered by a novel type of reproducing kernel Hilbert space method (RKHSM). To begin, we define the unknown variables in terms of the reproducing kernel function. The roots of the Shifted Chebyshev polynomials (SCPs) are then utilized to collocate the resulting system. Finally, Newton’s iterative method is employed to find the unknown expansion coefficients. The solutions of this system of equations, which arise from the flow of an electrically conducting nanofluid over an impermeable stretching cylinder, are numerically analyzed, and convergence analysis is discussed to demonstrate the reliability of the presented method (PM). Tables and figures are provided to further discuss the solutions and assess the effectiveness of the method in comparison to other techniques in the literature.