Reproducing Kernel Hilbert Space Research Articles

A Kernel approach for extending nonparametric multivariate analysis of variance in high-dimensional settings

AbstractThe nonparametric multivariate analysis of variance (NPMANOVA) testing procedure has been proven to be a valuable tool for comparing groups. In the present paper, we propose a kernel extension of this technique in order to effectively confront high-dimensionality, a recurrent problem in many fields of science. The new method is called kernel multivariate analysis of variance (KMANOVA). The basic idea is to take advantage of the kernel framework: we propose to project the data from the original data space to a Hilbert space generated by a given kernel function and then perform the NPMANOVA method in the reproducing kernel Hilbert space (RKHS). Dispersion of the embedded points can be measured by the distance induced by the inner product in the RKHS but also by many other distances best suited in high-dimensional settings. For this purpose, we study two promising distances: a Manhattan-type distance and a distance based on an orthogonal projection of the embedded points in the direction of the group centroids. We show that the NPMANOVA method and the KMANOVA method with the induced distance are essentially equivalent. We also show that the KMANOVA method with the other two distances performs considerably better than the NPMANOVA method. We illustrate the advantages of our approach in the context of genetic association studies and demonstrate its usefulness on Alzheimer’s disease data. We also provide a software implementation of the method that is available on GitHub https://github.com/8699vicente/Kmanova.

Banach Spaces of Solenoidal Solutions of the Vector Helmholtz Equation in the Space and Its Associated Reproducing Kernel

Banach Spaces of Solenoidal Solutions of the Vector Helmholtz Equation in the Space and Its Associated Reproducing Kernel

Euler Kernel Mapping for Hyperspectral Image Clustering via Self-Paced Learning

Clustering, as a classical unsupervised artificial intelligence technology, is commonly employed for hyperspectral image clustering tasks. However, most existing clustering methods designed for remote sensing tasks aim to solve a non-convex objective function, which can be optimized iteratively, beginning with random initializations. Consequently, during the learning phase of the clustering model, it may easily fall into bad local optimal solutions and finally hurt the clustering performance. Additionally, prevailing approaches often exhibit limitations in capturing the intricate structures inherent in hyperspectral images and are very sensitive to noise and outliers that widely exist in remote sensing data. To address these issues, we proposed a novel Euler kernel mapping for hyperspectral image clustering via self-paced learning (EKM-SPL). EKM-SPL first employs self-paced learning to learn the clustering model in a meaningful order by progressing samples from easy to complex, which can help to remove bad local optimal solutions. Secondly, a probabilistic soft weighting scheme is employed to measure complexity across the data sample, which makes the optimization process more reasonable. Thirdly, in order to more accurately characterize the intricate structure of hyperspectral images, Euler kernel mapping is used to convert the original data into a reproduced kernel Hilbert space, where the nonlinearly inseparable clusters may become linearly separable. Moreover, we innovatively integrate the coordinate descent technique into the optimization algorithm to circumvent the computational inefficiencies and information loss typically associated with conventional kernel methods. Extensive experiments conducted on classic benchmark hyperspectral image datasets illustrate the effectiveness and superiority of our proposed model.

Feature screening filter for high dimensional survival data in the reproducing kernel Hilbert space

We consider the issue of screening uninformative variables associate with survival outcome in ultra high-dimensional situation. specifically, we focus on Hilbert-Schmits Independent criteria (HSIC) statistic defined in the Reproducing kernel Hilbert Space(RKHS) and use the HSIC as a feature screening approach to avoid from difficulty occurred from the ultra high dimensionality. Generally, it is known that the HSIC can account for the flexible relationship between the outcome and predictor variables. Compared with the typical approaches such as regularization and model free screening approaches, our approach does not require the complex non parametric computation or the complicated numerical optimization, Thereby, our proposed approach effectively and flexibly filters out the uninformative predictor. We will show the advantages of our method in numerical simulation studies and illustrate the usefulness of it by applying to diffuse large-B-cell lymphoma data.

On the classification of function algebras on subvarieties of noncommutative operator balls

We study algebras of bounded noncommutative (nc) functions on unit balls of operator spaces (nc operator balls) and on their subvarieties. Considering the example of the nc unit polydisk we show that these algebras, while having a natural operator algebra structure, might not be the multiplier algebra of any reasonable nc reproducing kernel Hilbert space (RKHS). After examining additional subtleties of the nc RKHS approach, we turn to study the structure and representation theory of these algebras using function theoretic and operator algebraic tools. We show that the underlying nc variety is a complete invariant for the algebra of uniformly continuous nc functions on a homogeneous subvariety, in the sense that two such algebras are completely isometrically isomorphic if and only if the subvarieties are nc biholomorphic. We obtain extension and rigidity results for nc maps between subvarieties of nc operator balls corresponding to injective spaces that imply that a biholomorphism between homogeneous varieties extends to a biholomorphism between the ambient balls, which can be modified to a linear isomorphism. Thus, the algebra of uniformly continuous nc functions on nc operator balls, and even its restriction to certain subvarieties, completely determine the operator space up to completely isometric isomorphism.

A new approach for Solving a nonlinear system of second-order BVPs

In this paper, we introduce a new approach based on the Reproducing Kernel Method (RKM) for solving a nonlinear system of second-order Boundary Value Problems (BVPs) without the Gram-Schmidt orthogonalization process. What motivates us to use the RKM without the Gram-Schmidt orthogonalization process is its easy implementation, elimination of the Gram-Schmidt process, fewer calculations, and high accuracy. Finally, the compatibility of numerical results and theorems demonstrates that the Present method is effective.

Error analysis of kernel/GP methods for nonlinear and parametric PDEs

We introduce a priori Sobolev-space error estimates for the solution of arbitrary nonlinear, and possibly parametric, PDEs that are defined in the strong sense, using Gaussian process and kernel based methods. The primary assumptions are: (1) a continuous embedding of the reproducing kernel Hilbert space of the kernel into a Sobolev space of sufficient regularity; and (2) the stability of the differential operator and the solution map of the PDE between corresponding Sobolev spaces. The proof is articulated around Sobolev norm error estimates for kernel interpolants and relies on the minimizing norm property of the solution. The error estimates demonstrate dimension-benign convergence rates if the solution space of the PDE is smooth enough. We illustrate these points with applications to high-dimensional nonlinear elliptic PDEs and parametric PDEs. Although some recent machine learning methods have been presented as breaking the curse of dimensionality in solving high-dimensional PDEs, our analysis suggests a more nuanced picture: there is a trade-off between the regularity of the solution and the presence of the curse of dimensionality. Therefore, our results are in line with the understanding that the curse is absent when the solution is regular enough.

Invariant subspaces of analytic perturbations

Analytic perturbations are understood here as shifts of the form M z + F M_z + F , where M z M_z is the unilateral shift and F F is a finite rank operator on the Hardy space over the open unit disk. Here the term “a shift” refers to the multiplication operator M z M_z on some analytic reproducing kernel Hilbert space. In this paper, first, a natural class of finite rank operators is isolated for which the corresponding perturbations are analytic, and then a complete classification of invariant subspaces of those analytic perturbations is presented. Some instructive examples and several distinctive properties (like cyclicity, essential normality, hyponormality, etc.) of analytic perturbations are also described.

Learning Hamiltonian dynamics with reproducing kernel Hilbert spaces and random features

A method for learning Hamiltonian dynamics from a limited and noisy dataset is proposed. The method learns a Hamiltonian vector field on a reproducing kernel Hilbert space (RKHS) of inherently Hamiltonian vector fields, and in particular, odd Hamiltonian vector fields. This is done with a symplectic kernel, and it is shown how the kernel can be modified to an odd symplectic kernel to impose the odd symmetry. A random feature approximation is developed for the proposed odd kernel to reduce the problem size. The performance of the method is validated in simulations for three Hamiltonian systems. It is demonstrated that the use of an odd symplectic kernel improves prediction accuracy and data efficiency, and that the learned vector fields are Hamiltonian and exhibit the imposed odd symmetry characteristics.

Polynomial-based kernel reproduced gradient descent for stochastic optimization

Stochastic gradient descent (SGD) is widely applied to machine learning tasks. However, the existence of gradient variance degrades the convergence performance of SGD, which cannot be tackled well for tasks with one-time data sampling per sample. In this paper, a kernel reproduced gradient descent algorithm is proposed where the gradient of the risk function is estimated by utilizing the Reproducing Kernel Hilbert Space (RKHS) theory. Meanwhile, to ensure a low storage requirement, a kernel function linearization strategy is used to design the polynomial form of the kernel learning algorithm. It is shown that the error resulting from linearization decreases as the model parameter size increases, and the proposed polynomial-like kernel reproduced gradient descent (PKRGD) algorithm can converge faster than SGD. From experiments on models such as the latest image matching model, LightGlue, it is verified that the reproduced gradients have lower variances, and existing optimizers using the reproduced gradients can further accelerate model training.

An integrated approach for mechanical fault diagnosis using maximum mean square discrepancy representation and CNN-based mixed information fusion

Bearings are an essential component of modern industry and cross-domain diagnosis of bearings holds significant importance. However, in practical applications, issues such as insufficient training data and differences between equipment present challenges. Transfer learning has become one of the effective methods to address these problems. This article proposes a fault diagnosis method that combines maximum mean square discrepancy (MMSD) measurement with the convolutional neural network (CNN) with mixed information (MIXCNN). MIXCNN enhances spatial position discrimination through deep convolution and achieves cross-channel information interaction using traditional convolution. The introduction of residual connections reduces information loss, while increasing network depth focuses on highly distinguishable features. MMSD constructs a metric that comprehensively reflects the mean and variance information of data samples in the reproducing kernel Hilbert space, thereby enhancing domain confusion. Experimental results show that this method achieves high diagnostic accuracy in various transfer tasks, with a maximum accuracy of 99.29%, providing reliable support for bearing fault diagnosis.

Q\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{q}$$\\end{document}-rational Functions and Interpolation with Complete Nevanlinna–Pick Kernels

In this paper we introduce the concept of matrix-valued q-rational functions. In comparison to the classical case, we give different characterizations with principal emphasis on realizations and discuss algebraic manipulations. We also study the concept of Schur multipliers and complete Nevanlinna–Pick kernels in the context of q-deformed reproducing kernel Hilbert spaces and provide first applications in terms of an interpolation problem using Schur multipliers and complete Nevanlinna–Pick kernels.

Functional quantile regression with missing data in reproducing kernel Hilbert space

We, in this article, focus on functional partially linear quantile regression, where the observations are missing at random, which allows the response or covariates or response and covariates simultaneously missing. Estimation of the unknown function is done based on reproducing kernel method. Under suitable assumptions, we discuss consistency with rates of the estimators, and establish asymptotic normality of the estimator for the parameter. At the same time, we study hypothesis test of the parameter, and prove asymptotic distributions of restricted estimators of the parameter and test statistic under null hypothesis and local alternative hypothesis, respectively. Also, we study variable selection of the linear part of the model. By simulation and real data, finite sample performance of the proposed methods is analyzed.

On the Berezin Number of Operators on the Reproducing Kernel of Hilbert Space and Related Problems

On the Berezin Number of Operators on the Reproducing Kernel of Hilbert Space and Related Problems

Approximating Reproducing Kernel Hilbert Space Functions by Bernstein Operators

Motivated by kernel methods in machine learning theory, we study the uniform approximation of functions from reproducing kernel Hilbert spaces by Bernstein operators. Rates of approximation are provided in terms of the function norm in the reproducing kernel Hilbert space. A case study of contracting properties of the Bernstein operators is also presented.

A novel numerical approach for the third order Emden–Fowler type equations

AbstractThis article aims to achieve robust numerical results by applying the Chebyshev reproducing kernel method without homogenizing the initial‐boundary conditions of the Emden–Fowler (E‐F) equation, thereby introducing a new perspective to the literature. A novel numerical approach is presented for solving the initial‐boundary value problem of third‐order E‐F equations using Chebyshev reproducing kernel theory. Unlike previous applications, which were confined to homogeneous initial‐boundary value problems or required homogenization, the proposed method is effective for both homogeneous and nonhomogeneous cases. To handle the initial‐boundary conditions of the E‐F equations, additional basis functions are introduced rather than imposing conditions on the reproducing kernel Hilbert space. The method's effectiveness is demonstrated through five examples, which validate the theoretical analysis. Overall, the results emphasize the method's efficiency.

Machine learning in viscoelastic fluids via energy-based kernel embedding

The ability to measure differences in collected data is of fundamental importance for quantitative science and machine learning, motivating the establishment of metrics grounded in physical principles. In this study, we focus on the development of such metrics for viscoelastic fluid flows governed by a large class of linear and nonlinear stress models. To do this, we introduce energy-compatible families of kernel functions corresponding to a given viscoelastic stress model. Each kernel implicitly embeds flowfield snapshots into a Reproducing Kernel Hilbert Space (RKHS) in which distances and angles are computed and whose squared norm equals the total mechanical energy. Additionally, we present a solution to the preimage problem for these kernels, enabling accurate reconstruction of flowfields from their RKHS representations. Through numerical experiments on an unsteady viscoelastic lid-driven cavity flow, we demonstrate the utility of energy-compatible kernels for extracting energetically-dominant coherent structures in viscoelastic flows across a range of Reynolds and Weissenberg numbers. Specifically, the features extracted by Kernel Principal Component Analysis (KPCA) of flowfield snapshots using energy-compatible kernel functions yield reconstructions with superior accuracy in terms of mechanical energy compared to conventional methods such as ordinary Principal Component Analysis (PCA) with naïvely-defined state vectors or KPCA with ad-hoc choices of kernel functions. Our findings underscore the importance of principled choices of metrics in both scientific and machine learning investigations of complex fluid systems.

Ab initio study of the interaction between the charged system (FH+) and the He atom

We investigated the molecular structure and spectroscopic properties of the charged system (FH+) in interaction with a helium atom using an ab initio quantum chemistry approach in Jacobi coordinates. Our study focused on the ground state X2Π of (FH+) with various theoretical methods including UCCSD, UCCSD(T) and UCCSD(T)-F12, alongside basis sets like aug-cc-pVnZ (n = T, Q, 5, and 6) and cc-pVQZ-F12. We evaluated how these methods and basis sets affect the minimum energies We assessed the impact of the methods, basis sets, and extrapolation approaches on the minimum energies. Potential energy surfaces (PES) were generated using the correlated UCCSD(T)-F12 method with cc-pVQZ-F12 for hydrogen and fluorine, and cc-pVQZ-F12/optri for helium. These surfaces, considering spin-orbit coupling, are degenerate for linear geometries (θ=0° and θ=180°) of the (FH+)-He complex and correlate with the doubly degenerate X2Π state of (FH+). Our results reveal a strong anisotropic character at short and intermediate distances and a quasi-isotropic character upon dissociation into FH+ and He. We compared the spectroscopic parameters of the (FH+)-He complex with existing theoretical data, finding that the linear arrangement exhibits the highest stability, consistent with previous results for similar complexes. This conclusion is supported by the Milliken atomic charge distribution and a three-dimensional map of the Molecular Electrostatic Potential. Additionally, we used the LEVEL program, to calculate the bound rovibronic levels of the (FH+)-He complex for angular momentum values Ω=1/2 and Ω=3/2. Utilizing our ab initio results and a general interpolation approach based on the Reproducing Kernel Hilbert Space method, we generated contour maps illustrating the analytical potential for the (FH+)-He system. These findings will be employed to study the stabilities, geometries, energetics, and structures of the FH+ charged system within helium clusters.

Simultaneous predictive bands for functional time series using minimum entropy sets

Functional Time Series (FTS) are sequences of dependent random elements taking values on some functional space. Most of the research on this domain focuses on producing a predictor able to forecast the next function, having observed a part of the sequence. For this, the Autoregressive Hilbertian process is a suitable framework. Here, we address the problem of constructing simultaneous predictive confidence bands for a stationary FTS. The method is based on an entropy measure for stochastic processes. To construct predictive bands, we use a Reproducing Kernel Hilbert Space (RKHS) to represent the functions and a functional bootstrap procedure that allows us to estimate the prediction law and a Reproducing Kernel Hilbert Spaces (RKHS) to represent the functions, considering then the basis associated to the reproducing kernel. We then classify the points on the projected space according to those that belong to the minimum entropy set (MES) and those that do not. We map the minimum entropy set back to the functional space and construct a band using the regularity property of the RKHS. The proposed methodology is illustrated through artificial and real data sets.

Evaluation of deep learning for predicting rice traits using structural and single-nucleotide genomic variants

BackgroundStructural genomic variants (SVs) are prevalent in plant genomes and have played an important role in evolution and domestication, as they constitute a significant source of genomic and phenotypic variability. Nevertheless, most methods in quantitative genetics focusing on crop improvement, such as genomic prediction, consider only Single Nucleotide Polymorphisms (SNPs). Deep Learning (DL) is a promising strategy for genomic prediction, but its performance using SVs and SNPs as genetic markers remains unknown.ResultsWe used rice to investigate whether combining SVs and SNPs can result in better trait prediction over SNPs alone and examine the potential advantage of Deep Learning (DL) networks over Bayesian Linear models. Specifically, the performances of BayesC (considering additive effects) and a Bayesian Reproducible Kernel Hilbert space (RKHS) regression (considering both additive and non-additive effects) were compared to those of two different DL architectures, the Multilayer Perceptron, and the Convolution Neural Network, to explore their prediction ability by using various marker input strategies. We found that exploiting structural and nucleotide variation slightly improved prediction ability on complex traits in 87% of the cases. DL models outperformed Bayesian models in 75% of the studied cases, considering the four traits and the two validation strategies used. Finally, DL systematically improved prediction ability of binary traits against the Bayesian models.ConclusionsOur study reveals that the use of structural genomic variants can improve trait prediction in rice, independently of the methodology used. Also, our results suggest that Deep Learning (DL) networks can perform better than Bayesian models in the prediction of binary traits, and in quantitative traits when the training and target sets are not closely related. This highlights the potential of DL to enhance crop improvement in specific scenarios and the importance to consider SVs in addition to SNPs in genomic selection.