Probability Error Bounds for Approximation of Functions in Reproducing Kernel Hilbert Spaces

Ata Deniz Aydın,Aurelian Gheondea

doi:10.1155/2021/6617774

Abstract

We find probability error bounds for approximations of functions f in a separable reproducing kernel Hilbert space H with reproducing kernel K on a base space X , firstly in terms of finite linear combinations of functions of type K x i and then in terms of the projection π x n on span K x i i = 1 n , for random sequences of points x = x i i in X . Given a probability measure P , letting P K be the measure defined by d P K x = K x , x d P x , x ∈ X , our approach is based on the nonexpansive operator L 2 X ; P K ∋ λ ↦ L P , K λ ≔ ∫ X λ x K x d P x ∈ H , where the integral exists in the Bochner sense. Using this operator, we then define a new reproducing kernel Hilbert space, denoted by H P , that is the operator range of L P , K . Our main result establishes bounds, in terms of the operator L P , K , on the probability that the Hilbert space distance between an arbitrary function f in H and linear combinations of functions of type K x i , for x i i sampled independently from P , falls below a given threshold. For sequences of points x i i = 1 ∞ constituting a so-called uniqueness set, the orthogonal projections π x n to span K x i i = 1 n converge in the strong operator topology to the identity operator. We prove that, under the assumption that H P is dense in H , any sequence of points sampled independently from P yields a uniqueness set with probability 1. This result improves on previous error bounds in weaker norms, such as uniform or L p norms, which yield only convergence in probability and not almost certain convergence. Two examples that show the applicability of this result to a uniform distribution on a compact interval and to the Hardy space H 2 D are presented as well.

Highlights

Several machine learning algorithms that use positive semidefinite kernels, such as support vector machines (SVM), have been analysed and justified rigorously using the theory of reproducing kernel Hilbert spaces (RKHS), yielding statements of optimality, convergence, and Lp approximation bounds, e.g., see Cucker and Smale [1]
We present two examples that point out the applicability, and the limitations of our results as well, the first to the uniform probability distribution on the compact interval 1⁄2−π, π, together with a class of bounded continuous kernels, and the second to the
Throughout this section, we consider a probability measure space ðX ; Σ ; PÞ and a RKHS ðH ; h·, · iÞ in FX, with norm denoted by k·kH, such that its reproducing kernel K is measurable

Summary

Introduction

Several machine learning algorithms that use positive semidefinite kernels, such as support vector machines (SVM), have been analysed and justified rigorously using the theory of reproducing kernel Hilbert spaces (RKHS), yielding statements of optimality, convergence, and Lp approximation bounds, e.g., see Cucker and Smale [1]. Reproducing kernel Hilbert spaces are Hilbert spaces of functions associated to a suitable kernel such that convergence with respect to the Hilbert space norm implies pointwise convergence, and in the context of approximation possess various favourable properties resulting from the Hilbert space structure. Under certain conditions on the kernel, every function in the Hilbert space is sufficiently differentiable, and differentiation is a nonexpansive linear map with respect to the Hilbert space norm, e.g., see ([2], Subsection 2.1.3). The theory considers a data space X ⊆ Rd on which an unknown probability distribution P is defined, a Journal of Function Spaces hypothesis set H , and a loss function V : H × X ⟶ R+, such that one wishes to find a hypothesis h ∈ H that minimizes the expected risk ð

Objectives

Results

Conclusion