Average Case Approximation: Convergence and Tractability of Gaussian Kernels

Abstract

AbstractWe study the problem of approximating functions of d variables in the average case setting for a separable Banach space \({\mathcal{F}}_{d}\) equipped with a zero-mean Gaussian measure. The covariance kernel of this Gaussian measure takes the form of a Gaussian that depends on shape parameters \({\gamma }_{\mathcal{l}}\). We stress that d can be arbitrarily large. Our approximation error is defined in the \({\mathcal{L}}_{2}\) norm, and we study the minimal average case error \({e}_{d}^{\mathrm{avg}}(n)\) of algorithms that use at most n linear functionals or function values. For \({\gamma }_{\mathcal{l}} = {\mathcal{l}}^{-\alpha }\) with \(\alpha \geq 0\), we prove that \({e}_{d}^{\mathrm{avg}}(n)\) has a polynomial bound of roughly order \({n}^{-(\alpha -1/2)}\) independent of d iff \(\alpha \,\,>\,\,1/2\). This property is equivalent to strong polynomial tractability and says that the minimal number of linear functionals or function values needed to achieve an average case error \(\epsilon \) has a bound independent of d proportional roughly to \({\epsilon }^{-1/(\alpha -1/2)}\). In the case of algorithms that use only function values the proof is non-constructive. In order to compare the average case with the worst case studied in our earlier paper we specialize the function space \({\mathcal{F}}_{d}\) to a reproducing kernel Hilbert space whose kernel is a Gaussian kernel with shape parameters \({\gamma }_{\mathcal{l}}^{\,\mathrm{rep}}\). To allow for a fair comparison we further equip this space with a zero-mean Gaussian measure whose covariance operator has eigenvalues that depend on a positive parameter q. We prove that the average cases for the whole space and for the unit ball of \({\mathcal{F}}_{d}\) are roughly the same provided the \({\gamma }_{\mathcal{l}}^{\,\mathrm{rep}}\) decay quickly enough. Furthermore, for a particular choice of q the dimension-independent convergence for the worst and average case settings are essentially the same.KeywordsUnit BallGaussian KernelAverage CaseReproduce Kernel Hilbert SpaceIsotropic Gaussian KernelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.