Korobov Spaces Research Articles

Approximation of functionals on Korobov spaces with Fourier Functional Networks

Learning from functional data with deep neural networks has become increasingly useful, and numerous neural network architectures have been developed to tackle high-dimensional problems raised in practical domains. Despite the impressive practical achievements, theoretical foundations underpinning the ability of neural networks to learn from functional data largely remain unexplored. In this paper, we investigate the approximation capacity of a functional neural network, called Fourier Functional Network, consisting of Fourier neural operators and deep convolutional neural networks with a great reduction in parameters. We establish rates of approximating by Fourier Functional Networks nonlinear continuous functionals defined on Korobov spaces of periodic functions. Finally, our results demonstrate dimension-independent convergence rates, which overcomes the curse of dimension.

Tractability of sampling recovery on unweighted function classes

It is well-known that the problem of sampling recovery in the L 2 L_2 -norm on unweighted Korobov spaces (Sobolev spaces with mixed smoothness) as well as classical smoothness classes such as Hölder classes suffers from the curse of dimensionality. We show that the problem is tractable for those classes if they are intersected with the Wiener algebra of functions with summable Fourier coefficients. In fact, this is a relatively simple implication of powerful results from the theory of compressed sensing. Tractability is achieved by the use of non-linear algorithms, while linear algorithms cannot do the job.

Approximation of functions from Korobov spaces by shallow neural networks

In this paper, we consider the problem of approximating functions from a Korobov space on [−1,1]d by ReLU shallow neural networks and present a rate O(m−25(1+2d)log⁡m) of uniform approximation by networks of m hidden neurons. This is achieved by combining a novel Fourier analysis approach and a probability argument. We apply our approximation theory to a learning algorithm for regression based on ReLU shallow neural networks and derive learning rates of order O(N−4(d+2)9d+8log⁡N) for the excess generalization error with the sample size N when the regression function lies in the Korobov space.

Learning Korobov Functions by Correntropy and Convolutional Neural Networks.

Combining information-theoretic learning with deep learning has gained significant attention in recent years, as it offers a promising approach to tackle the challenges posed by big data. However, the theoretical understanding of convolutional structures, which are vital to many structured deep learning models, remains incomplete. To partially bridge this gap, this letter aims to develop generalization analysis for deep convolutional neural network (CNN) algorithms using learning theory. Specifically, we focus on investigating robust regression using correntropy-induced loss functions derived from information-theoretic learning. Our analysis demonstrates an explicit convergence rate for deep CNN-based robust regression algorithms when the target function resides in the Korobov space. This study sheds light on the theoretical underpinnings of CNNs and provides a framework for understanding their performance and limitations.

On the power of standard information for tractability for L∞ approximation of periodic functions in the worst case setting

We study multivariate approximation of periodic functions in the worst case setting with the error measured in the L∞ norm. We consider algorithms that use standard information Λstd consisting of function values or general linear information Λall consisting of arbitrary continuous linear functionals. We investigate equivalences of various notions of algebraic and exponential tractability for Λstd and Λall under the absolute or normalized error criterion, and show that the power of Λstd is the same as the one of Λall for various notions of algebraic and exponential tractability. Our results can be applied to weighted Korobov spaces and Korobov spaces with exponential weights. This gives a special solution to Open Problem 145 as posed by Novak and Woźniakowski (2012) [40].

Random-prime–fixed-vector randomised lattice-based algorithm for high-dimensional integration

We show that a very simple randomised algorithm for numerical integration can produce a near optimal rate of convergence for integrals of functions in the d-dimensional weighted Korobov space. This algorithm uses a lattice rule with a fixed generating vector and the only random element is the choice of the number of function evaluations. For a given computational budget n of a maximum allowed number of function evaluations, we uniformly pick a prime p in the range n/2<p≤n. We show error bounds for the randomised error, which is defined as the worst case expected error, of the form O(n−α−1/2+δ), with δ>0, for a Korobov space with smoothness α>1/2 and general weights. The implied constant in the bound is dimension-independent given the usual conditions on the weights. We present an algorithm that can construct suitable generating vectors offline ahead of time at cost O(dn4/ln⁡n) when the weight parameters defining the Korobov spaces are so-called product weights. For this case, numerical experiments confirm our theory that the new randomised algorithm achieves the near optimal rate of the randomised error.

Density Estimation in RKHS with Application to Korobov Spaces in High Dimensions

A kernel method for estimating a probability density function from an independent and identically distributed sample drawn from such density is presented. Our estimator is a linear combination of kernel functions, the coefficients of which are determined by a linear equation. An error analysis for the mean integrated squared error is established in a general reproducing kernel Hilbert space setting. The theory developed is then applied to estimate probability density functions belonging to weighted Korobov spaces, for which a dimension-independent convergence rate is established. Under a suitable smoothness assumption, our method attains a rate arbitrarily close to the optimal rate. Numerical results support our theory.

Approximation of functions from Korobov spaces by deep convolutional neural networks

The efficiency of deep convolutional neural networks (DCNNs) has been demonstrated empirically in many practical applications. In this paper, we establish a theory for approximating functions from Korobov spaces by DCNNs. It verifies rigorously the efficiency of DCNNs in approximating functions of many variables with some variable structures and their abilities in overcoming the curse of dimensionality.

Scaled lattice rules for integration on ℝ^{𝕕} achieving higher-order convergence with error analysis in terms of orthogonal projections onto periodic spaces

We introduce a new method to approximate integrals∫Rdf(x)dx\int _{\mathbb {R}^d} f(\boldsymbol {x}) \,\mathrm {d}\boldsymbol {x}which simply scales lattice rules from the unit cube[0,1]d[0,1]^dto properly sized boxes onRd\mathbb {R}^d, hereby achieving higher-order convergence that matches the smoothness of the integrand functionffin a certain Sobolev space of dominating mixed smoothness. Our method only assumes that we can evaluate the integrand functionffand does not assume a particular density nor the ability to sample from it. In particular, for the theoretical analysis we show a new result that the method of adding Bernoulli polynomials to a function to make it “periodic” on a box without changing its integral value over the box is equivalent to an orthogonal projection from a well chosen Sobolev space of dominating mixed smoothness to an associated periodic Sobolev space of the same dominating mixed smoothness, which we call a Korobov space. We note that the Bernoulli polynomial method is often not used because of its excessive computational complexity and also here we only make use of it in our theoretical analysis. We show that our new method of applying scaled lattice rules to increasing boxes can be interpreted as orthogonal projections with decreasing projection error. Such a method would not work on the unit cube since then the committed error caused by non-periodicity of the integrand would be constant, but for integration on the Euclidean space we can use the certain decay towards zero when the boxes grow. Hence we can bound the truncation error as well as the projection error and show higher-order convergence in applying scaled lattice rules for integration on Euclidean space. We illustrate our theoretical analysis by numerical experiments which confirm our findings.

Component-by-component construction of randomized rank-1 lattice rules achieving almost the optimal randomized error rate

We study a randomized quadrature algorithm to approximate the integral of periodic functions defined over the high-dimensional unit cube. Recent work by Kritzer, Kuo, Nuyens and Ullrich [J. Approx. Theory 240 (2019), pp. 96–113] shows that rank-1 lattice rules with a randomly chosen number of points and good generating vector achieve almost the optimal order of the randomized error in weighted Korobov spaces, and moreover, that the error is bounded independently of the dimension if the weight parameters,γj\gamma _j, satisfy the summability condition∑j=1∞γj1/α&gt;∞\sum _{j=1}^{\infty }\gamma _j^{1/\alpha }&gt;\infty, whereα\alphais a smoothness parameter. The argument is based on the existence result that at least half of the possible generating vectors yield almost the optimal order of the worst-case error in the same function spaces.In this paper we provide a component-by-component construction algorithm of such randomized rank-1 lattice rules, without any need to check whether the constructed generating vectors satisfy a desired worst-case error bound. Similarly to the above-mentioned work, we prove that our algorithm achieves almost the optimal order of the randomized error and that the error bound is independent of the dimension if the same condition∑j=1∞γj1/α&gt;∞\sum _{j=1}^{\infty }\gamma _j^{1/\alpha }&gt;\inftyholds. We also provide analogous results for tent-transformed lattice rules for weighted half-period cosine spaces and for polynomial lattice rules in weighted Walsh spaces, respectively.

Construction-Free Median Quasi-Monte Carlo Rules for Function Spaces with Unspecified Smoothness and General Weights

We study quasi-Monte Carlo (QMC) integration of smooth functions defined over the multi-dimensional unit cube. Inspired by a recent work of Pan and Owen, we study a new construction-free median QMC rule which can exploit the smoothness and the weights of function spaces adaptively. For weighted Korobov spaces, we draw a sample of $r$ independent generating vectors of rank-1 lattice rules, compute the integral estimate for each, and approximate the true integral by the median of these $r$ estimates. For weighted Sobolev spaces, we use the same approach but with the rank-1 lattice rules replaced by high-order polynomial lattice rules. A major advantage over the existing approaches is that we do not need to construct good generating vectors by a computer search algorithm, while our median QMC rule achieves almost the optimal worst-case error rate for the respective function space with any smoothness and weights, with a probability that converges to 1 exponentially fast as $r$ increases. Numerical experiments illustrate and support our theoretical findings.

EC-(t1,t2)-tractability of approximation in weighted Korobov spaces in the worst case setting

We investigate multivariate approximation problems in the weighted Korobov spaces in the worst case setting. We consider algorithms that use finitely many evaluations of arbitrary continuous linear functionals. We discuss the exponential convergence-(t1,t2)-weak tractability (EC-(t1,t2)-WT) under the absolute or normalized error criterion. Especially, we give the sufficient and necessary conditions for EC-WT and EC-(t1,1)-WT with t1<1.

A note on EC-tractability of multivariate approximation in weighted Korobov spaces for the standard information class

In this paper we study exponential tractability of multivariate approximation problem for weighted Korobov spaces in the worst case setting. The considered algorithms are constructive and use the class Λstd of only function evaluations. We give matching necessary and sufficient conditions for notions of EC-quasi-polynomial tractability and EC-uniform weak tractability in terms of two weight parameters of the problem.

Tractability of approximation in the weighted Korobov space in the worst-case setting — a complete picture

In this paper, we study tractability of L2-approximation of one-periodic functions from weighted Korobov spaces in the worst-case setting. The considered weights are of product form. For the algorithms we allow information from the class Λall consisting of all continuous linear functionals and from the class Λstd, which only consists of function evaluations. We provide necessary and sufficient conditions on the weights of the function space for quasi-polynomial tractability, uniform weak tractability, weak tractability and (σ,τ)-weak tractability. Together with the already known results for strong polynomial and polynomial tractability, our findings provide a complete picture of the weight conditions for all current standard notions of tractability.

A note on Korobov lattice rules for integration of analytic functions

We study numerical integration for a weighted Korobov space of analytic periodic functions for which the Fourier coefficients decay exponentially fast. In particular, we are interested in how the error depends on the dimension d. Many recent papers deal with this problem or similar problems and provide matching necessary and sufficient conditions for various notions of tractability. In most cases even simple algorithms are known which allow to achieve these notions of tractability. However, there is a gap in the literature: while for the notion of exponential-weak tractability one knows matching necessary and sufficient conditions, so far no explicit algorithm has been known which yields the desired result.In this paper we close this gap and prove that Korobov lattice rules are suitable algorithms in order to achieve exponential-weak tractability for integration in weighted Korobov spaces of analytic periodic functions.

Stability of lattice rules and polynomial lattice rules constructed by the component-by-component algorithm

We study quasi-Monte Carlo (QMC) methods for numerical integration of multivariate functions defined over the high-dimensional unit cube. Lattice rules and polynomial lattice rules, which are special classes of QMC methods, have been intensively studied and the so-called component-by-component (CBC) algorithm has been well-established to construct rules which achieve the almost optimal rate of convergence with good tractability properties for given smoothness and set of weights. Since the CBC algorithm constructs rules for given smoothness and weights, not much is known when such rules are used for function classes with different smoothness and/or weights.In this paper we prove that a lattice rule constructed by the CBC algorithm for the weighted Korobov space with given smoothness and weights achieves the almost optimal rate of convergence with good tractability properties for general classes of smoothness and weights which satisfy some summability conditions. Such a stability result also can be shown for polynomial lattice rules in weighted Walsh spaces. We further give bounds on the weighted star discrepancy and discuss the tractability properties for these QMC rules. The results are comparable to those obtained for Halton, Sobol and Niederreiter sequences.

Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units

Deep neural networks with rectified linear units (ReLU) are getting more and more popular due to their universal representation power and successful applications. Some theoretical progress regarding the approximation power of deep ReLU network for functions in Sobolev space and Korobov space have recently been made by [D. Yarotsky, Neural Network, 94:103-114, 2017] and [H. Montanelli and Q. Du, SIAM J Math. Data Sci., 1:78-92, 2019], etc. In this paper, we show that deep networks with rectified power units (RePU) can give better approximations for smooth functions than deep ReLU networks. Our analysis bases on classical polynomial approximation theory and some efficient algorithms proposed in this paper to convert polynomials into deep RePU networks of optimal size with no approximation error. Comparing to the results on ReLU networks, the sizes of RePU networks required to approximate functions in Sobolev space and Korobov space with an error tolerance $\varepsilon$, by our constructive proofs, are in general $\mathcal{O}(\log\frac{1}{\varepsilon})$ times smaller than the sizes of corresponding ReLU networks constructed in most of the existing literature. Comparing to the classical results of Mhaskar [Mhaskar, Adv. Comput. Math. 1:61-80, 1993], our constructions use less number of activation functions and numerically more stable, they can be served as good initials of deep RePU networks and further trained to break the limit of linear approximation theory. The functions represented by RePU networks are smooth functions, so they naturally fit in the places where derivatives are involved in the loss function.

ОБ ОЦЕНКЕ СНИЗУ В ЗАДАЧЕ ПРИБЛИЖЕННОГО ВОССТАНОВЛЕНИЯ ФУНКЦИЙ ПО ИХ ЗНАЧЕНИЯМ ПРЕОБРАЗОВАНИЙ РАДОНА

In this paper, we study the problem of function reconstruction by values of Radon transforms within the framework of the Computational (Numerical) Diameter (C(N)D) approach. The meaning of C(N)D is to solve two independent problems: obtaining lower bounds of the reconstruction error by exact information and specifying the computing tool that implements the upper bounds (preferably coinciding with the lower bound up to constants). The C(N)D approach is a mathematical model of experiments for describing various processes (physical, chemical, technical, etc.). An important role in setting up such experiments is played by types of measuring instruments, which is reflected in C(N)D as types of functionals. The next important point is the choice of location and balancing of instruments, i.e. selection of functionals’ parameters. The final step is to build an optimal computing tool using the obtained data. The most studied types of functionals are function values at points and Fourier coefficients. An important difference of this work from previously obtained results is the study of the approximation capabilities of another type of functionals – Radon transforms, i.e. mathematical model of the use of tomography and similar technologies. This paper is devoted to obtaining lower bounds for the error in reconstructing functions from Sobolev and Korobov spaces.

Explicit error bounds for randomized Smolyak algorithms and an application to infinite-dimensional integration

Smolyak’s method, also known as sparse grid method, is a powerful tool to tackle multivariate tensor product problems solely with the help of efficient algorithms for the corresponding univariate problem.In this paper we study the randomized setting, i.e., we randomize Smolyak’s method. We provide upper and lower error bounds for randomized Smolyak algorithms with explicitly given dependence on the number of variables and the number of information evaluations used. The error criteria we consider are the worst case root mean square error (the typical error criterion for randomized algorithms, sometimes referred to as “randomized error”, ) and the root mean square worst case error (sometimes referred to as “worst-case error”).Randomized Smolyak algorithms can be used as building blocks for efficient methods such as multilevel algorithms, multivariate decomposition methods or dimension-wise quadrature methods to tackle successfully high-dimensional or even infinite-dimensional problems. As an example, we provide a very general and sharp result on the convergence rate of Nth minimal errors of infinite-dimensional integration on weighted reproducing kernel Hilbert spaces. Moreover, we are able to characterize the spaces for which randomized algorithms for infinite-dimensional integration are superior to deterministic ones. We illustrate our findings for the special instance of weighted Korobov spaces. We indicate how these results can be extended, e.g., to spaces of functions whose smooth dependence on successive variables increases (“spaces of increasing smoothness”) and to the problem of L2-approximation (function recovery).

EC-tractability of [formula omitted]-approximation in Korobov spaces with exponential weights

This study aims at investigating Lp-approximation with 1≤p≤∞ defined over weighted Korobov spaces. The considered algorithms use finitely many evaluations of arbitrary linear functionals. In the worst case setting, we give matching necessary and sufficient conditions for some notions of EC-tractability in terms of two weight parameters of the problem. Our results give an affirmative answer to the open problem raised recently by Kritzer et al. (2017).