Approximation Of Multivariate Functions Research Articles

Constructing Embedded Lattice-Based Algorithms for Multivariate Function Approximation with a Composite Number of Points

Constructing Embedded Lattice-Based Algorithms for Multivariate Function Approximation with a Composite Number of Points

Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks

The expressive power of deep neural networks is manifested by their remarkable ability to approximate multivariate functions in a way that appears to overcome the curse of dimensionality. This ability is exemplified by their success in solving high-dimensional problems where traditional numerical solvers fail due to their limitations in accurately representing high-dimensional structures. To provide a theoretical framework for explaining this phenomenon, we analyze the approximation of Hölder functions defined on a d-dimensional smooth manifold M embedded in RD, with d≪D, using deep neural networks. We prove that the uniform convergence estimates of the approximation and generalization errors by deep neural networks with ReLU activation functions do not depend on the ambient dimension D of the function but only on its lower manifold dimension d, in a precise sense. Our result improves existing results from the literature where approximation and generalization errors were shown to depend weakly on D.

Smooth Uniform Approximation of Multivariate Spline Function

Multivariable spline approximation is of great significance both in mathematics and in practice. This involves the approximation of non-smooth functions. In this paper, the approximation of the non-smooth function , which is the most representative problem in robust statistics, is discussed. We propose a compact smooth uniform approximation algorithm using spline curves.

Chebyshev approximation of multivariable functions with the interpolation of the value of the function and its partial derivative

Chebyshev approximation of multivariable functions with the interpolation of the value of the function and its partial derivative

Polynomial approximation of symmetric functions

We study the polynomial approximation of symmetric multivariate functions and of multi-set functions. Specifically, we consider f ( x 1 , … , x N ) f(x_1, \dots , x_N) , where x i ∈ R d x_i \in \mathbb {R}^d , and f f is invariant under permutations of its N N arguments. We demonstrate how these symmetries can be exploited to improve the cost versus error ratio in a polynomial approximation of the function f f , and in particular study the dependence of that ratio on d , N d, N and the polynomial degree. These results are then used to construct approximations and prove approximation rates for functions defined on multi-sets where N N becomes a parameter of the input.

Chebyshev Approximation of Multivariable Functions by a Logarithmic Expression

Chebyshev Approximation of Multivariable Functions by a Logarithmic Expression

An Adaptive Sampling and Domain Learning Strategy for Multivariate Function Approximation on Unknown Domains

Many problems arising in computational science and engineering can be described in terms of approximating a smooth function of variables, defined over an unknown domain of interest , from sample data. Here both the underlying dimensionality of the problem (in the case ) as well as the lack of domain knowledge—with potentially irregular and/or disconnected—are confounding factors for sampling-based methods. Naïve approaches for such problems often lead to wasted samples and inefficient approximation schemes. For example, uniform sampling can result in upward of 20% wasted samples in some problems considered herein. In applications such as surrogate model construction in computational uncertainty quantification, the high cost of computing samples necessitates a more efficient sampling procedure. Over the last several years methods for computing such approximations from sample data have been studied in the case of irregular domains, and the advantages of computing sampling measures depending on an approximation space of have been shown. More specifically, such approaches confer advantages such as stability and well-conditioning, with an asymptotically optimal sample complexity scaling . The recently proposed adaptive sampling for general domains (ASGD) strategy is one such technique to construct these sampling measures. The main contribution of this paper is a procedure to improve upon the ASGD approach by adaptively updating the sampling measure in the case of unknown domains. We achieve this by first introducing a general domain adaptivity strategy, which computes an approximation of the function and domain of interest from the sample points. Second, we propose an adaptive sampling strategy, termed adaptive sampling for unknown domains (ASUD), which generates sampling measures over a domain that may not be known in advance, based on the ideas introduced in the ASGD approach. We then derive (weighted) least squares and augmented least squares techniques for polynomial approximation on unknown domains. We present numerical experiments demonstrating the efficacy of the adaptive sampling techniques with least squares–based polynomial approximation schemes. Our results show that the ASUD approach consistently achieves errors the same as or smaller than uniform sampling, but using fewer, and often significantly fewer, function evaluations.

Chebyshev Approximation of Multivariable Functions by a Constrained Rational Expression

Chebyshev Approximation of Multivariable Functions by a Constrained Rational Expression

CAS4DL: Christoffel adaptive sampling for function approximation via deep learning

The problem of approximating smooth, multivariate functions from sample points arises in many applications in scientific computing, e.g., in computational Uncertainty Quantification (UQ) for science and engineering. In these applications, the target function may represent a desired quantity of interest of a parameterized Partial Differential Equation (PDE). Due to the large cost of solving such problems, where each sample is computed by solving a PDE, sample efficiency is a key concern in these applications. Recently, there has been increasing focus on the use of Deep Neural Networks (DNN) and Deep Learning (DL) for learning such functions from data. In this work, we propose an adaptive sampling strategy, CAS4DL (Christoffel Adaptive Sampling for Deep Learning) to increase the sample efficiency of DL for multivariate function approximation. Our novel approach is based on interpreting the second to last layer of a DNN as a dictionary of functions defined by the nodes on that layer. With this viewpoint, we then define an adaptive sampling strategy motivated by adaptive sampling schemes recently proposed for linear approximation schemes, wherein samples are drawn randomly with respect to the Christoffel function of the subspace spanned by this dictionary. We present numerical experiments comparing CAS4DL with standard Monte Carlo (MC) sampling. Our results demonstrate that CAS4DL often yields substantial savings in the number of samples required to achieve a given accuracy, particularly in the case of smooth activation functions, and it shows a better stability in comparison to MC. These results therefore are a promising step towards fully adapting DL towards scientific computing applications.

Efficient randomized tensor-based algorithms for function approximation and low-rank kernel interactions

In this paper, we introduce a method for multivariate function approximation using function evaluations, Chebyshev polynomials, and tensor-based compression techniques via the Tucker format. We develop novel randomized techniques to accomplish the tensor compression, provide a detailed analysis of the computational costs, provide insight into the error of the resulting approximations, and discuss the benefits of the proposed approaches. We also apply the tensor-based function approximation to develop low-rank matrix approximations to kernel matrices that describe pairwise interactions between two sets of points; the resulting low-rank approximations are efficient to compute and store (the complexity is linear in the number of points). We present an adaptive version of the function and kernel approximation that determines an approximation that satisfies a user-specified relative error over a set of random points. We extend our approach to the case where the kernel requires repeated evaluations for many values of (hyper)parameters that govern the kernel. We give detailed numerical experiments on example problems involving multivariate function approximation, low-rank matrix approximations of kernel matrices involving well-separated clusters of sources and target points, and a global low-rank approximation of kernel matrices with an application to Gaussian processes. We observe speedups up to 18X over standard matrix-based approaches.

FRACTAL DIMENSION OF MULTIVARIATE α-FRACTAL FUNCTIONS AND APPROXIMATION ASPECTS

In this paper, we explore the concept of dimension preserving approximation of continuous multivariate functions defined on the domain [Formula: see text] (q-times) where [Formula: see text] is a natural number). We establish a few well-known multivariate constrained approximation results in terms of dimension preserving approximants. In particular, we indicate the construction of multivariate dimension preserving approximants using the concept of [Formula: see text]-fractal interpolation functions. We also prove the existence of one-sided approximation of multivariate function using fractal functions. Moreover, we provide an upper bound for the fractal dimension of the graph of the [Formula: see text]-fractal function. Further, we study the approximation aspects of [Formula: see text]-fractal functions and establish the existence of the Schauder basis consisting of multivariate fractal functions for the space of all real valued continuous functions defined on [Formula: see text] and prove the existence of multivariate fractal polynomials for the approximation.

Mixed Multinomial Probit Model Accommodating Flexible Covariance Structure and Random Taste Variation: An Application to Commute Mode Choice Behavior

This paper developed a mixed multinomial probit (MMNP) model with alternative error specification and random coefficients (for both generic variables and personal attributes) to accommodate flexible covariance structure and taste variation. The MMNP model can be efficiently estimated with analytic approximations of multivariate normal cumulative distribution functions, which avoid defects of simulation-based integration in the mixed multinomial logit (MMNL) model. The integral dimension of the MMNL model increases as random coefficients increase, but it only depends on the number of available alternatives in the MMNP model. Simulation experiments and empirical analysis of Shanghai commuters’ mode choice behavior were undertaken to examine the performance of MMNP models. Both simulation results and empirical results show that MMNP models can well accommodate flexible covariance structures and taste variation reflected through random coefficients being associated with both generic and personal variables. Empirical results indicate that the MMNP model performs better than traditional discrete choice models, such as the multinomial logit, the cross-nested logit, MMNL, and multinomial probit models. Random coefficients of “in-vehicle time of car” and “number of companions” indicate taste heterogeneity and the identifiability of random coefficients associated with both generic and personal attributes. Pairwise positive correlations between car/taxi, bus/metro, and bus/bus and metro are to be expected. However, the positive correlation between the car and metro modes may be unique to the Chinese city, Shanghai, because of the developed metro system. Unequal error variances reflect heterogeneities in unspecified factors in commute modes’ utilities. The MMNP model will offer an alternative efficient way to accommodate taste heterogeneity and flexible error covariance structure in discrete choice models. Compared with the MMNL model, the MMNP model can accommodate more random coefficients without increasing computational complexity.

Matrix and tensor rigidity and Lp-approximation

We apply methods originated in Complexity theory to some problems of Approximation. We notice that the construction of Alman and Williams that disproves the rigidity of Walsh-Hadamard matrices, provides good ℓp-approximation for p<2. Hence the first n functions of Walsh system can be approximated by a low-dimensional linear space:dn1−δ({w1,…,wn},Lp[0,1])⩽n−δ,1≤p<2, where δ=δ(p)>0. We do not know if this is possible for the trigonometric system. We show that the algebraic method of Alon–Frankl–Rödl for bounding the number of low-signum-rank matrices, works for tensors: almost all signum-tensors have large signum-rank and can't be ℓ1-approximated by low-rank tensors. This implies lower bounds for Θm — the error of m-term approximation of multivariate functions by sums of tensor products u1(x1)⋯ud(xd). For the set of trigonometric polynomials with spectrum in ∏j=1d[−nj,nj] and of norm ‖t‖∞⩽1, and m⩽c(d)∏nj/max⁡{nj}, we haveΘm(T(n1,…,nd)∞,L1[−π,π]d)⩾c1(d)>0. Sharp bounds follow for classes of dominated mixed smoothness in the case 2⩽p⩽∞,1⩽q⩽2:Θm(Wp(r,r,…,r),Lq[0,1]d)≍m−rdd−1.

On the Power of Standard Information for Tractability for L&lt;sub&gt;2&lt;/sub&gt;- Approximation in the Randomized Setting

We study the approximation of multivariate functions from a separable Hilbert space in the randomized setting with the error measured in the weighted L2 norm. We consider algorithms that use standard information Λstd consisting of function values or general linear information Λall consisting of arbitrary linear functionals. We investigate the equivalences of various notions of algebraic and exponential tractability in the randomized setting for Λstd and Λall for the normalized or absolute error criterion. For the normalized error criterion, we show that the power of Λstd is the same as that of Λall for all notions of exponential tractability and some notions of algebraic tractability without any condition. For the absolute error criterion, we show that the power of Λstd is the same as that of Λall for all notions of algebraic and exponential tractability without any condition. Specifically, we solve Open Problems 98, 101, 102 and almost solve Open Problem 100 as posed by E.Novak and H.Wo´zniakowski in the book: Tractability of Multivariate Problems, Volume III: Standard Information for Operators, EMS Tracts in Mathematics, Zürich, 2012.

Chebyshev approximation of multivariable functions with the interpolation

A method of constructing a Chebyshev approximation of multivariable functions by a generalized polynomial with the exact reproduction of its values at a given points is proposed. It is based on the sequential construction of mean-power approximations, taking into account the interpolation condition. The mean-power approximation is calculated using an iterative scheme based on the method of least squares with the variable weight function. An algorithm for calculating the Chebyshev approximation parameters with the interpolation condition for absolute and relative error is described. The presented results of solving test examples confirm the rapid convergence of the method when calculating the parameters of the Chebyshev approximation of tabular continuous functions of one, two and three variables with the reproduction of the values of the function at given points.

Approximation, characterization, and continuity of multivariate monotonic regression functions

We deal with monotonic regression of multivariate functions [Formula: see text] on a compact rectangular domain [Formula: see text] in [Formula: see text], where monotonicity is understood in a generalized sense: as isotonicity in some coordinate directions and antitonicity in some other coordinate directions. As usual, the monotonic regression of a given function [Formula: see text] is the monotonic function [Formula: see text] that has the smallest (weighted) mean-squared distance from [Formula: see text]. We establish a simple general approach to compute monotonic regression functions: namely, we show that the monotonic regression [Formula: see text] of a given function [Formula: see text] can be approximated arbitrarily well — with simple bounds on the approximation error in both the [Formula: see text]-norm and the [Formula: see text]-norm — by the monotonic regression [Formula: see text] of grid-constant functions [Formula: see text]. monotonic regression algorithms. We also establish the continuity of the monotonic regression [Formula: see text] of a continuous function [Formula: see text] along with an explicit averaging formula for [Formula: see text]. And finally, we deal with generalized monotonic regression where the mean-squared distance from standard monotonic regression is replaced by more complex distance measures which arise, for instance, in maximum smoothed likelihood estimation. We will see that the solution of such generalized monotonic regression problems is simply given by the standard monotonic regression [Formula: see text].

A deterministic algorithm for constructing multiple rank-1 lattices of near-optimal size

In this paper we present the first known deterministic algorithm for the construction of multiple rank-1 lattices for the approximation of periodic functions of many variables. The algorithm works by converting a potentially large reconstructing single rank-1 lattice for some d-dimensional frequency set I ⊂{0,…,N − 1}d into a collection of much smaller rank-1 lattices which allow for accurate and efficient reconstruction of trigonometric polynomials with coefficients in I (and, therefore, for the approximation of multivariate periodic functions). The total number of sampling points in the resulting multiple rank-1 lattices is theoretically shown to be less than \( \mathcal {O}\left (|I| \log ^{2 }(N |I|) \right ) \) with constants independent of d, and by performing one-dimensional fast Fourier transforms on samples of trigonometric polynomials with Fourier support in I at these points, we obtain exact reconstruction of all Fourier coefficients in fewer than \( \mathcal {O}\left (d |I|\log ^{4} (N|I|)\right ) \) total operations. Additionally, we present a second multiple rank-1 lattice construction algorithm which constructs lattices with even fewer sampling points at the cost of only being able to reconstruct exact trigonometric polynomials rather than having additional theoretical approximation guarantees. Both algorithms are tested numerically and surpass the theoretical bounds. Notably, we observe that the oversampling factors #samples/|I| appear to grow only logarithmically in |I| for the first algorithm and appear near-optimally bounded by four in the second algorithm.

Approximation by quasi-interpolation operators and Smolyak's algorithm

We study approximation of multivariate periodic functions from Besov and Triebel–Lizorkin spaces of dominating mixed smoothness by the Smolyak algorithm constructed using a special class of quasi-interpolation operators of Kantorovich-type. These operators are defined similar to the classical sampling operators by replacing samples with the average values of a function on small intervals (or more generally with sampled values of a convolution of a given function with an appropriate kernel). In this paper, we estimate the rate of convergence of the corresponding Smolyak algorithm in the Lq-norm for functions from the Besov spaces Bp,θs(Td) and the Triebel–Lizorkin spaces Fp,θs(Td) for all s>0 and admissible 1≤p,θ≤∞ as well as provide analogues of the Littlewood–Paley-type characterizations of these spaces in terms of families of quasi-interpolation operators.

DNN Expression Rate Analysis of High-Dimensional PDEs: Application to Option Pricing

We analyze approximation rates by deep ReLU networks of a class of multivariate solutions of Kolmogorov equations which arise in option pricing. Key technical devices are deep ReLU architectures capable of efficiently approximating tensor products. Combining this with results concerning the approximation of well-behaved (i.e., fulfilling some smoothness properties) univariate functions, this provides insights into rates of deep ReLU approximation of multivariate functions with tensor structures. We apply this in particular to the model problem given by the price of a European maximum option on a basket of d assets within the Black–Scholes model for European maximum option pricing. We prove that the solution to the d-variate option pricing problem can be approximated up to an varepsilon -error by a deep ReLU network with depth {mathcal {O}}big (ln (d)ln (varepsilon ^{-1})+ln (d)^2big ) and {mathcal {O}}big (d^{2+frac{1}{n}}varepsilon ^{-frac{1}{n}}big ) nonzero weights, where nin {mathbb {N}} is arbitrary (with the constant implied in {mathcal {O}}(cdot ) depending on n). The techniques developed in the constructive proof are of independent interest in the analysis of the expressive power of deep neural networks for solution manifolds of PDEs in high dimension.

Chebyshev Approximation of Multivariable Functions by the Exponential Expression

Chebyshev Approximation of Multivariable Functions by the Exponential Expression