Approximation Of Multivariate Functions Research Articles

Approximation by ridge functions and neural networks with a bounded number of neurons

In this paper, we consider the problem of approximation of continuous multivariate functions by neural networks with a bounded number of neurons in hidden layers. We prove the existence of single-hidden-layer networks with bounded number of neurons, which have approximation capabilities not worse than those of networks with arbitrarily many neurons. Our analysis is based on the properties of ridge functions.

Mathematical-thermodynamic solubility model developed by the application of discrete Volterra functional series theory

Based on the discrete form of Volterra functional series in approximation of multivariable functions, a mathematical model proposed for the modeling of solute solubility in supercritical fluids with four independent variables, which can be regarded as the general form of majority of available empirical models in literatures. As far as we know, it is the first time the thermodynamic-mathematical development of empirical models is presented. A substantial number of data points for different solutes and solvents is used to analyze the performance and reliability of the proposed model. Peng–Robinson equation of state together with the two adjustable parameters van der Waals mixing rule, as an example of theoretical models, and a number of recent empirical models, as examples of empirical/statistical models, were considered to demonstrate the superiority of the proposed model. The particle swarm optimization (PSO) was used for determination of the optimum coefficients of all models. The results were obtained and discussed in details.

Approximation of multivariate periodic functions by trigonometric polynomials based on sampling along rank-1 lattice with generating vector of Korobov form

In this paper, we present error estimates for the approximation of multivariate periodic functions in periodic Hilbert spaces of isotropic and dominating mixed smoothness by trigonometric polynomials. The approximation is based on sampling of the multivariate functions on rank-1 lattices. We use reconstructing rank-1 lattices with generating vectors of Korobov form for the sampling and generalize the technique from Temlyakov (1986), in order to show that the aliasing error of that approximation is of the same order as the error of the approximation using the partial sum of the Fourier series. The main advantage of our method is that the computation of the Fourier coefficients of such a trigonometric polynomial, which we use as approximant, is based mainly on a one-dimensional fast Fourier transform, cf. Kämmerer et al. (2013), Kämmerer (2014). This means that the arithmetic complexity of the computation depends only on the cardinality of the support of the trigonometric polynomial in the frequency domain. Numerical results are presented up to dimension d=10.

Reductive enhanced multivariance product representation for multi-way arrays

The main purpose of this work is to develop a new multi-way decomposition technique by considering statistical structure of target multi-way array. To this end enhanced multivariance products representation (EMPR), which is an expansion extended from high dimensional model representation (HDMR), is used. EMPR provides quite successful results on representation and approximation of multivariate functions when they have high level multiplicativity. Hence this urges us to reconstruct EMPR as a multi-way array decomposer. This paper presents this decomposition technique with all reconstruction formulations and numerical experiments on synthetic and real-life data sets to denote EMPR’s efficiency as a decomposer and also presents a combined method Reductive-EMPR (R-EMPR) as a multi-way array decomposition technique.

Multivariate Jackson-type inequality for a new type neural network approximation

In this paper, we introduce a new type neural networks by superpositions of a sigmoidal function and study its approximation capability. We investigate the multivariate quantitative constructive approximation of real continuous multivariate functions on a cube by such type neural networks. This approximation is derived by establishing multivariate Jackson-type inequalities involving the multivariate modulus of smoothness of the target function. Our networks require no training in the traditional sense.

Low-rank tensor completion by Riemannian optimization

In tensor completion, the goal is to fill in missing entries of a partially known tensor under a low-rank constraint. We propose a new algorithm that performs Riemannian optimization techniques on the manifold of tensors of fixed multilinear rank. More specifically, a variant of the nonlinear conjugate gradient method is developed. Paying particular attention to efficient implementation, our algorithm scales linearly in the size of the tensor. Examples with synthetic data demonstrate good recovery even if the vast majority of the entries are unknown. We illustrate the use of the developed algorithm for the recovery of multidimensional images and for the approximation of multivariate functions.

CHEBINT

We present the fast approximation of multivariate functions based on Chebyshev series for two types of Chebyshev lattices and show how a fast Fourier transform (FFT) based discrete cosine transform (DCT) can be used to reduce the complexity of this operation. Approximating multivariate functions using rank-1 Chebyshev lattices can be seen as a one-dimensional DCT while a full-rank Chebyshev lattice leads to a multivariate DCT. We also present a MATLAB/Octave toolbox which uses this fast algorithms to approximate functions on a axis aligned hyper-rectangle. Given a certain accuracy of this approximation, interpolation of the original function can be achieved by evaluating the approximation while the definite integral over the domain can be estimated based on this Chebyshev approximation. We conclude with an example for both operations and actual timings of the two methods presented.

A novel piecewise multivariate function approximation method via universal matrix representation

Multivariance in science and engineering causes problematic situations even for continous and discrete cases. One way to overcome this situation is to decrease the multivariance level of the problem by using a divide—and—conquer based method. In this sense, Enhanced Multivariance Product Representation (EMPR) plays a part in the considered scenario and acts successfully. This method brings up a finite expansion to represent a multivariate function in terms of less-variate functions with the assistance of univariate support functions. This work aims to propose a new EMPR based algorithm which has two new features that improves the determination process of each expansion component through Fluctuation Free Integration method, which is an efficient method in evaluating multiple integrals through a universal matrix representation, and increases the approximation quality through inserting a piecewise structure into the standard EMPR algorithm. This new method is called Fluctuation Free Integration based piecewise EMPR. Some numerical implementations are also given to examine the performance of this proposed method.

Decomposition and approximation of multivariate functions on the cube

In this paper, we present a decomposition method of multivariate functions. This method shows that any multivariate function f on [0, 1] d is a finite sum of the form Σ j ϕ j ψ j , where each ϕ j can be extended to a smooth periodic function, each ψ j is an algebraic polynomial, and each φ j ψ j is a product of separated variable type and its smoothness is same as f. Since any smooth periodic function can be approximated well by trigonometric polynomials, using our decomposition method, we find that any smooth multivariate function on [0, 1] d can be approximated well by a combination of algebraic polynomials and trigonometric polynomials. Meanwhile, we give a precise estimate of the approximation error.

A method of multivariable Hermite basis function approximation

A method of multivariable (multivariate) Hermite function based approximation is presented and discussed. The multivariable basis is constructed as a product of one-variable Hermite functions with adjustable scaling parameters. Thanks basis orthonormality, the approximated function expansion coefficients are calculated by using explicit, non-search formulae. The scaling parameters are determined via a search algorithm. Initially, an excessive number of functions in the basis is calculated, then a simple pruning method is applied. Only those are taken which contribute the most to error decrease, down to a desired level. The method ensures a very good generalization property. This claim is supported by both theoretical considerations and working examples.

Approximation of multivariate periodic functions on the [formula omitted] space with a Gaussian measure

This paper is devoted to studying the approximation of multivariate periodic functions in the average case setting. We equip the L2 space of multivariate periodic functions with a Gaussian measure μ such that its Cameron–Martin space is the anisotropic multivariate periodic space. With respect to this Gaussian measure, we discuss the best approximation of functions by trigonometric polynomials with harmonics from parallelepipeds and the approximation by the corresponding anisotropic Fourier partial summation operators and Vallée-Poussin operators, and get the average error estimation. We shall show that, in the average case setting, with the average being with respect to this Gaussian measure μ, the anisotropic trigonometric polynomial subspaces are order optimal in the Lq metric for 1⩽q<∞, and the anisotropic Fourier partial summation operators and Vallée-Poussin operators are the order optimal linear operators, which are as good as optimal nonlinear operators in the Lq metric for 1⩽q<∞.

Non-uniform Randomized Sampling for Multivariate Approximation by High Order Parzen Windows

AbstractWe consider approximation of multivariate functions in Sobolev spaces by high order Parzen windows in a non-uniform sampling setting. Sampling points are neither i.i.d. nor regular, but are noised from regular grids by non-uniform shifts of a probability density function. Sample function values at sampling points are drawn according to probability measures with expected values being values of the approximated function. The approximation orders are estimated by means of regularity of the approximated function, the density function, and the order of the Parzen windows, under suitable choices of the scaling parameter.

The errors of simultaneous approximation of multivariate functions by neural networks

There have been many studies on the simultaneous approximation capability of feed-forward neural networks (FNNs). Most of these, however, are only concerned with the density or feasibility of performing simultaneous approximations. This paper considers the simultaneous approximation of algebraic polynomials, employing Taylor expansion and an algebraic constructive approach, to construct a class of FNNs which realize the simultaneous approximation of any smooth multivariate function and all of its derivatives. We also present an upper bound on the approximation accuracy of the FNNs, expressed in terms of the modulus of continuity of the functions to be approximated.

Quick approximation of bivariate functions

This paper presents two experiments where participants had to approximate function values at various generalization points of a square, using given function values at a small set of data points. A representative set of standard function approximation models was trained to exactly fit the function values at data points, and models' responses at generalization points were compared to those of humans. Then one defined a large class of possible models (including the best two identified predictors) and the class maximal possible prediction accuracy was evaluated. A new model of quick multivariate function approximation belonging to this class was proposed. Its prediction accuracy was close to the maximum possible, and significantly better than that of all other models tested. The new model also provided a significant account of human response variability. Finally, it was shown that this model is more particularly suitable for problems in which the visual system can perform some specific structuring of the data space. This model is therefore considered as a suitable starting point for further investigations into quick multivariate function approximation, which is to date an inadequately explored question in cognitive psychology.

Preface to the special issue, CMAM 2011, no. 3.

Abstract In the recent years, multidimensional numerical simulations with tensor-structured data formats have been recognized as the basic concept for breaking the "curse of dimensionality". Modern applications of tensor methods include the challenging high-dimensional problems of material sciences, bio-science, stochastic modeling, signal processing, machine learning, and data mining, financial mathematics, etc. The guiding principle of the tensor methods is an approximation of multivariate functions and operators with some separation of variables to keep the computational process in a low parametric tensor-structured manifold. Tensors structures had been wildly used as models of data and discussed in the contexts of differential geometry, mechanics, algebraic geometry, data analysis etc. before tensor methods recently have penetrated into numerical computations. On the one hand, the existing tensor representation formats remained to be of a limited use in many high-dimensional problems because of lack of sufficiently reliable and fast software. On the other hand, for moderate dimensional problems (e.g. in "ab-initio" quantum chemistry) as well as for selected model problems of very high dimensions, the application of traditional canonical and Tucker formats in combination with the ideas of multilevel methods has led to the new efficient algorithms. The recent progress in tensor numerical methods is achieved with new representation formats now known as "tensor-train representations" and "hierarchical Tucker representations". Note that the formats themselves could have been picked up earlier in the literature on the modeling of quantum systems. Until 2009 they lived in a closed world of those quantum theory publications and never trespassed the territory of numerical analysis. The tremendous progress during the very recent years shows the new tensor tools in various applications and in the development of these tools and study of their approximation and algebraic properties. This special issue treats tensors as a base for efficient numerical algorithms in various modern applications and with special emphases on the new representation formats.

Approximation of functions with spatial inhomogeneity based on “true” ortho-ridgelet neural network

To approximate the multivariate functions with spatial inhomogeneity, in this paper we proposed an ortho-ridgelet neural network (ORNN) model. By taking orthonormal ridgelet, which is a “true” ridgelet function different with the “classic” ridgelet, as the activation function of the hidden neurons, the network is characterized of more efficient representation of a set of functions with linear and curvilinear singularities. Although the ortho-ridgelet is initially more complicated to be understood than ridgelet, it can be easily applied to function approximation theory in a more effective manner. Moreover, we extended the ORNN model to the time-varied situation, and the time-varied ortho-ridgelet neural network (TV-ORRN) is established for time-varied function approximation, with an incremental learning of the ridgelets output layer. The theoretical analysis is presented and some experiments are taken, and the experimental results prove its efficiency in approximation of multivariate functions with linear and curvilinear singularities, in both the stationary and time-varied environment.

Adaptive Cross Approximation of Multivariate Functions

In this article we present and analyze a new scheme for the approximation of multivariate functions (d=3,4) by sums of products of univariate functions. The method is based on the Adaptive Cross Approximation (ACA) initially designed for the approximation of bivariate functions. To demonstrate the linear complexity of the schemes, we apply it to large-scale multidimensional arrays generated by the evaluation of functions.

Piecewise linear approximations of multivariate functions: A multiresolution-based compression algorithm suitable for circuit implementation

This paper is concerned with a multiresolution approach to the piecewise-linear approximation of multivariate nonlinear continuous functions. The proposed technique has no restrictions on the number of variables the functions depend on, and is based on the use of piecewise-linear “hat” functions. The approximation levels are related to nested function spaces. The multiresolution approach allows one to define a simple model reduction strategy that is based on a proper error definition. The interest in the piecewise-linear approximations and in the hat functions is motivated by the simplicity of their circuit implementations. The efficiency of the method is tested via two benchmark examples, one of which concerns the approximation of the vector field of a nonlinear dynamical system.

The Chebyshev rank of multivariate polynomials in [formula omitted]-approximation

Weighted best L 1 -approximation of multivariate continuous real-valued functions by multivariate polynomials is studied. The largest possible dimension of the sets of best L 1 -approximations is determined.

Approximation abilities of neuro-fuzzy networks

The paper presents the operation of two neuro-fuzzy systems of an adaptive type, intended for solving problems of the approximation of multi-variable functions in the domain of real numbers. Neuro-fuzzy systems being a combination of the methodology of artificial neural networks and fuzzy sets operate on the basis of a set of fuzzy rules if-then, generated by means of the self-organization of data grouping and the estimation of relations between fuzzy experiment results. The article includes a description of neuro-fuzzy systems by Takaga-Sugeno-Kang (TSK) and Wang-Mendel (WM), and in order to complement the problem in question, a hierarchical structural self-organizing method of teaching a fuzzy network. A multi-layer structure of the systems is a structure analogous to the structure of classic neural ne- tworks. In its final part the article presents selected areas of application of neuro-fuzzy systems in the field of geodesy and surveying engineering. Numerical examples showing how the systems work concerned: the approximation of functions of several variables to be used as algorithms in the Geographic Information Systems (the approximation of a terrain model), the transformation of coordinates, and the prediction of a time series. The accuracy characteristics of the results obtained have been taken into consideration.