Approximation Properties Of Functions Research Articles

On the turnpike to design of deep neural networks: Explicit depth bounds

It is well-known that the training of Deep Neural Networks (DNN) can be formalized in the language of optimal control. In this context, this paper leverages classical turnpike properties of optimal control problems to attempt a quantifiable answer to the question of how many layers should be considered in a DNN. The underlying assumption is that the number of neurons per layer—i.e., the width of the DNN—is kept constant. Pursuing a different route than the classical analysis of approximation properties of sigmoidal functions, we prove explicit bounds on the required depths of DNNs based on asymptotic reachability assumptions and a dissipativity-inducing choice of the regularization terms in the training problem. Numerical results obtained for the two spiral task data set for classification indicate that the proposed constructive estimates can provide non-conservative depth bounds.

On [formula omitted]-Gamma type operators in a polynomial weighted space

The objective of this paper is to examine various approximation properties of the q-Gamma operators developed by Cheng et al. (2020) in the context of a polynomial weighted space. Following that, we modify these operators to investigate the approximation properties of differentiable functions in the same space, providing evidence that the modified operators have a better rate of convergence.

On the convergence properties of generalized Szász–Kantorovich type operators involving Frobenious–Euler–Simsek-type polynomials

<p>This work focuses on the study of approximation properties of functions by Szász type operators involving Frobenius–Euler–Simsek-type polynomials, which have become more popular recently because of their special characteristics and functional organization. The convergence properties such as uniformly convergence and pointwise convergence in terms of modulus of continuity and Peetre-$ \it K $ functional are investigated with the help of these sequences of operators in depth. This paper also includes the estimation of the error of the approximation of these sequences of operators to some particular class of functions. The estimates are depicted using the Maple scientific computing program and presented in tables.</p>

Sparse Grid Approximation in Weighted Wiener Spaces

We study approximation properties of multivariate periodic functions from weighted Wiener spaces by sparse grid methods constructed with the help of quasi-interpolation operators. The class of such operators includes classical interpolation and sampling operators, Kantorovich-type operators, scaling expansions associated with wavelet constructions, and others. We obtain the rate of convergence of the corresponding sparse grid methods in weighted Wiener norms as well as analogues of the Littlewood–Paley-type characterizations in terms of families of quasi-interpolation operators.

General function approximation of a class of cascade chaotic fuzzy systems

This paper presents an innovative architecture called cascade chaotic fuzzy system (CCFS) for the function approximation and chaotic modeling. The proposed model can dominate complications in the type-2 fuzzy systems and increase the chaotic performance of a whole framework. The proposed cascade structure is based on combining two or more one-dimensional chaotic maps. The combination provides a new chaotic map with more high nonlinearity than its grain maps. The fusion of cascade chaotic structure into the neurons of the membership layer of a conventional fuzzy system makes the CCFS more capable of confronting nonlinear problems. Based on the General Function Approximation and Stone-Weierstrass theorem, we show that the proposed model has the function approximation property. By analyzing the bifurcation diagram and applying the CCFS to the problem of chaotic modeling, the new model is investigated. Simulation results and analysis are demonstrated to illustrate the concept of general function approximation.

On approximation properties of functions by means of Fourier and Faber series in weighted Lebesgue spaces with variable exponent

In this paper the approximation of functions by linear means of Fourier series in weighted variable exponent Lebesgue spaces was studied. This result was applied to the approximation of the functions by linear means of Faber series in Smirnov classes with variable exponent defined on simply connected domain of the complex plane.

Approximation Properties of Newman Type Interpolation Rational Functions with Fewer Nodes

Approximation Properties of Newman Type Interpolation Rational Functions with Fewer Nodes

Approximation properties of wavelets on p-adic fields

Refinable functions play an important role in the construction and properties of wavelets. Basically, most of the wavelets are generated from refinable functions. In this paper, a study on the approximation properties of refinable functions on [Formula: see text]-adic fields is carried out with necessary theoretical background. Various equivalent forms of approximation order and the connection between the approximation order and the Strang–Fix condition are derived. Finally a characterization for the approximation order of a refinable function is given in terms of order of the sum rules associated with the refinement mask.

Scale-Free Fractal Interpolation

An iterated function system that defines a fractal interpolation function, where ordinate scaling is replaced by a nonlinear contraction, is investigated here. In such a manner, fractal interpolation functions associated with Matkowski contractions for finite as well as infinite (countable) sets of data are obtained. Furthermore, we construct an extension of the concept of α-fractal interpolation functions, herein called R-fractal interpolation functions, related to a finite as well as to a countable iterated function system and provide approximation properties of the R-fractal functions. Moreover, we obtain smooth R-fractal interpolation functions and provide results that ensure the existence of differentiable R-fractal interpolation functions both for the finite and the infinite (countable) cases.

NCFS: new chaotic fuzzy system as a general function approximator

Conventional fuzzy systems (type-1 and type-2) are universal approximators. The goal of this paper is to design and implement a new chaotic fuzzy system (NCFS) based on the Lee oscillator for function approximation and chaotic modelling. NCFS incorporates fuzzy reasoning of the fuzzy systems, self-adaptation of the neural networks, and chaotic signal generation in a unique structure. These features enable the structure to handle uncertainties by generating new information or by chaotic search among prior knowledge. The fusion of chaotic structure into the neurons of the membership layer of a conventional fuzzy system makes the NCFS more capable of confronting nonlinear problems. Based on the GFA and Stone-Weierstrass theorems, we show that the proposed model has the function approximation property. The NCFS performance is investigated by applying it to the problem of chaotic modelling. Simulation results are demonstrated to illustrate the concept of function approximation.

Adaptive multi‐dimensional Taylor network tracking control for nonlinear systems with input saturation and full state time‐varying constraints

SummaryIn this article, an adaptive multi‐dimensional Taylor network (MTN) control approach is developed for nonlinear systems subject to input saturation and full state time‐varying constraints. Using the function approximation property of MTN, an adaptive tracking control method is developed by incorporating the concept of time‐varying state constraints into the construct of barrier Lyapunov functions. The proposed approach not only ensures all closed‐loop system signals are bounded but also that all states of the system never violate the given time‐varying constraints. The most important advantage of the proposed control strategy is that it can obtain satisfactory control effects with low computational cost. Finally, two examples are given to validate the effectiveness of the proposed method.

Differential radial basis function network for sequence modelling

We propose a differential radial basis function (RBF) network termed RBF-DiffNet—whose hidden layer blocks are partial differential equations (PDEs) linear in terms of the RBF—to make the baseline RBF network robust to noise in sequential data. Assuming that the sequential data derives from the discretisation of the solution to an underlying PDE, the differential RBF network learns constant linear coefficients of the PDE, consequently regularising the RBF network by following modified backward-Euler updates. We experimentally validate the differential RBF network on the logistic map chaotic timeseries as well as on 30 real-world timeseries provided by Walmart in the M5 forecasting competition. The proposed model is compared with the normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM) blocks. From the experimental results, RBF-DiffNet consistently shows a marked reduction in the prediction error over the baseline RBF network (e.g., 41% reduction in the root mean squared scaled error on the M5 dataset, and 53% reduction in the mean absolute error on the logistic map); RBF-DiffNet also shows a comparable performance to the LSTM ensemble but requires 99% less computational time. Our proposed network consequently enables more accurate predictions—in the presence of observational noise—in sequence modelling tasks such as timeseries forecasting that leverage the model interpretability, fast training, and function approximation properties of the RBF network.

Deep-Learning-Enhanced NOMA Transceiver Design for Massive MTC: Challenges, State of the Art, and Future Directions

Non-orthogonal multiple access (NOMA) is a promising evolution path to meet the requirements of massive machine type communications (mMTC) in 5G and beyond. However, the deployment of NOMA is hindered by the non-unified signal processing architectures of various NOMA schemes and the inflexibility resulting from the offline design paradigm. The block-wise optimized transceivers make its performance far from the limit. The recent breakthrough of deep learning and its positive applications to wireless communications have paved the way to tackle these challenges. This article studies the effectiveness and efficiency of deep learning in enhancing NOMA performance. Specifically, we first present the deep neural network (DNN), which is constructed via a uniform signal processing architecture, and use it as the unified multiuser receiver in both data and model-driven approaches. This enables the end-to-end optimization of NOMA transceivers due to the universal function approximation property of DNN. On the other hand, with DNN we can automatically extract the user access behaviors out of the time-series signals and optimize the transceivers to match these cross-lay-er behaviors. We further analyze the integration of non-orthogonal communication and neural computation to accomplish high-efficiency data transmission at low cost. Finally, we identify some essential future directions of deep-learning-enhanced NOMA from the perspectives of online reconfigurability and adaptability toward the ever changing environment in future mMTC.

A new adaptive deep neural network controller based on sparse auto‐encoder for the antilock bracking system systems subject to high constraints

AbstractWe contribute in the current paper to extend the universal function approximation property of deep learning hybrid strategy to design both : (a) adaptive deep neural network observer to estimate derivatives of the tracking error dynamics (b) and robust deep neural network (DNN) output feedback control scheme (OFCS) that will overcome successfully effects of both parametric variations and modeling errors. First, the designed controller employs OFCS to linearize the partially known antilock bracking (ABS) dynamics, and then, the dynamic compensator is involved to stabilize the linearized system. However, conventional controllers suffer from limitations due to the presence of these high uncertainties. Therefore, we aim to demonstrate for the first time in the control area the feasibility of applying deep learning algorithm based on sparse auto‐encoder as an approximator for neglected dynamics and uncertain parameters of the ABS. The estimated states are used as inputs to the neural network (NN) and in the adaptation laws as an error signal. Simulations of the proposed control algorithm based adaptive DNN observer are conducted then compared to bang–bang controller, PI controller, and adaptive controller‐based single hidden layer‐neural network with only one‐neuron in hidden layers (SHL(1N)NN) to demonstrate its practical potential. Furthermore, both feasibility and efficiency of involving deep learning algorithm (DLA) in the control area have been successfully confirmed through robustness test.

Summation-Integral Type Operators Based on Lupas-Jain Functions

We introduce a genuine summation-integral type operators based on Lupaş-Jain type base functions related to the unbounded sequences. We investigated their degree of approximation in terms of modulus of continuity and ????-functional for the functions from bounded and continuous functions space. Furthermore, we give some theorems for the local approximation properties of functions belonging to Lipschitz class. Also, we give Voronovskaja theorem for these operators.

A continuous distributed control algorithm for time‐varying networks of nonlinear agents with input saturation

AbstractIn this article, we explore the possibility of designing a continuous control algorithm for time‐varying networks of high‐order nonlinear agents with input saturation. A class of neural network‐based (NN) distributed control algorithms are proposed for the consensus of nonlinear agents with input saturation and nonidentical unknown control directions (UCDs) under uniformly quasi‐strongly ‐connected graphs. An NN is used to cancel the unknown part of agent dynamics due to the well known NN function approximation property. Also, an adaptive smooth term is employed to compensate for the bounded disturbance, the approximation error. Moreover, an auxiliary system is designed to counteract the effect of input saturation. Furthermore, the control law is augmented with the Nussbaum‐type function to obtain an estimation of the UCDs. It is proven rigorously that the consensus of nonlinear agents with input saturation and nonidentical UCDs can be guaranteed, and the boundedness of all signals of closed‐loop systems is ensured. Finally, simulation examples are given to show the effectiveness of our theoretical results.

On some smooth approximations on thick periodic multi-structures

In this paper we study the approximation properties of measurable and square-integrable functions. In particular we show that any $L^2$-bounded function can be approximated in $L^2$-norm by smooth functions defined on a highly oscillating boundary of thick multi-structures in ${\mathbb{R}}^n$. We derive the norm estimates for the approximating functions and study their asymptotic behaviour.

Approximation Properties of Ridge Functions and Extreme Learning Machines

For a compact set $D\subset\mathbb{R}^{m}$ we consider the problem of approximating a function $f$ over $D$ by sums of ridge functions ${x}\mapsto\varphi({w}^{T}{x})$ with ${w}$ in a given set $\mathcal{W}$. While such sums are dense in $C(D)$ if $\mathcal{W}=\mathbb{R}^{m}$, to understand the effectiveness of these approximations we consider finite $\mathcal{W}$ and estimate the infimum of $\left\Vert f-g\right\Vert _{\infty}$ over $g\in V_{\mathcal{W}}:={span}\left\{ \,{x}\mapsto\varphi({w}^{T}{x})\mid\varphi\in C(\mathbb{R}),\,{w}\in\mathcal{W}\,\right\} $ in terms of the Lipschitz constant of $f$. In particular, we show lower bounds for the worst case approximation of functions of Lipschitz constant one by considering unapproximable functions $f$ ($\left\Vert f-g\right\Vert _{\infty}\geq\left\Vert f\right\Vert _{\infty}$ for any $g\in V_{\mathcal{W}}$) of Lipschitz constant one. Accurate approximations appear to require $\left|\mathcal{W}\right|=\Omega(m^{2})$ for $D$ a unit hypercube in $\mathbb{R}^{m}$: if $\left|\mathcal{W}\right|\leq\frac{1}{2}m(m-1)$ and $m\geq2$, then $\sup_{f}\inf_{g\in V_{\mathcal{W}}}\left\Vert f-g\right\Vert _{\infty}\geq1/\sqrt{m-1}$, where $f$ ranges over Lipschitz functions of Lipschitz constant one. Similar results hold for sums of products of pairs of ridge functions with weight vectors in $\mathcal{W}$.

Control Volume Isogeometric Analysis for groundwater flow modeling in heterogeneous porous media

Many important groundwater transport applications require solving the Darcy flow in heterogeneous porous media. Flow simulations, especially in large, highly heterogeneous aquifers, require extensive computational resources, a multiresolution (multiscale) approach to resolve the different heterogeneity scales and an accurate calculation of the velocity field. Common methods, such as finite volumes and elements, assume a discontinuous conductivity field introducing velocity discontinuities along the cell or element interfaces due to using classic discrete operators or Lagrangian basis functions. Over the last decade, the development of isogeometric analysis (IGA) eliminates many of the aforementioned limitations bridging the gap between CAD and numerical analysis. Since classic IGA uses the Galerkin and collocation approach, in this paper, we present a third concept in the form of Control Volume IsoGeometric Analysis (CV-IGA) enabling local and global mass conservation as well as multiresolution description of all heterogeneity scales. Due to the approximation properties of spline basis functions, the velocity field and its derivatives are continuous and are obtained by an optimal convergence rate. The CV-IGA methodology is verified with 2-D numerical and stochastic benchmark flow simulations, including comparisons with classic methods and two other IGA formulations as well as the convergence analysis of the head and velocity fields for different orders of Fup and B-spline basis functions.

DeepNOMA: A Unified Framework for NOMA Using Deep Multi-Task Learning

Non-orthogonal multiple access (NOMA) will provide massive connectivity for future Internet of Things. However, the intrinsic non-orthogonality in NOMA makes it non-trivial to approach the performance limit with only conventional communication-theoretic tools. In this paper, we resort to deep multi-task learning for end-to-end optimization of NOMA, by regarding the overlapped transmissions as multiple distinctive but correlated learning tasks. First of all, we establish a unified multi-task deep neural network (DNN) framework for NOMA, namely DeepNOMA, which consists of a channel module, a multiple access signature mapping module, namely DeepMAS, and a multi-user detection module, namely DeepMUD. DeepMAS and DeepMUD are automatically trained in a data-driven fashion, and a multi-task balancing technique is then proposed to guarantee fairness among tasks as well as to avoid local optima. To further exploit the benefits of communication-domain expertise, we introduce constellation shape prior and inter-task interference cancellation structure into DeepMAS and DeepMUD, respectively. These sophisticated designs help to reduce the implementation complexity without sacrificing DNN’s universal function approximation property, which makes DeepNOMA a universal transceiver optimization approach. Detailed experiments and link-level simulations show that higher transmission accuracy and lower computational complexity can be simultaneously achieved by DeepNOMA under various channel models, compared with state-of-the-art.