Function Approximation Research Articles

Anti-derivatives approximator for enhancing physics-informed neural networks

This study presents a novel strategy for constructing an approximator for arbitrary univariate functions. The proposed approximation utilizes the anti-derivatives of a Fourier series expansion for the presumed piecewise function, resulting in a remarkable feature that enables the simultaneous approximation of an arbitrary function and its anti-derivatives. These anti-derivatives can be employed to discover solution curves for systems of ordinary differential equations based on an optimization scheme, even in the presence of chaotic dynamics. Additionally, the anti-derivatives approximator is extended as an adaptive activation function for physics-informed neural networks, leveraging the high-order differentiability of the anti-derivatives. Systematic experiments have demonstrated the outstanding merits of the proposed anti-derivatives-based approximator, including its ability to construct regression models for volatile data and their anti-derivatives, solve differential equations, and enhance the capabilities of physics-informed neural networks.

Robust optimal control for uncertain wheeled mobile robot based on reinforcement learning: ADP approach

This paper presents a robust optimal control approach for the wheel mobile robot system, which considers the effects of external disturbances, uncertainties, and wheel slipping. The proposed method utilizes an adaptive dynamic programming (ADP) technique in conjunction with a disturbance observer. Initially, the system's state space model is formulated through the utilization of kinematic and dynamic models. Subsequently, the ADP method is employed to establish an online adaptive optimal controller, which solely relies on a single neural network for the purpose of function approximation. The utilization of the disturbance observer in conjunction with the compensation controller serves to alleviate the effects of disturbances. The Lyapunov theorem establishes the stability of the complete closed-loop system and the convergence of the weights of the neural network. The proposed approach has been shown to be effective through simulation under the effect of the disturbances and the change of the desired trajectory.

Sharp upper and lower estimates for the approximation of bivariate functions by sums of univariate functions

Sharp upper and lower estimates for the approximation of bivariate functions by sums of univariate functions

Semiparametric estimation of average treatment effects in observational studies

AbstractWe propose a semiparametric method to estimate average treatment effects in observational studies based on the assumption of unconfoundedness. Assume that the propensity score model and outcome model are a general single index model, which are estimated by the kernel method and the unknown index parameter is estimated via linearized maximum rank correlation method. The proposed estimator is computationally tractable, allows for large dimension covariates and not involves the approximation of link functions. We showed that the proposed estimator is consistent and asymptotically normally distributed. In general, the proposed estimator is superior to existing methods when the model is incorrectly specified. We also provide an empirical analysis on the average treatment effect and average treatment effect on the treated of 401(k) eligibility on net financial assets.

High-Order Extended Kalman Filter for State Estimation of Nonlinear Systems

In general, the extended Kalman filter (EKF) has a wide range of applications, aiming to minimize symmetric loss function (mean square error) and improve the accuracy and efficiency of state estimation. As the nonlinear model complexity increases, rounding errors gradually amplify, leading to performance degradation. After multiple iterations, divergence may occur. The traditional extended Kalman filter cannot accurately estimate the nonlinear model, and these errors still have an impact on the accuracy. To improve the filtering performance of the extended Kalman filter (EKF), this paper proposes a new extended Kalman filter (REKF) method that utilizes the statistical properties of the rounding error to enhance the estimation accuracy. After establishing the state model and measurement model, the residual term is used to replace the higher-order term in the Taylor expansion, and the least squares method is applied to identify the residual term step by step. Then, the iterative process of updating the extended Kalman filter is carried out. Within the Kalman filter framework, a higher-order rounding error-based extended Kalman filter (REKF) is designed for the joint estimation of rounding error and random variables, and the solution method for the rounding error is considered for the multilevel approximation of the original function. Through numerical simulations on a general nonlinear model, the higher-order rounding error-based extended Kalman filter (REKF) achieves better estimation results than the extended Kalman filter (EKF) and improves the filtering accuracy by utilizing the higher-order rounding error information, which also proves the effectiveness of the proposed method.

Accelerated Gradient Approach For Deep Neural Network-Based Adaptive Control of Unknown Nonlinear Systems.

Recent connections in the adaptive control literature to continuous-time analogs of Nesterov's accelerated gradient method have led to the development of new real-time adaptation laws based on accelerated gradient methods. However, previous results assume that the system's uncertainties are linear-in-the-parameters (LIP). To compensate for non-LIP uncertainties, our preliminary results developed a neural network (NN)-based accelerated gradient adaptive controller to achieve trajectory tracking for nonlinear systems; however, the development and analysis only considered single-hidden-layer NNs. In this article, a generalized deep NN (DNN) architecture with an arbitrary number of hidden layers is considered, and a new DNN-based accelerated gradient adaptation scheme is developed to generate estimates of all the DNN weights in real-time. A nonsmooth Lyapunov-based analysis is used to guarantee the developed accelerated gradient-based DNN adaptation design achieves global asymptotic tracking error convergence for general nonlinear control affine systems subject to unknown (non-LIP) drift dynamics and exogenous disturbances. A comprehensive set of simulation studies are conducted on a two-state nonlinear system, a robotic manipulator, and a complex 20-D nonlinear system to demonstrate the improved performance of the developed method. Our simulation studies demonstrate enhanced tracking and function approximation performance from both DNN architectures and accelerated gradient adaptation.

New asymptotic expansion formula via Malliavin calculus and its application to rough differential equation driven by fractional Brownian motion

This paper presents a novel generic asymptotic expansion formula of expectations of multidimensional Wiener functionals through a Malliavin calculus technique. The uniform estimate of the asymptotic expansion is shown under a weaker condition on the Malliavin covariance matrix of the target Wiener functional. In particular, the method provides a tractable expansion for the expectation of an irregular functional of the solution to a multidimensional rough differential equation driven by fractional Brownian motion with Hurst index H < 1 / 2, without using complicated fractional integral calculus for the singular kernel. In a numerical experiment, our expansion shows a much better approximation for a probability distribution function than its normal approximation, which demonstrates the validity of the proposed method.

Point Spread Function Approximation of High-Rank Hessians with Locally Supported Nonnegative Integral Kernels

Point Spread Function Approximation of High-Rank Hessians with Locally Supported Nonnegative Integral Kernels

Efficient function approximation in enriched approximation spaces

Abstract An enriched approximation space is the span of a conventional basis with a few extra functions included, for example to capture known features of the solution to a computational problem. Adding functions to a basis makes it overcomplete and, consequently, the corresponding discretized approximation problem may require solving an ill-conditioned system. Recent research indicates that these systems can still provide highly accurate numerical approximations under reasonable conditions. In this paper we propose an efficient algorithm to compute such approximations. It is based on the AZ algorithm for overcomplete sets and frames, which simplifies in the case of an enriched basis. In addition, analysis of the original AZ algorithm and of the proposed variant gives constructive insights on how to achieve optimal and stable discretizations using enriched bases. We apply the algorithm to examples of enriched approximation spaces in literature, including a few nonstandard approximation problems and an enriched spectral method for a 2D boundary value problem, and show that the simplified AZ algorithm is indeed stable, accurate and efficient.

FAST: Filter-Adapted Spatio-Temporal Sampling for Real-Time Rendering

Stochastic sampling techniques are ubiquitous in real-time rendering, where performance constraints force the use of low sample counts, leading to noisy intermediate results. To remove this noise, the post-processing step of temporal and spatial denoising is an integral part of the real-time graphics pipeline. The main insight presented in this paper is that we can optimize the samples used in stochastic sampling such that the post-processing error is minimized. The core of our method is an analytical loss function which measures post-filtering error for a class of integrands --- multidimensional Heaviside functions. These integrands are an approximation of the discontinuous functions commonly found in rendering. Our analysis applies to arbitrary spatial and spatiotemporal filters, scalar and vector sample values, and uniform and non-uniform probability distributions. We show that the spectrum of Monte Carlo noise resulting from our sampling method is adapted to the shape of the filter, resulting in less noisy final images. We demonstrate improvements over state-of-the-art sampling methods in three representative rendering tasks: ambient occlusion, volumetric ray-marching, and color image dithering. Common use noise textures, and noise generation code is available at https://github.com/electronicarts/fastnoise.

Electronic heat conductivity in a two-temperature state

Heat transport in solids is governed by two fundamental contributions, atomic and electronic. The electronic energy transport in transient excited states is a defining factor in the problem of ultrafast material irradiation. Here, we calculate the electronic heat conductivity at elevated electron temperatures up to 40,000 K. We apply the novel combined method of tight binding formalism to calculate the electron-phonon contribution to the electronic heat conductivity, and the linear response theory (in the single-pole Ritchie-Howie loss function approximation) for its electron-electron counterpart, implemented in the hybrid code XTANT-3. It allows us to evaluate the electronic heat conductivity in a wide range of materials – fcc metals: Al, Ca, Ni, Cu, Sr, Y, Zr, Rh, Pd, Ag, Ir, Pt, Au, and Pb; hcp metals: Mg, Sc, Ti, Co, Zn, Tc, Ru, Cd, Hf, Re, and Os; bcc metals: V, Cr, Fe, Nb, Mo, Ba, Ta, and W; other metals: Sn, Ga, In, Mn, Te, and Se; semimetal graphite; semiconductors – group IV: Si, Ge, and SiC; group III-V: AlAs, AlP, GaP, GaAs, and GaSb; oxides: ZnO, TiO2, and Cu2O; and others: PbI2, ZnS, and B4C.

Characterization of Deformational Isomerization Potential and Interconversion Dynamics with Ultrafast X-ray Solution Scattering.

Dimeric complexes composed of d8 square planar metal centers and rigid bridging ligands provide model systems to understand the interplay between attractive dispersion forces and steric strain in order to assist the development of reliable methods to model metal dimer complexes more broadly. [Ir2 (dimen)4]2+ (dimen = para-diisocyanomenthane) presents a unique case study for such phenomena, as distortions of the optimal structure of a ligand with limited conformational flexibility counteract the attractive dispersive forces from the metal and ligand to yield a complex with two ground state deformational isomers. Here, we use ultrafast X-ray solution scattering (XSS) and optical transient absorption spectroscopy (OTAS) to reveal the nature of the equilibrium distribution and the exchange rate between the deformational isomers. The two ground state isomers have spectrally distinct electronic excitations that enable the selective excitation of one isomer or the other using a femtosecond duration pulse of visible light. We then track the dynamics of the nonequilibrium depletion of the electronic ground state population─often termed the ground state hole─with ultrafast XSS and OTAS, revealing a restoration of the ground state equilibrium in 2.3 ps. This combined experimental and theoretical study provides a critical test of various density functional approximations in the description of bridged d8-d8 metal complexes. The results show that density functional theory calculations can reproduce the primary experimental observations if dispersion interactions are added, and a hybrid functional, which includes exact exchange, is used.

Advanced fusion of MTM-LSTM and MLP models for time series forecasting: An application for forecasting the solar radiation

Accurate time series forecasting has become increasingly important across various domains such as finance, energy, and medicine. This study introduces an innovative hybrid model that leverages the power of neural networks, precisely Many To Many LSTM (MTM LSTM) and Multilayer Perceptron (MLP), to improve time series forecasting accuracy. In this new combination, we trained network MTM LSTM to approximate the target at each step, and finally, we used network MLP to combine these approximations. To perform the evaluation, we made a forecast for the amount of solar energy radiation in the city of Mashhad, Iran. The experiment results concerning MSE and MAE showed that the proposed method with five lags outperforms the standard models.We hypothesize that MTM LSTM can effectively capture solar radiation's intricate temporal dependencies and nonlinearity. At the same time, MLP can enhance function approximation by modeling complex interactions, resulting in improved forecast accuracy. By employing the hybrid MTM LSTM and MLP model, we achieved improved accuracy in predicting solar energy radiation, which has significant implications for the renewable energy sector and its energy management and planning applications. This research advances time series forecasting techniques, highlighting the effectiveness of combining neural networks to address complex and dynamic patterns in time-dependent data.Overall, our findings underscore the potential and efficacy of the proposed hybrid model as a robust tool for accurate time series forecasting in various domains, supporting effective decision-making and planning processes.

Sliding Mode Speed Control in Synchronous Motors for Agriculture Machinery: A Chattering Suppression Approach

Synchronous motors have extended their presence in different applications, specifically in high-demand environments such as agronomy. These uses need advanced and better control strategies to improve energy efficiency. Within this context, sliding mode control has demonstrated effectiveness in electric machine control due to its advantages in robustness and quick adaptation to uncertain dynamic system disturbances. Nevertheless, this control technique presents the undesirable chattering phenomenon due to the discontinuous control action. This paper introduces a novel speed integral control scheme based on sliding modes for synchronous motors. This approach is designed to track smooth speed profiles and is evaluated through several numeric simulations to verify its robustness against variable torque loads. This approach addresses using electric motors for different applications such as irrigation systems, greenhouses, pumps, and others. Moreover, to address the chattering problem, different sign function approximations are evaluated in the control scheme. Then, the most effective functions for suppressing the chattering phenomenon through extensive comparative analysis are identified. Integral compensation in this technique demonstrates improvement in motor performance, while sign function approximations show a chattering reduction. Different study cases prove the robustness of this control scheme for large-scale synchronous motors. The simulation results validate the proposed control scheme based on sliding modes with integral compensation, by achieving chattering reduction and obtaining an efficient control scheme against uncertain disturbances in synchronous motors for agronomy applications.

Current density functional framework for spin-orbit coupling: Extension to periodic systems.

Spin-orbit coupling induces a current density in the ground state, which consequently requires a generalization for meta-generalized gradient approximations. That is, the exchange-correlation energy has to be constructed as an explicit functional of the current density, and a generalized kinetic energy density has to be formed to satisfy theoretical constraints. Herein, we generalize our previously presented formalism of spin-orbit current density functional theory [Holzer et al., J. Chem. Phys. 157, 204102 (2022)] to non-magnetic and magnetic periodic systems of arbitrary dimension. In addition to the ground-state exchange-correlation potential, analytical derivatives such as geometry gradients and stress tensors are implemented. The importance of the current density is assessed for band gaps, lattice constants, magnetic transitions, and Rashba splittings. In the latter, the impact of the current density may be larger than the deviation between different density functional approximations.

A Novel Consolidation Analysis Framework: Universal Function Approximators Regularized by Physical Principles

Analytical and numerical techniques are widely used to analyse and interpret soil consolidation problems. An important limitation is the requirement for significant geotechnical knowledge and expertise to derive ‘true’ solutions. This study proposes an alternative: a universal function approximator regularized with known physical principles. The proposed approach here advances previous work to solve one-dimensional (1D) consolidation considering both self-weight and large strains, as well as two-dimensional (2D) consolidation by vertical drains. Initial and boundary conditions of the studied consolidation problems are first strongly and weakly enforced for comparison. A neural network is adopted here as the function approximator. To boost prediction accuracy, a novel strategy is proposed to adaptively sample data points for training. An estimate of epistemic uncertainty is achieved using the confidence interval of ensembled multiple outputs. The results show that the proposed approach accurately predicts the behaviours of complex consolidation processes. Results also indicate that regularization using weak physical constraints can alleviate the imbalance of back-propagated gradients of different loss terms and, in turn, achieve higher accuracy. The proposed method is generic, mesh-free, more robust and can be applied to a wide range of geotechnical problems.

Thermodynamics of the gas-phase dimerization of formic acid: Fully anharmonic finite temperature calculations at the CCSD(T) and many DFT levels.

A proof-of-concept study is undertaken to demonstrate the utility of the machine learning combined with the thermodynamic perturbation theory (MLPT) to test the accuracy of electronic structure methods in finite-temperature thermodynamic calculations. As a test example, formic acid dimer is chosen, which is one of the systems included in the popular benchmark set S22 [Jurečka et al., Phys. Chem. Chem. Phys. 8, 1985-1993 (2006)]. Starting from the explicit molecular dynamics and thermodynamic integration performed at the PBE + D2 level, the MLPT is used to obtain fully anharmonic dimerization free and internal energies at the reference quality CCSD(T) level and 19 different density functional approximations, including GGA, meta-GGA, non-local, and hybrid functionals with and without dispersion corrections. Our finite-temperature results are shown to be both qualitatively and quantitatively different from those obtained using the conventional benchmarking strategy based on fixed structures. The hybrid functional HSE06 is identified as the best performing approximate method tested, with the errors in free and internal energies of dimerization being 36 and 41meV, respectively.

Self-Consistent Convolutional Density Functional Approximations: Application to Adsorption at Metal Surfaces.

The exchange-correlation (XC) functional in density functional theory is used to approximate multi-electron interactions. A plethora of different functionals are available, but nearly all are based on the hierarchy of inputs commonly referred to as "Jacob's ladder." This paper introduces an approach to construct XC functionals with inputs from convolutions of arbitrary kernels with the electron density, providing a route to move beyond Jacob's ladder. We derive the variational derivative of these functionals, showing consistency with the generalized gradient approximation (GGA), and provide equations for variational derivatives based on multipole features from convolutional kernels. A proof-of-concept functional, PBEq, which generalizes the PBE framework with being a spatially-resolved function of the monopole of the electron density, is presented and implemented. It allows a single functional to use different GGAs at different spatial points in a system, while obeying PBE constraints. Analysis of the results underlines the importance of error cancellation and the XC potential in data-driven functional design. After testing on small molecules, bulk metals, and surface catalysts, the results indicate that this approach is a promising route to simultaneously optimize multiple properties of interest.

Multi-Year Cropland Mapping Based on Remote Sensing Data: A Case Study for the Khabarovsk Territory, Russia

Cropland mapping using remote sensing data is the basis for effective crop monitoring, crop rotation control, and the detection of irrational land use. Classification using Normalized Difference Vegetation Index (NDVI) time series from multi-year data requires additional time costs, especially when sentinel data are sparse. Approximation by nonlinear functions was proposed to solve this problem. Time series of weekly NDVI composites were plotted using multispectral Sentinel-2 (Level-2A) images at a resolution of 10 m for sites in Khabarovsk District from April to October in the years 2021 and 2022. Missing values due to the lack of suitable images for analysis were recovered using cubic polynomial, Fourier series, and double sinusoidal function approximation. The classes that were considered included crops, namely, soybean, buckwheat, oat, and perennial grasses, and fallow. The mean absolute percentage error (MAPE) of each class fitting was calculated. It was found that Fourier series fitting showed the highest accuracy, with a mean error of 8.2%. Different classifiers, such as the support vector machine (SVM), random forest (RF), and gradient boosting (GB), were comparatively evaluated. The overall accuracy (OA) for the site pixels during the cross-validation (Fourier series restored) was 67.3%, 87.2%, and 85.9% for the SVM, RF, and GB classifiers, respectively. Thus, it was established that the best result in terms of combined accuracy, performance, and limitations in cropland mapping was achieved by composite construction using Fourier series and machine learning using GB. Similar results should be expected in regions with similar cropland structures and crop phenological cycles, including other regions of the Far East.

On application of Gaussian kernels and Laplace functions combined with Kolmogorov's theorem for approximation of functions of several variables

A special class of approximations of measurable functions of several variables on the unit coordinate cube is investigated. The class is constructed on the base of Kolmogorov's theorem (in version by Sprecher–Golubkov) stating that a continuous function $f$ of several variables can be represented as a finite superposition of continuous single-variable functions — so called outer functions (which depend of $f$) and inner one $\Psi$ (which is independent of $f$ and is monotone). In the case of continuous functions $f$ the class under study is obtained with outer functions approximated by linear combinations of quadratic exponentials (also known as Gaussian functions or Gaussian kernels) and with the inner function $\Psi$ approximated by Laplace functions. As is known, a measurable function $f$ can be approximated by a continuous one (up to a set of small measure) with the help of classical Luzin's theorem. The effectiveness of such approach is based on assertions that, firstly, the Mexican hat mother wavelet on any fixed bounded interval can be approximated as accurately as desired by a linear combination of two Gaussian functions, and, secondly, that a continuous monotone function on such an interval can be approximated as accurately as desired by a linear combination of translations and dilations of the Laplace integral (in other words, Laplace functions). It is proved that the class of approximations under study is dense everywhere in the class of continuous multivariable functions on the coordinate cube. For the case of continuous and piecewise continuous functions of two variables, numerical results are presented that confirm the effectiveness of approximations of the studied class.