Function Approximation Research Articles

Efficient and stable SAV-based methods for gradient flows arising from deep learning

The optimization algorithm plays an important role in deep learning and significantly affects the stability and efficiency of the training process, and consequently the accuracy of the neural network approximation. A suitable (initial) learning rate is crucial for the optimization algorithm in deep learning. However, a small learning rate is usually needed to guarantee the convergence of the optimization algorithm, resulting in a slow training process. We develop in this work efficient and energy stable optimization methods for function approximation problems as well as solving partial differential equations using deep learning. In particular, we consider the gradient flows arising from deep learning from the continuous point of view, and employ the particle method (PM) and the smoothed particle method (SPM) for the space discretization while we adopt SAV-based schemes, combined with the adaptive strategy used in Adam algorithm, for the time discretization. To illustrate the effectiveness of the proposed methods, we present a number of numerical tests to demonstrate that the SAV-based schemes significantly improve the efficiency and stability of the training as well as the accuracy of the neural network approximation. We also show that the SPM approach gives a slightly better accuracy than the PM approach. We further demonstrate the advantage of using adaptive learning rate in dealing with more complex problems.

Adaptive Approximation Sliding-Mode Control of an Uncertain Continuum Robot with Input Nonlinearities and Disturbances

This paper develops an adaptive nonsingular fast terminal sliding-mode control (ANFTSMC) scheme for the continuum robot subjected to uncertain dynamics, external disturbances, and input nonlinearities (e.g., actuator deadzones/faults). Concretely, a function approximation technique (FAT) is utilized to estimate unknown robot dynamics and actuator deadzones/faults online. Furthermore, a disturbance observer (DO) is devised to make up for unknown external disturbances. Then, an ANFTSMC scheme combined with FAT and DO is developed, to expedite the restoration of the stability for the continuum robot. The proposed ANFTSMC not only can retain the benefits of traditional terminal sliding-mode control (TSMC), containing easy enforcement, quick response, and robustness to uncertainties but also dispose of the latent singularity for traditional faster TSMC designs. Afterward, the simulation results show that the proposed controller can effectively improve the trajectory tracking accuracy of the continuum robot, and the tracking root-mean-square errors are 0.0115 and 0.0128 rad. Finally, the effectiveness of ANFTSMC scheme is validated by experiments.

Benchmark Study on the Calculation of 207Pb NMR Chemical Shifts.

A benchmark set for the computation of 207Pb nuclear magnetic resonance (NMR) chemical shifts is presented. The PbS50 set includes conformer ensembles of 50 lead-containing molecular compounds and their experimentally measured 207Pb NMR chemical shifts. Various bonding motifs at the Pb center with up to seven bonding partners are included. Six different solvents were used in the measurements. The respective shifts lie in the range between +10745 and -5030 ppm. Several calculation settings are assessed by evaluating computed 207Pb NMR shifts for the use with different density functional approximations (DFAs), relativistic approaches, treatment of the conformational space, and levels for geometry optimization. Relativistic effects were included explicitly with the zeroth order regular approximation (ZORA), for which only the spin-orbit variant was able to yield reliable results. In total, seven GGAs and three hybrid DFAs were tested. Hybrid DFAs significantly outperform GGAs. The most accurate DFAs are mPW1PW with a mean absolute deviation (MAD) of 429 ppm and PBE0 with an MAD of 446 ppm. Conformational influences are small as most compounds are rigid, but more flexible structures still benefit from Boltzmann averaging. Including explicit relativistic treatments such as SO-ZORA in the geometry optimization does not show any significant improvement over the use of effective core potentials (ECPs).

Stochastic LQ optimal control for Markov jumping systems with multiplicative noise using reinforcement learning

The stochastic linear quadratic (LQ) problem of discrete-time linear Markov jumping systems with multiplicative noise is investigated in this paper. Two reinforcement learning algorithms, one model-based and one model-free, are designed. The algorithms can be readily adapted to handle nonlinear cases by employing an appropriate function approximator. In the linear case, the network structure is designed to enable the problem to be solved without succumbing to modeling errors. For the model-free algorithm, an approach based on moving averages is proposed to reduce estimation variance. A proof of convergence is also provided. Finally, the effectiveness of the proposed algorithms is demonstrated through a numerical example.

DYNAMICS OF PLASMA IN A DUAL-FREQUENCY CAPACITIVE DISCHARGE IN AN AR/XE MIXTURE

The spatiotemporal dynamics of the plasma of a symmetrical dual-frequency 81 MHz/1.76 MHz capacitive discharge under the in uence of a low-frequency eld of 1.76 MHz has been studied.Using the method of phase-resolved optical emission spectroscopy, the dynamics of the radiation intensity of argon and xenon in plasma was obtained.The electron energy distribution function at the center of the discharge was measured using a Langmuir probe and the electron density was measured using a microwave probe. The main result is the dynamics of the intensity ratios of the selected argon and xenon lines depending on di erent conditions: at pressures of 40, 200 and 400 mTorr, LF voltage amplitude of 100, 200 and 400 V, input power at 81 MHz of 3 W and 15 W. The dynamics of high-energy electrons is studied based on a two-temperature approximation of the electron energy distribution function.

Advanced Optimal Control Problem and Numerical Method for its Solving

The advanced statement of the optimal control problem is presented. The difference between the extended setting of the problem and the classical one is that the model of the control object consists of two subsystems, a reference model, which generates an optimal motion path and a dynamic model of the control object with a system for stabilizing movement along the optimal trajectory. In the problem, it is necessary to find a program control function whose argument is time and a stabilization system function whose argument is the deviation of the state vector of the control object from the optimal program trajectory. The task has many initial conditions, one of which is used in the search for software control, and the rest for the search for a stabilization system. The control quality criterion is defined as the sum of the original quality criterion for all specified initial conditions. The procedure for trans forming the classical setting of the optimal control problem to an extended setting based on the refinement of the problem for its practical implementation is presented. To solve the extended optimal control problem, a universal numerical method is proposed based on a piecemeal linear approximation of the control function using evolutionary algorithms and symbol regression methods for structurally parametric optimization of the stabilization system function. An example of solving an extended optimal control problem for spatial motion by a quadcopter, which should conduct reconnaissance of a given territory in a minimum time, is given.

Time-discrete momentum consensus-based optimization algorithm and its application to Lyapunov function approximation

In this paper, we study a discrete momentum consensus-based optimization (Momentum-CBO) algorithm which corresponds to a second-order generalization of the discrete first-order CBO [S.-Y. Ha, S. Jin and D. Kim, Convergence of a first-order consensus-based global optimization algorithm, Math. Models Methods Appl. Sci. 30 (2020) 2417–2444]. The proposed algorithm can be understood as the modification of ADAM-CBO, replacing the normalization term by unity. For the proposed Momentum-CBO, we provide a sufficient framework which guarantees the convergence of algorithm toward a global minimum of the objective function. Moreover, we present several experimental results showing that Momentum-CBO has an improved success rate of finding the global minimum compared to vanilla-CBO and show the stability of Momentum-CBO under different initialization schemes. We also show that Momentum-CBO can be used as the alternative of ADAM-CBO which does not have a proper convergence analysis. Finally, we give an application of Momentum-CBO for Lyapunov function approximation using symbolic regression techniques.

A data-driven implicit deep adaptive neuro-fuzzy inference system capable of manifold learning for function approximation

Fuzzy Neural Networks (FNN) have the ability of decision-making based on constructing semi-ellipsoidal clusters in the input space as the antecedent parts of their fuzzy rules. To determine the output value for each input instance, FNNs consider its membership degree to different sub-regions of the input space. However, forming such meaningful sub-regions is not possible in all applications due to the nonlinear interactions among input variables and their low information gain. Indeed, the samples could be distributed on a manifold in the input space. Therefore, to cover the input space, we need lots of rules, each representing a small region of input space. This issue decreases the generalization ability of the model along with its explainability. Consequently, to efficiently form fuzzy rules, first, it is necessary to unfold the manifold by mapping the samples to an appropriate embedding space. Next, the fuzzy rules in the form of semi-ellipsoidal regions should be constructed in this extracted feature space. Deep Fuzzy Neural Networks address this problem by representation learning through stacking multiple cascade mapping layers. In this paper, we propose a novel approach for nonlinear function approximation and time-series prediction problems, based on using the kernel trick to implicitly learn the mapping function to the new feature space. Moreover, to initialize the fuzzy rules, a KNN-based method using the kernel trick is proposed. A hierarchical Levenberg–Marquardt approach is applied to learn the model’s parameters. The performance and structure of the proposed method are studied and compared with some other relevant methods in synthetic and real-world benchmarks. Based on these experiments, the proposed method has the best performance with the most parsimonious architecture. According to these experiments, the test RMSE of the proposed method is 0.002 for Mc-Glass chaotic time-series prediction, 0.015 for a Nonlinear dynamic system identification, 0.0345 for Box–Jenkins nonlinear system identification, 0.0609 for Fuel consumption prediction of automobiles, 10.24 for Sydney stock price tracking, and 0.595 for California housing.

Computational study of oxygen evolution reaction on flat and stepped surfaces of strontium titanate

Oxygen evolution reaction (OER) is reconciled with the bottleneck in hydrogen production via photo- or electrocatalytic water splitting. One of the remedies to overcome this shortcoming is to develop catalytically efficient anode materials. Strontium titanate (STO) is a potential candidate for the OER by referring to recent experimental reports on enhanced photocatalytic activity of faceted nanoparticles. In this paper, we perform electronic structure calculations in the density functional theory approximation to study the OER on flat and stepped surfaces, such as encountered in nano-sized STO. We demonstrate that stepped surfaces reveal higher OER activity than flat surfaces, which is consistent with experimental data. We also observe partial breaking of OER scaling relation on stepped surfaces, albeit this does not necessarily cause higher catalytic activity. Finally, we propose recommendations for thorough implementation of zero-point energy calculations as the calculation method may impact the free-energy landscape of the OER.

Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks

The expressive power of deep neural networks is manifested by their remarkable ability to approximate multivariate functions in a way that appears to overcome the curse of dimensionality. This ability is exemplified by their success in solving high-dimensional problems where traditional numerical solvers fail due to their limitations in accurately representing high-dimensional structures. To provide a theoretical framework for explaining this phenomenon, we analyze the approximation of Hölder functions defined on a d-dimensional smooth manifold M embedded in RD, with d≪D, using deep neural networks. We prove that the uniform convergence estimates of the approximation and generalization errors by deep neural networks with ReLU activation functions do not depend on the ambient dimension D of the function but only on its lower manifold dimension d, in a precise sense. Our result improves existing results from the literature where approximation and generalization errors were shown to depend weakly on D.

Variational principles for the hydrodynamics of the classical one-component plasma

Hydrodynamic equations for a one-component plasma are derived as a unification of the Euler equations with long-range Coulomb interaction. By using a variational principle, these equations self-consistently unify thermodynamics, dispersion laws, nonlinear motion, and conservation laws. In the moderate and strong coupling limits, it is argued that these equations work down to the length scale of the interparticle spacing. The use of a variational principle also ensures that closure is achieved self-consistently. Hydrodynamic equations are evaluated in both the Eulerian frame, where the fluid variables depend on the position in the laboratory, and the Lagrangian frame, where they depend on the position in some reference state, such as the initial position. Each frame has its advantages and our final theory combines elements of both. The properties of longitudinal and transverse dispersion laws are calculated for the hydrodynamic equations. A simple step function approximation for the pair distribution function enables simple calculations that reveal the structure of the equations of motion. The obtained dispersion laws are compared to molecular dynamics simulations and the theory of quasilocalized charge approximation. The action, which gives excellent agreement for both longitudinal and transverse dispersion laws for a wide range of coupling strengths, is elucidated. Agreement with numerical experiments shows that such a hydrodynamic approach can be used to accurately describe a one-component plasma at very small length scales comparable to the average interparticle spacing. The validity of this approach suggests considering nonlinear flows and other systems with long-range interactions in the future.

Approximation by Nörlund type means in the grand Lebesgue spaces with variable exponent

In the present paper the approximation of functions by Nörlund type means in the generalized grand Lebesgue spaces with variable exponent is studied.

Making quantum chemistry compressive and expressive: Toward practical ab‐initio simulation

AbstractAb‐initio quantum chemistry simulations are essential for understanding electronic structure of molecules and materials in almost all areas of chemistry. A broad variety of electronic structure theories and implementations has been developed in the past decades to hopefully solve the many‐body Schrödinger equation in an approximate manner on modern computers. In this review, we present recent progress in advancing low‐rank electronic structure methodologies that rely on the wavefunction sparsity and compressibility to select the important subset of electronic configurations for both weakly and strongly correlated molecules. Representative chemistry applications that require the many‐body treatment beyond traditional density functional approximations are discussed. The low‐rank electronic structure theories have further prompted us to highlight compressive and expressive principles that are useful to catalyze idea of quantum learning models. The intersection of the low‐rank correlated feature design and the modern deep neural network learning provides new feasibilities to predict chemically accurate correlation energies of unknown molecules that are not represented in the training dataset. The results by others and us are discussed to reveal that the electronic feature sets from an extremely low‐rank correlation representation, which is very poor for explicit energy computation, are however sufficiently expressive for capturing and transferring electron correlation patterns across distinct molecular compositions, bond types and geometries.This article is categorized under: Electronic Structure Theory > Ab Initio Electronic Structure Methods Software > Quantum Chemistry Software > Simulation Methods

A numerical Poisson solver with improved radial solutions for a self-consistent locally scaled self-interaction correction method

The universal applicability of density functional approximations is limited by self-interaction error made by these functionals. Recently, a novel one-electron self-interaction-correction (SIC) method that uses an iso-orbital indicator to apply the SIC at each point in space by scaling the exchange-correlation and Coulomb energy densities was proposed. The locally scaled SIC (LSIC) method is exact for the one-electron densities, and unlike the well-known Perdew–Zunger SIC (PZSIC) method recovers the uniform electron gas limit of the uncorrected density functional approximation, and reduces to PZSIC method as a special case when isoorbital indicator is set to the unity. Here, we present a numerical scheme that we have adopted to evaluate the Coulomb potential of the electron density scaled by the iso-orbital indicator required for the self-consistent LSIC calculations. After analyzing the behavior of the finite difference method (FDM) and the green function solution to the radial part of the Poisson equation, we adopt a hybrid approach that uses the FDM for the Coulomb potential due to the monopole and the GF for all higher-order terms. The performance of the resultant hybrid method is assessed using a variety of systems. The results show improved accuracy than earlier numerical schemes. We also find that, even with a generic set of radial grid parameters, accurate energy differences can be obtained using a numerical Coulomb solver in standard density functional studies.

Composite adaptive fuzzy bipartite consensus of fractional-order multiagent systems with a switched event-triggered mechanism

This paper focuses on online recorded-data-based composite adaptive fuzzy bipartite consensus control for uncertain fractional-order multiagent systems with interconnected terms and external disturbances by employing a switched-threshold-based event-triggered mechanism (ETM) under the backstepping structure. Fuzzy logic system is used as a universal function approximation to deal with function uncertainties that are not prone to model in the system. A new composite learning adaptive parameter design scheme that synthesizes both prediction error and tracking error is developed to enhance the tracking performance, where the prediction error is raised from the utilization of online recorded data and instantaneous data. A unique switched-threshold-based ETM is introduced, in which the information transmission between the sensor and the controller is imposed on one of the individuals. One merit of this work consists in that it can automatically and rapidly switch and adjust between the fixed threshold and relative threshold ETM according to the amplitude of input signals to balance the network resources and impede the occurrence of pulse phenomenon. In addition, it is theoretically proven that the proposed scheme can ensure that all internal signals of the closed-loop system are bounded and achieve local bipartite consistent errors through the fractional Lyapunov stability criterion. Finally, a numerical example is provided to confirm the feasibility of the proposed approach.

The role of the coupling matrix elements in time-dependent density functional theory on the simulation of core-level spectra of transition metal complexes

Time-dependent density functional theory (TD-DFT) stands out as an efficient tool for computing core-level spectra in large molecules, particularly transition metal complexes. However, despite their relatively moderate computational demands, TD-DFT methods can still pose challenges for typical computations involving transition metal complexes with over a thousand basis functions. In this study, we investigate the role of the Coulomb, Hartree–Fock exchange, and exchange-correlation kernel contributions to the TD-DFT coupling matrix elements when simulating core-level spectra in transition metal complexes. Our observations reveal that the exchange-correlation kernel contribution, responsible for more than 50% of the computational time in a hybrid TD-DFT calculation, surprisingly has no discernible impact on the qualitative aspects of the calculated spectra. While the Coulomb term plays a crucial role in describing L2,3 -edge spectra, its significance becomes negligible when considering K, L 1, and M4,5 edges. In contrast, the scaled Hartree–Fock exchange is demonstrated to be the most influential term, underscoring the necessity for hybrid density functional approximations in accurately simulating core-level spectra. These trends hold irrespective of the chosen basis set and exchange-correlation functional, providing valuable insights for the development of approximate methods for incorporating two-electron interactions within the realm of core-level spectroscopies.

A unifying perspective on non-stationary kernels for deeper Gaussian processes

The Gaussian process (GP) is a popular statistical technique for stochastic function approximation and uncertainty quantification from data. GPs have been adopted into the realm of machine learning (ML) in the last two decades because of their superior prediction abilities, especially in data-sparse scenarios, and their inherent ability to provide robust uncertainty estimates. Even so, their performance highly depends on intricate customizations of the core methodology, which often leads to dissatisfaction among practitioners when standard setups and off-the-shelf software tools are being deployed. Arguably, the most important building block of a GP is the kernel function, which assumes the role of a covariance operator. Stationary kernels of the Matérn class are used in the vast majority of applied studies; poor prediction performance and unrealistic uncertainty quantification are often the consequences. Non-stationary kernels show improved performance but are rarely used due to their more complicated functional form and the associated effort and expertise needed to define and tune them optimally. In this perspective, we want to help ML practitioners make sense of some of the most common forms of non-stationarity for Gaussian processes. We show a variety of kernels in action using representative datasets, carefully study their properties, and compare their performances. Based on our findings, we propose a new kernel that combines some of the identified advantages of existing kernels.

PREDICTION OF INDUCED STRESSES WITHIN SOIL MASS USING ARTIFICIAL NEURAL NETWORK

In this paper, an Artificial Neural Network (ANN) is applied to predict the soil stress within a soil mass for a variety of depths and displacements under applied loading. The neural network model is created and applied for two cases, point load, and Uniform rectangular load.Theses two cases were selected among many other cases of loading as a representation of the capabilities of ANN in finding proper solutions. The first case needs one input to get one output, where in the second case we need two inputs, to get one output only.Results revealed that function approximation using neural network can be applied easily and can give accurate results by choosing the appropriate learning algorithm, number of layers, and number of neurons to solve the problem. ANN model can provide reasonable accuracy for civil engineering problems, and a more effective tool for engineering applications.

Generation of normal distributions revisited

AbstractNormally distributed random numbers are commonly used in scientific computing in various fields. It is important to generate a set of random numbers as close to a normal distribution as possible for reducing initial fluctuations. Two types of samples from a uniform distribution are examined as source samples for inverse transform sampling methods. Three types of inverse transform sampling methods with new approximations of inverse cumulative distribution functions are also discussed for converting uniformly distributed source samples to normally distributed samples.

On Universality of Some Beurling Zeta-Functions

Let P be the set of generalized prime numbers, and ζP(s), s=σ+it, denote the Beurling zeta-function associated with P. In the paper, we consider the approximation of analytic functions by using shifts ζP(s+iτ), τ∈R. We assume the classical axioms for the number of generalized integers and the mean of the generalized von Mangoldt function, the linear independence of the set {logp:p∈P}, and the existence of a bounded mean square for ζP(s). Under the above hypotheses, we obtain the universality of the function ζP(s). This means that the set of shifts ζP(s+iτ) approximating a given analytic function defined on a certain strip σ^<σ<1 has a positive lower density. This result opens a new chapter in the theory of Beurling zeta functions. Moreover, it supports the Linnik–Ibragimov conjecture on the universality of Dirichlet series. For the proof, a probabilistic approach is applied.