Recursive Approximations Research Articles

Spiking neural networks for nonlinear regression of complex transient signals on sustainable neuromorphic processors

In recent years, spiking neural networks were introduced in science as the third generation of artificial neural networks leading to a tremendous energy saving on neuromorphic processors. This sustainable effect is due to the sparse nature of signal processing in-between spiking neurons leading to much less scalar multiplications as in second-generation networks. The spiking neuron’s efficiency is even more pronounced by their inherently recurrent nature being useful for recursive function approximations. We believe that there is a need for a general regression framework for SNNs to explore the high potential of neuromorphic computations. However, besides many classification studies with SNNs in the literature, nonlinear neuromorphic regression analysis represents a gap in research. Hence, we propose a general SNN approach for function approximation applicable for complex transient signal processing taking surrogate gradients due to the discontinuous spike representation into account. However, to pay attention to the need for high memory access during deep SNN network communications, additional spiking Legrendre Memory Units are introduced in the neuromorphic architecture. Path-dependencies and evolutions of signals can be tackled in this way. Furthermore, interfaces between real physical and binary spiking values are necessary. Following this intention, a hybrid approach is introduced, exhibiting an autoencoding strategy between dense and spiking layers. However, to verify the presented framework of nonlinear regression for a wide spectrum of scientific purposes, we see the need for obtaining realistic complex transient short-time signals by an extensive experimental set-up. Hence, a measurement technique for benchmark experiments is proposed with high-frequency oscillations measured by capacitive and piezoelectric sensors resulting in wave propagations and inelastic solid deformations to be predicted by the developed SNN regression analysis. Hence, the proposed nonlinear regression framework can be deployed to a wide range of scientific and technical applications.

Effectiveness of the recursive-lattice technique in the investigation of magnetic systems with the pyrochlore structure.

A well-defined higher recursive approximation of the pyrochlore lattice is introduced and its relevance and effectiveness for the systematic investigation of magnetic systems with the pyrochlore structure is studied within the classical antiferromagnetic as well as ferromagnetic spin-1/2 Ising model in the presence of the external magnetic field. The exact solution of the model is found with the explicit analytic expression for the free energy per site of the lattice. The magnetization and entropy properties of all ground states of the antiferromagnetic model are determined and compared to those obtained within the lower recursive approximation of the model on the recursive tetrahedral lattice. The exact analysis of the residual entropies of the model on the introduced recursive lattice explicitly shows that the well-known Pauling entropy of the water ice cannot represent the true residual entropy of the antiferromagnetic model on the regular pyrochlore lattice in the zero external magnetic field. The improvement of the value of the critical temperature of the ferromagnetic model obtained within the introduced higher recursive approximation is also discussed. The analysis performed demonstrates the great efficiency of recursive approximations in the investigation of various magnetic systems with the pyrochlore structure, especially for frustrated antiferromagnetic systems.

High-order, large time-stepping integrators for scalar hyperbolic conservation laws

In this paper, we construct a class of temporal up-to-fourth-order integrators that preserve strong stability of conservation laws for relatively large time steps. The new methods approximate the exponential functions obtained from stabilization integrating factor Runge–Kutta schemes using recursive approximations without destroying the convergence. The time delay introduced by the stabilization parameter is then quantified and eliminated using a relaxation technique. The proposed schemes are further adopted with a monotonicity-preserving conservative scheme for the spatial discretization to solve scalar hyperbolic conservation laws. Several 1D and 2D benchmark examples are given to validate the superiority and effectiveness of the proposed schemes.

Energy simulations for some optic systems: the Heisenberg ferromagnetic and the recursive vortex filament approximations

Energy simulations for some optic systems: the Heisenberg ferromagnetic and the recursive vortex filament approximations

Recursive convex approximations for optimal power flow solution in direct current networks

<p>The optimal power flow problem in direct current (DC) networks considering dispersal generation is addressed in this paper from the recursive programming point of view. The nonlinear programming model is transformed into two quadratic programming approximations that are convex since the power balance constraint is approximated between affine equivalents. These models are recursively (iteratively) solved from the initial point v<sup>t</sup> equal to 1.0 pu with t equal to 0, until that the error between both consecutive voltage iterations reaches the desired convergence criteria. The main advantage of the proposed quadratic programming models is that the global optimum finding is ensured due to the convexity of the solution space around v<sup>t</sup>. Numerical results in the DC version of the IEEE 69-bus system demonstrate the effectiveness and robustness of both proposals when compared with classical metaheuristic approaches such as particle swarm and antlion optimizers, among others. All the numerical validations are carried out in the MATLAB programming environment version 2021b with the software for disciplined convex programming known as CVX tool in conjuction with the Gurobi solver version 9.0; while the metaheuristic optimizers are directly implemented in the MATLAB scripts.</p>

Multilevel Picard approximations for McKean-Vlasov stochastic differential equations

In the literature there exist approximation methods for McKean-Vlasov stochastic differential equations which have a computational effort of order 3. In this article we introduce full-history recursive multilevel Picard approximations for McKean-Vlasov stochastic differential equations. We prove that these MLP approximations have computational effort of order 2+ which is essentially optimal in high dimensions.

Circular orbit transfer employing recursive nonlinear state transformations and nonlinear feedback

A recursive application of approximate feedback linearization is applied to low-Earth circular orbit rendezvous utilizing continuous low thrust propulsion. A family of solutions is established up to the $$\rho $$ -th degree utilizing the null space that appears at each step in the computations. A convergence analysis is presented to guarantee that the recursive process converges as $$\rho \rightarrow \infty $$ . During the recursive computations, patterns were detected as part of the solution process that illuminated a family of exact nonlinear control solutions obtained by judiciously selecting the null space coefficients. Examples of various analytic solutions are presented. The relationship between the $$\rho $$ -th degree exact solution obtained through the recursive approximations and a known exact solution is illustrated where it is shown that different exact solutions can have different performance in terms of fuel usage leading to the possibility of optimization considerations in selecting the desired exact solution.

An approach to exact nonlinear feedback control design employing recursive approximations and the null space

A strategy to design exact nonlinear feedback controllers based on a recursive application of approximate linearization methods is examined. The computations are algebraic and computationally simpler than solving the set of coupled nonlinear partial differential equations thereby facilitating practical symbolic computer computations enabling discernment of evolving patterns in the approximate solutions as the order of approximation increases. Utilizing the null space that appears at each step in the computations as part of the computations, a family of analytic solutions can be generated asymptotically. There are possibilities for optimizing the performance by judiciously choice of analytic solution that emerge from the selective use of the null space.

Overcoming the curse of dimensionality in the numerical approximation of Allen–Cahn partial differential equations via truncated full-history recursive multilevel Picard approximations

Abstract One of the most challenging problems in applied mathematics is the approximate solution of nonlinear partial differential equations (PDEs) in high dimensions. Standard deterministic approximation methods like finite differences or finite elements suffer from the curse of dimensionality in the sense that the computational effort grows exponentially in the dimension. In this work we overcome this difficulty in the case of reaction–diffusion type PDEs with a locally Lipschitz continuous coervice nonlinearity (such as Allen–Cahn PDEs) by introducing and analyzing truncated variants of the recently introduced full-history recursive multilevel Picard approximation schemes.

A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations

Deep neural networks and other deep learning methods have very successfully been applied to the numerical approximation of high-dimensional nonlinear parabolic partial differential equations (PDEs), which are widely used in finance, engineering, and natural sciences. In particular, simulations indicate that algorithms based on deep learning overcome the curse of dimensionality in the numerical approximation of solutions of semilinear PDEs. For certain linear PDEs it has also been proved mathematically that deep neural networks overcome the curse of dimensionality in the numerical approximation of solutions of such linear PDEs. The key contribution of this article is to rigorously prove this for the first time for a class of nonlinear PDEs. More precisely, we prove in the case of semilinear heat equations with gradient-independent nonlinearities that the numbers of parameters of the employed deep neural networks grow at most polynomially in both the PDE dimension and the reciprocal of the prescribed approximation accuracy. Our proof relies on recently introduced full history recursive multilevel Picard approximations for semilinear PDEs.

Relevance of recursive lattice approximations for description of frustrated magnetic systems: Star kagome-like recursive lattice approximation

We introduce and investigate in detail thermodynamic properties of the antiferromagnetic spin-1∕2 Ising model with the multisite interaction in the presence of the external magnetic field on the star kagome-like recursive lattice that takes into account not only basic triangular structure of the regular kagome lattice responsible for the geometric frustration but also a typical star-like structure of the kagome lattice consisting of six site-connected triangles. It is shown that the model exhibits single solution for arbitrary values of the parameters of the model, which is driven by a sixth-order polynomial equation. The free energy per site of the model is found and the magnetization, the entropy, and the specific heat capacity properties of the model are investigated and compared to those obtained in the framework of the model on the kagome-like Husimi lattice. The system of all ground states of the model is determined and exact expressions for their magnetization properties as well as for their residual entropies are derived. It is shown that all results obtained in the framework of the model on the star kagome-like recursive lattice are unprecedentedly close to those obtained on the much simpler kagome-like Husimi lattice. Besides, in the case when an exact result of the model on the real kagome lattice exists it is very close to the corresponding results obtained on the recursive lattices. These quite surprising but nontrivial facts allow one to make important assumptions about the properties of the model on the real two-dimensional kagome lattice even in the nonzero external magnetic field. First of all, one can suppose that various thermodynamic properties of the model on the kagome lattice must be qualitatively the same as the corresponding properties obtained in the framework of the model on various kagome-like recursive lattices. Moreover, one can also suppose that even the quantitative results obtained for the model on various kagome-like recursive lattices must be very close (it seems that some of them are even identical) to those that should be valid on the regular kagome lattice, although the exact solution of the model on the kagome lattice does not exist yet.

Multilevel Monte Carlo for Smoothing via Transport Methods

In this article we consider recursive approximations of the smoothing distribution associated to partially observed stochastic differential equations (SDEs), which are observed discretely in time. ...

Approximate linear minimum variance filters for continuous-discrete state space models: convergence and practical adaptive algorithms

Abstract In this article, approximate linear minimum variance (LMV) filters for continuous-discrete state space models are introduced. The filters are derived from a wide class of recursive approximations to the predictions for the first two conditional moments of the state equation between each pair of consecutive observations. The convergence of the approximate filters to the exact LMV filter is proved when the error between the predictions and their approximations decreases no matter the time distance between observations. As particular instance, the order-$\beta$ local linearization filters are presented and expounded in detail. Practical adaptive algorithms are also provided and their performance in simulation is illustrated with various examples. The proposed filters are intended for the recurrent practical situation where a stochastic dynamical system should be identified from a reduced number of partial and noisy observations distant in time.

Inversion of the Fundamental Thermodynamic Equations for Isentropic Nozzle Flow Analysis

The isentropic flow equations relating the thermodynamic pressures, temperatures, and densities to their stagnation properties are solved in terms of the area expansion and specific heat ratios. These fundamental thermofluid relations are inverted asymptotically and presented to arbitrary order. Both subsonic and supersonic branches of the possible solutions are systematically identified and exacted. Furthermore, for each branch of solutions, two types of recursive approximations are provided: a property-specific formulation and a more general, universal representation that encompasses all three properties under consideration. In the case of the subsonic branch, the asymptotic series expansion is shown to be recoverable from Bürmann’s theorem of classical analysis. Bosley’s technique is then applied to verify the theoretical truncation order in each approximation. The final expressions enable us to estimate the pressure, temperature, and density for arbitrary area expansion and specific heat ratios with no intermediate Mach number calculation or iteration. The analytical framework is described in sufficient detail to facilitate its portability to other nonlinear and highly transcendental relations where closed-form solutions may be desirable.

Direct calculation of the average local Mach number in converging–diverging nozzles

Foremost amongst rocket nozzle relations is the area–Mach number expression linking the local velocity normalized by the speed of sound to the area ratio At/A, and the ratio of specific heats. Known as Stodolaʼs equation, the attendant expression is transcendental and requires iteration or numerical root finding in extracting the solution under subsonic or supersonic nozzle operation. In this work, a novel analytical inversion of the problem is pursued to the extent of providing the local Mach number directly at any given cross-section. The inversion process is carried out using two unique approaches. In the first, Bürmannʼs theorem is employed to undertake a functional reversion from which the subsonic solution may be retrieved. In the second, the Successive Approximation Approach is repeatedly applied to arrive at a closed-form representation of the supersonic root. Both methods give rise to unique recursive approximations that permit the selective extraction of the desired solution to an arbitrary level of accuracy. Results are verified numerically and the precision associated with the supersonic solution is shown to improve with successive increases in the ratio of specific heats.

Role of model selection criteria in geostatistical inverse estimation of statistical data‐ and model‐parameters

We analyze theoretically the ability of model quality (sometimes termed information or discrimination) criteria such as the negative log likelihood NLL, Bayesian criteria BIC and KIC and information theoretic criteria AIC, AICc, and HIC to estimate (1) the parameter vector of the variogram of hydraulic log conductivity (Y = ln K), and (2) statistical parameters and proportional to head and log conductivity measurement error variances, respectively, in the context of geostatistical groundwater flow inversion. Our analysis extends the work of Hernandez et al. (2003, 2006) and Riva et al. (2009), who developed nonlinear stochastic inverse algorithms that allow conditioning estimates of steady state and transient hydraulic heads, fluxes and their associated uncertainty on information about conductivity and head data collected in a randomly heterogeneous confined aquifer. Their algorithms are based on recursive numerical approximations of exact nonlocal conditional equations describing the mean and (co)variance of groundwater flow. Log conductivity is parameterized geostatistically based on measured values at discrete locations and unknown values at discrete “pilot points.” Optionally, the maximum likelihood function on which the inverse estimation of Y at pilot points is based may include a regularization term reflecting prior information about Y. The relative weight assigned to this term and its components and , as well as are evaluated separately from other model parameters to avoid bias and instability. This evaluation is done on the basis of criteria such as NLL, KIC, BIC, HIC, AIC, and AICc. We demonstrate theoretically that, whereas all these six criteria make it possible to estimate , KIC alone allows one to estimate validly and (and thus ). We illustrate this discriminatory power of KIC numerically by using a differential evolution genetic search algorithm to minimize it in the context of a two‐dimensional steady state groundwater flow problem. We find that whereas , and the integral scale of Y can be estimated on the basis of a zero‐order mean flow equation, the sill of the Y‐variogram is estimated more accurately by a second‐order approximation of flow. This notwithstanding, KIC prefers the simpler zero‐order moment over the more complex second‐order version.

Stochastic characterization of the Montalto Uffugo research site (Italy) by geostatistical inversion of moment equations of groundwater flow

Summary We assess the applicability and performance of a methodology of inverting stochastic mean groundwater flow equations to characterize the spatial variability of (natural) log-transmissivity (Y) of a heterogeneous aquifer. The methodology, originally proposed by Hernandez et al., 2003 , Hernandez et al., 2006 , relies on a nonlinear geostatistical inverse algorithm for recursive approximations of steady-state (ensemble) mean groundwater flow that allows estimating jointly the spatial variability of Y, the underlying variogram parameters, and the variance–covariance of the estimates. Estimates of prediction errors of hydraulic heads and fluxes are then calculated a posteriori, upon solving equations satisfied by the corresponding covariances. Here, we extend the methodology to quasi-steady state flow conditions and present its first field application by using information collected during a pumping test performed at the Montalto Uffugo research site (Italy). Log-transmissivity is parameterized geostatistically on the basis of an available measured value and a set of unknown values at discrete pilot points. Best estimates of Y at the measurement location and at the pilot points are obtained by a maximum likelihood fit of computed and measured heads. These posterior estimates are then projected onto the computational grid by kriging. Information on head drawdowns is provided through self-potential signals recorded by 47 surface electrodes during the test. The maximum likelihood-based objective function includes a regularization term reflecting prior information about Y. The relative weight assigned to this term is evaluated separately from other model parameters to avoid bias and instability. We explore the effectiveness of both a zero- and a second-order closure of the mean flow equation at providing a proper geostatistical characterization of the log-transmissivity field. The parameters of the variogram of Y are estimated a posteriori using formal model selection criteria. The adoption of a second-order mean flow model renders the sharpest definition of the regularization term and of the Y variogram parameters.

Inverse analysis of stochastic moment equations for transient flow in randomly heterogeneous media

We present a nonlinear stochastic inverse algorithm that allows conditioning estimates of transient hydraulic heads, fluxes and their associated uncertainty on information about hydraulic conductivity ( K) and hydraulic head ( h) data collected in a randomly heterogeneous confined aquifer. Our algorithm is based on Laplace-transformed recursive finite-element approximations of exact nonlocal first and second conditional stochastic moment equations of transient flow. It makes it possible to estimate jointly spatial variations in natural log-conductivity ( Y = ln K ) , the parameters of its underlying variogram, and the variance–covariance of these estimates. Log-conductivity is parameterized geostatistically based on measured values at discrete locations and unknown values at discrete “pilot points”. Whereas prior values of Y at pilot point are obtained by generalized kriging, posterior estimates at pilot points are obtained through a maximum likelihood fit of computed and measured transient heads. These posterior estimates are then projected onto the computational grid by kriging. Optionally, the maximum likelihood function may include a regularization term reflecting prior information about Y. The relative weight assigned to this term is evaluated separately from other model parameters to avoid bias and instability. We illustrate and explore our algorithm by means of a synthetic example involving a pumping well. We find that whereas Y and h can be reproduced quite well with parameters estimated on the basis of zero-order mean flow equations, all model quality criteria identify the second-order results as being superior to zero-order results. Identifying the weight of the regularization term and variogram parameters can be done with much lesser ambiguity based on second- than on zero-order results. A second-order model is required to compute predictive error variances of hydraulic head (and flux) a posteriori. Conditioning the inversion jointly on conductivity and hydraulic head data results in lesser predictive uncertainty than conditioning on conductivity or head data alone.

Effects of evolving scales of heterogeneity on hydraulic head predictions under convergent flow conditions

Two-dimensional steady-state flow towards a well that fully penetrates a randomly heterogeneous aquifer with a deterministically prescribed constant head boundary is considered. Flow occurs over an infinite hierarchy of mutually uncorrelated, statistically homogeneous and isotropic random fields (modes) of natural log-transmissivity, Y, each of which is associated with a Gaussian variogram. A lower and upper cut-off of the hierarchy, respectively related to the length scale of the domain and data support (sample volume), is introduced. This allows the direct incorporation of the scale dependence of the integral scale of Y into groundwater flow (ensemble) moments and leads to a geostatistical description of the system in terms of a (stationary) truncated power variogram (TPV). An analytical solution for the hydraulic head mean and variance is developed based on recursive approximations of exact nonlocal moment equations. The solution allows assessment of the functional dependences of the distribution of the leading (statistical) moments of hydraulic head on parameters of the variogram associated with the hierarchy of log-transmissivity modes.

Inverse stochastic moment analysis of steady state flow in randomly heterogeneous media

Nonlocal stochastic moment equations have been used successfully to analyze steady state and transient flow in randomly heterogeneous media conditional on measured values of medium properties. We present a nonlinear geostatistical inverse algorithm for steady state flow that makes it possible to further condition such analyses on measured values of state variables. Our approach accounts for all scales of spatial variability resolvable by the computational grid. It is based on recursive finite element approximations of exact nonlocal first and second conditional moment equations. Hydraulic conductivity is parameterized geostatistically on the basis of measured values at discrete locations and unknown values at discrete “pilot points.” Prior estimates of pilot point values are obtained (optionally) by generalized kriging. Posterior estimates at pilot points and (optionally) at measurement points are obtained by calibrating mean flow against measured values of head. The parameters are projected onto a computational grid via kriging. Maximum likelihood calibration allows estimating not only hydraulic but also (optionally) unknown variogram parameters with or without prior information about the former. The approach yields covariance matrices for parameter estimation as well as head and flux prediction errors, the latter being obtained a posteriori. We implement our inverse approach on highly and mildly heterogeneous media under superimposed mean uniform and convergent flows in a bounded two‐dimensional domain. Our examples illustrate how conductivity and head data act separately and jointly to reduce parameter estimation errors and to model predictive uncertainty. We also evaluate the functional form of the log conductivity variogram and its parameters using likelihood‐based model discrimination criteria.