Nonlinear Approximation Research Articles

A De-Nesting Hybrid Reliability Analysis Method and Its Application in Marine Structure

In recent years, marine structures have been widely used in the world, making significant contributions to the utilization of marine resources. In the design of marine structures, there is a hybrid reliability problem arising from aleatory uncertainty and epistemic uncertainty. In many cases, epistemic uncertainty is estimated by interval parameters. Traditional methods for hybrid reliability analysis usually require a nested optimization framework, which will lead to too many calls to the limit state function (LSF) and result in poor computational efficiency. In response to this problem, this paper proposes a de-nesting hybrid reliability analysis method creatively. Firstly, it uses the p-box model to describe the epistemic uncertainty variables, and then the linear approximation (LA) model and the two-point adaptive nonlinear approximation (TANA) model are combined to approximate the upper and lower bounds of LSF with epistemic uncertainty. Based on the first-order reliability method (FORM), an iterative operation is used to obtain the interval of the non-probability hybrid reliability index. The traditional nested optimization structure is effectively eliminated by the above approximation method, which efficiently reduces the times of LSF calls and increases the calculation speed while preserving sufficient accuracy. Finally, one numerical example and two engineering examples are provided to show the greater effectiveness of this method than the traditional nested optimization method.

Enhanced support vector machine-based moving regression strategy for response prediction and reliability estimation of complex structure

For predicting response property and estimating reliability level of complex structure, enhanced support vector machine-based moving regression (MR-ESVM) strategy is proposed based on support vector machine (SVM), heuristic algorithm and moving least square (MLS) technique. Under this strategy, we develop four different SVM models including SVM-based moving regression (MR-SVM), SVM-based improved moving regression (IMR-SVM), improved SVM-based moving regression (MR-ISVM) and bi-optimized SVM-based moving regression (BiOMR-SVM) methods. In these developed MR-ESVM approaches, the MR-SVM method is explored by introducing the MLS technique into the SVM model; the IMR-SVM method is discussed by fusing the MR-SVM method and artificial rabbits optimization (ARO), and the ARO is used to search the optimal radius of compact region; the MR-ISVM method is raised by integrating the ARO into the MR-SVM, and the ARO is applied to find the optimal values in the SVM model; The BiOMR-SVM method is emerged by merging the IMR-SVM and MR-ISVM methods. To verify the effectiveness of these developed MR-ESVM strategies, a multivariate nonlinear function approximation is implemented to illustrate the advantages from the mathematics perspective, an aeroengine turbine blisk radial deformation reliability analysis and an aircraft hydraulic system low pressure reliability analysis are derived to demonstrate the applicability in engineering practice. The analytical results show that these four MR-ESVM approaches hold excellent merits in modeling features and simulation characteristics. The efforts of this work provide a novel idea for the response prediction of complex structure, and enrich the reliability estimation principle of surrogate models of complex structure.

A Comprehensive Review of Artificial Intelligence and Machine Learning in Control Theory

Abstract: Traditional control methods, such as proportional-integral-derivative (PID) controllers and linear-quadratic regulators (LQRs), have proven effective for linear and well-modeled systems. However, these methods often perform poorly in nonlinear, complex and dynamic environments. The paper aims to investigate the modern control systems by integrating artificial intelligence (AI) techniques, such as machine learning (ML), reinforcement learning (RL), deep learning, and fuzzy logic, to enhance their adaptive, robust, and predictive capabilities. And it reviews the literature and analyzes AI integration in control systems. The proposed strategies include supervised learning for trajectory optimization and fault detection, reinforcement learning for optimal control in dynamic environments, neural networks for complex nonlinear function approximation, and fuzzy logic for handling uncertainty and imprecise inputs. AI techniques significantly enhance the ability to tackle nonlinear problems and dynamic changes, demonstrating superior performance in applications like self-driving cars adapting to various road conditions and optimal energy distribution in smart grids. Despite the challenges of computational complexity, scalability, and the safety and reliability in the implementation of interpretable AI models, this paper suggests that hybrid approaches combining traditional control and AI techniques, along with the evolution of interpretable AI and convergence with quantum control, hold great promise for advancing AI-driven control systems.

A note on wavelet packet summation methods

The wavelet summation method is a method to obtain a non-linear approximation with ‘nice properties’ for a given signal. This technique has been shown to be effective in non-parametric regression and other fields, including signal denoising. By utilizing a shrinkage rule on the empirical wavelet packet coefficients, wavelet packet summation methods can be obtained. In this paper, we show that the wavelet packet expansion of any finite energy function converges pointwise almost everywhere under the wavelet packet projection, hard sampling, and soft sampling methods.

A Compact Constraint Incremental Method for Random Weight Networks and Its Application.

Incremental random weight networks (IRWNs) face the issues of weak generalization and complicated network structure. There is an important reason: the learning parameters of IRWNs are determined in a random fashion without guidance, which may increase many redundant hidden nodes, and thereby leading to inferior performance. To resolve this issue, a novel IRWN with compact constraint that guides the assignment of random learning parameters (CCIRWN) is developed in this brief. Using the iteration method of Greville, a compact constraint that simultaneously assures the quality of generated hidden nodes and the convergence of the CCIRWN is built to perform learning parameter configuration. Meanwhile, the output weights of the CCIRWN are assessed analytically. Two types of learning methods for constructing the CCIRWN are proposed. Finally, the performance evaluation of the proposed CCIRWN is undertaken on the 1-D nonlinear function approximation, several real-world datasets, and data-driven estimation based on the industrial data. Numerical and industrial examples indicate that the proposed CCIRWN with compact structure can achieve favorable generalization ability.

Wave Propagation in Prestressed Structures with Geometric Nonlinearities Through Carrera Unified Formulation

This paper deals with the analysis of wave propagation characteristics in various prestressed structures with geometric nonlinearities using the Carrera Unified Formulation (CUF). CUF provides a versatile platform to model a wide range of structures and nonlinearities that can take care of all wave propagation aspects. In this work, different geometric nonlinearities for which representative governing equations have been derived and numerical solutions have been obtained through a unified approach are considered. The study investigates in detail the effect of prestress and geometric nonlinearity on wave propagation behavior. The results indicate that prestress has a very influential effect on modal frequency and dispersion characteristics for wave propagation. Specifically, three CUF-modeled beams are considered herein, having a sandwich, metallic portal, and metallic box cross section, respectively. Initially, the principal cross-sectional modal shapes of the unstressed, linear, and full nonlinear (i.e., full three-dimensional Green–Lagrange strain matrix) beam with a prestress are investigated, among which torsional and flexural modes can be recognized. Afterward, the equilibrium curves of such structures for various geometrical nonlinear approximations are traced, highlighting that most types of nonlinearity induce a hardening behavior in the system, which increases with the preload, directly leading to a variation in modal frequencies. The dispersion relations of the full nonlinear structure examined as a function of the applied preload are further compared, enriching the investigation by exploiting wave finite element method capabilities. This knowledge paves the way toward the design and optimization of prestressed systems with enhanced acoustic performance, and that fosters the development of sound absorption, noise insulation, and structural isolation.

Unified LSPG Model Reduction Framework and Assessment for Hypersonic Computational Fluid Dynamics

A unified least-squares Petrov–Galerkin (LSPG) framework for projection-based model order reduction featuring three different approximation manifolds [affine manifold, quadratic manifold, and nonlinear manifold built using a deep artificial neural network (ANN)] is presented. Its performance was assessed for a variable-speed version of the double-cone hypersonic benchmark problem. First, a high-dimensional viscous computational fluid dynamics model (HDM) was constructed, verified, and validated. The dimensionality of the HDM was then reduced using LSPG, each of the aforementioned approximation manifolds, and a global right reduced-order basis trained in the range 8≤M∞≤13. Each resulting global projection-based reduced-order model (PROM) was hyper-reduced and transformed into a hyper-reduced PROM (HPROM). The accuracy of each constructed HPROM was assessed for various quantities of interest and contrasted with that of snapshot interpolation. For this purpose, three different error measures were considered and discussed in the context of shock-dominated problems. Wall-clock and CPU time speedup factors are reported. Overall, it was shown that using a relatively small set of training data, all constructed LSPG HPROMs were nonlinearly stable, real-time capable, and highly predictive. The LSPG HPROM constructed using a nonlinear approximation manifold and an ANN was the most computationally efficient.

Empirical Analysis of Stability of An+1BnO3n+1 Ruddlesden–Popper Phases Using Reciprocal n-Values

Layered An+1BnO3n+1 (n = 1…∞) Ruddlesden–Popper (RP) phases are a promising system for a variety of applications. Within the RP family, the thermodynamic properties of the phases are essentially additive with variation in the n value, but at present, there are no general approaches that would allow one to evaluate the individual stability of the RP phases and the possibility of their interconversion. The aim of this paper is to present a novel concept for performing a thermodynamic analysis of RP phases using the reciprocal values of the index n. We present an empirical equation ΔG1/n = ΔGP + B1/n + B2/n2, where ΔG1/n and ΔGP are the molar Gibbs energies of formation of the Ruddlesden–Popper (RP) phase (AO)1/nABO3 and the parent ABO3 perovskite, respectively, and n is a stoichiometry index of An+1BnO3n+1 RP phase. The correlation was validated using available thermodynamic data for the systems Sr-Ti-O, Ca-Ti-O, Sr-Zr-O, La-Ni-O, and La-Co-O. For all A-B combinations, the equation quantitatively describes the Gibbs energy of RP phase formation. Predicted values for the non-linear approximation lie within the experimental uncertainty in determining ΔG1/n. The proposed correlation was used to analyze the relative stability of the RP phases and to determine the feasibility of synthesizing new compounds.

Online adaptive critic designs with tensor product B-splines and incremental model techniques

As an effective optimal control scheme in the field of reinforcement learning, adaptive dynamic programming (ADP) has attracted extensive attention in recent decades. Neural networks (NNs) are commonly employed in ADP algorithms to realize nonlinear function approximation. However, the learning process with NN approximation typically requires a substantial amount of training data and places a heavy computational burden. To improve learning efficiency and alleviate computational load, this paper develops a novel model-free ADP approach based on multivariate splines and incremental control techniques, aimed at achieving online learning of nonlinear control. By utilizing input and output data, an incremental linear approximation model is identified online without any prior knowledge of system dynamics. To improve nonlinear approximation capabilities and lessen computational demands, tensor product B-splines, instead of NNs, are integrated into the critic module to approximate the value function, and the recursive least squares temporal difference (RLS-TD) algorithm is employed to update the weights of spline bases. Convergence analysis of the proposed control scheme is conducted based on the dual-timescale stochastic approximation theory. To illustrate the effectiveness and feasibility of the proposed control scheme, numerical simulations are performed in solving the online nonlinear control problem within the context of the inverted pendulum system.

Reservoir structure optimization of echo state networks: A detrended multiple cross-correlation pruning perspective

Reservoir structure optimization of echo state networks (ESN) is an important enabler for improving network performance. In this regard, pruning provides an effective means to optimize reservoir structure by removing redundant components in the network. Existing studies achieve reservoir pruning by removing insignificant neuronal connections. However, such processing causes the optimized neurons to still remain in the reservoir and thus hinder network inference by participating in computations, leading to suboptimal utilization of pruning benefits by the network. To solve this problem, this paper proposes an adaptive pruning algorithm for ESN within the detrended multiple cross-correlation (DMC2) framework, i.e., DMAP. On the whole, it contains two main functional parts: DMC2 measure of reservoir neurons and reservoir pruning. Specifically, the former is used to quantify the correlation among neurons. Based on this, the latter can remove neurons with high correlation from the reservoir completely, and finally obtain the optimal network structure by retraining the output weights. Experiment results show that DMAP-ESN outperforms its competitors in nonlinear approximation capability and reservoir stability.

Use of Deep-Learning-Accelerated Gradient Approximation for Reservoir Geological Parameter Estimation

The estimation of space-varying geological parameters is often not computationally affordable for high-dimensional subsurface reservoir modeling systems. The adjoint method is generally regarded as an efficient approach for obtaining analytical gradient and, thus, proceeding with the gradient-based iteration algorithm; however, the infeasible memory requirement and computational demands strictly prohibit its generic implementation, especially for high-dimensional problems. The autoregressive neural network (aNN) model, as a nonlinear surrogate approximation, has gradually received increasing popularity due to significant reduction of computational cost, but one prominent limitation is that the generic application of aNN to large-scale reservoir models inevitably poses challenges in the training procedure, which remains unresolved. To address this issue, model-order reduction could be a promising strategy, which enables us to train the neural network in a very efficient manner. A very popular projection-based linear reduction method, i.e., propel orthogonal decomposition (POD), is adopted to achieve dimensionality reduction. This paper presents an architecture of a projection-based autoregressive neural network that efficiently derives an easy-to-use adjoint model by the use of an auto-differentiation module inside the popular deep learning frameworks. This hybrid neural network proxy, referred to as POD-aNN, is capable of speeding up derivation of reduced-order adjoint models. The performance of POD-aNN is validated through a synthetic 2D subsurface transport model. The use of POD-aNN significantly reduces the computation cost while the accuracy remains. In addition, our proposed POD-aNN can easily obtain multiple posterior realizations for uncertainty evaluation. The developed POD-aNN emulator is a data-driven approach for reduced-order modeling of nonlinear dynamic systems and, thus, should be a very efficient modeling tool to address many engineering applications related to intensive simulation-based optimization.

Repercussion of quadratic density variation on nonlinear mixed convective Darcy–Forchhimer Al2O3, Cu\\EG flow over an inclined plate with exponential heat source and Arrhenius equation

Generating and controlling energy at high temperatures is extremely challenging in solar energy applications with hybrid nanofluids (HNFs). This challenge is tackled with nonlinear changes in density with temperature and concentration. Hence, a nonlinear Boussinesq approximation is taken on Darcy – Forchhimer hybrid flow containing aluminium oxide ( A l 2 O 3 ) and copper ( Cu ) nanoparticles with base fluid Ethylene Glycol (EG) over an inclined plate at an angle of ξ = π 6 . The physical model is supported by a space-based exponential heat source (EHS) along with a temperature-based heat source (THS) in the energy equation and Arrhenius equation in diffusion relation with suction and blowing on the wall. The regulating equations are transformed into standard differential equations using the similarity conversions, and these equations are subsequently solved in MAPLE 2022. The thermal field is affected positively by EHS and THS parameters, which fulfil the purpose of including these parameters in the flow model. The rate of heat transport increment for Q E parameter by 1.04% for HNF whereas 1.06% for NF when Su > 0 . Also, Su < 0 heat transfer rate increases by 3.68% for HNF whereas 3.75% for NF. As A l 2 O 3 , Cu ∖EG HNF move from suction to injection, the Nusselt number reduces by more than 50%.

An adaptive learning strategy for surrogate modeling of high-dimensional functions - Application to unsteady hypersonic flows in chemical nonequilibrium

Many engineering applications rely on the evaluation of expensive, non-linear high-dimensional functions. In this paper, we propose the RONAALP algorithm (Reduced Order Nonlinear Approximation with Adaptive Learning Procedure) to incrementally learn, on-the-fly as the application progresses, a fast and accurate reduced-order surrogate model of a target function. First, a combination of nonlinear auto-encoder, community clustering, and radial basis function networks allows us to learn an efficient and compact surrogate model with limited training data. Secondly, an active learning procedure overcomes any extrapolation issues during the online stage by adapting the surrogate model with high-fidelity evaluations that fall outside its current validity range. This approach results in generalizable, fast, and accurate reduced-order models of high-dimensional functions. The method is demonstrated on three direct numerical simulations of hypersonic flows in chemical nonequilibrium. Accurate simulations of these flows rely on detailed thermochemical gas models that dramatically increase the cost of such calculations. Using RONAALP to learn a reduced-order thermodynamic model surrogate on-the-fly, the cost of such simulations was reduced by up to 75% while maintaining an error of less than 10% on relevant quantities of interest.

Nonlinear Hierarchical Matrix Factorization-Based Tensor Ring Approximation for Multi-dimensional Image Recovery

Nonlinear Hierarchical Matrix Factorization-Based Tensor Ring Approximation for Multi-dimensional Image Recovery

Piecewise nonlinear approximation for non-smooth functions

Piecewise affine or linear approximation has garnered significant attention as a technique for approximating piecewise-smooth functions. In this study, we propose a novel approach: piecewise non-linear approximation based on rational approximation, aimed at approximating non-smooth functions. We introduce a method termed piecewise Padé Chebyshev (PiPC) tailored for approximating univariate piecewise smooth functions. Our investigation focuses on assessing the effectiveness of PiPC in mitigating the Gibbs phenomenon during the approximation of piecewise smooth functions. Additionally, we provide error estimates and convergence results of PiPC for non-smooth functions. Notably, our technique excels in capturing singularities, if present, within the function with minimal Gibbs oscillations, without necessitating the explicit specification of singularity locations. To the best of our knowledge, prior research has not explored the use of piecewise non-linear approximation for approximating non-smooth functions. Finally, we validate the efficacy of our methods through numerical experiments, employing PiPC to reconstruct a non-trivial non-smooth function, thus demonstrating its capability to significantly alleviate the Gibbs phenomenon.

The cell-centered positivity-preserving finite volume scheme for 3D convection–diffusion equation on distorted meshes

PurposeThis paper aims to construct positivity-preserving finite volume schemes for the three-dimensional convection–diffusion equation that are applicable to arbitrary polyhedral grids.Design/methodology/approachThe cell vertices are used to define the auxiliary unknowns, and the primary unknowns are defined at cell centers. The diffusion flux is discretized by the classical nonlinear two-point flux approximation. To ensure the fully discrete scheme has positivity-preserving property, an improved discretization method for the convection flux was presented. Besides, a new positivity-preserving vertex interpolation method is derived from the linear reconstruction in the discretization of convection flux. Moreover, the Picard iteration method may have slow convergence in solving the nonlinear system. Thus, the Anderson acceleration of Picard iteration method is used to solve the nonlinear system. A condition number monitor of matrix is employed in the Anderson acceleration method to achieve better robustness.FindingsThe new scheme is applicable to arbitrary polyhedral grids and has a second-order accuracy. The results of numerical experiments also confirm the positivity-preserving of the discretization scheme.Originality/value1. This article presents a new positivity-preserving finite volume scheme for the 3D convection–diffusion equation. 2. The new discretization scheme of convection flux is constructed. 3. A new second-order interpolation algorithm is given to eliminate the auxiliary unknowns in flux expressions. 4. An improved Anderson acceleration method is applied to accelerate the convergence of Picard iterations. 5. This scheme can solve the convection–diffusion equation on the distorted meshes with second-order accuracy.

Mini-Workshop: Nonlinear Approximation of High-dimensional Functions in Scientific Computing

Approximation techniques for high dimensional PDEs are crucial for contemporary scientific computing tasks and gained momentum in recent years due to the renewed interest in neural networks. It seems that especially nonlinear parametrizations will play an essential role in efficient and tractable approximations of high dimensional problems. We held a mini-workshop on the relation and possible synergy of neural networks and tensor product approximation. To reliably evaluate the prospect of different numerical experiments, the traditional talks were accompanied by live coding sessions.

Enhancing wind power prediction with self-attentive variational autoencoders: A comparative study

Accurate wind power prediction is critical for efficient grid management and the integration of renewable energy sources into the power grid. This study presents an effective deep-learning approach that improves short-term wind power forecasting accuracy. The method incorporates a Variational Autoencoder (VAE) with a self-attention mechanism applied in both the encoder and decoder. This empowers the model to leverage VAE's strengths in time-series modeling and nonlinear approximation while focusing on the most relevant features within the wind power data. The effectiveness of this approach is evaluated through a comprehensive comparison with eight established deep learning methods, including Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, Bidirectional LSTMs (BiLSTMs), Convolutional LSTMs (ConvLSTMs), Gated Recurrent Units (GRUs), Stacked Autoencoders (SAEs), Restricted Boltzmann Machines (RBMs), and vanilla VAEs. Real-world data from five wind turbines in France and Turkey is used for the evaluation. Five statistical metrics are employed to quantitatively assess the prediction performance of each method. The results indicate that the SA-VAE model consistently outperformed other models, achieving the highest average R2 value of 0.992, demonstrating its superior predictive capability compared to existing techniques.

A Non-Linear Successive Approximation Finite State Machine for ADCs with Robust Performance

This work presents the detailed design of a Successive Approximation Analog to Digital Data Converter (SAR ADC) using bulk 180 nm CMOS IC technology. The focus of the study is on replacing the typical Successive Approximation Register array with a Finite State Machine. This converter features a fully differential and bipolar architecture, which leads to the logic SAR nonlinear behavior. A novel digital control logic mitigates the conversion errors through the conditions in the previous logic states. The logic scheme, in combination with a robust continuous comparator, demonstrates tolerance to Process, Voltage, and Temperature variations. The architecture does not include calibration or additional redundancies in post-layout simulations to emphasize the exclusive benefits of the new SAR logic. The proposed SAR ADC achieves a 14.07 effective number of bits with 7.04 fJ/conversion step Walden figure of merit in biomedical applications.

Gaussian process learning of nonlinear dynamics

One of the pivotal tasks in scientific machine learning is to represent underlying dynamical systems from time series data. Many methods for such dynamics learning explicitly require the derivatives of state data, which are not directly available and can be approximated conventionally by finite differences. However, the discrete approximations of time derivatives may result in poor estimations when state data are scarce and/or corrupted by noise, thus compromising the predictiveness of the learned dynamical models. To overcome this technical hurdle, we propose a new method that learns nonlinear dynamics through a Bayesian inference of characterizing model parameters. This method leverages a Gaussian process representation of states, and constructs a likelihood function using the correlation between state data and their derivatives, yet prevents explicit evaluations of time derivatives. Through a Bayesian scheme, a probabilistic estimate of the model parameters is given by the posterior distribution, and thus a quantification is facilitated for uncertainties from noisy state data and the learning process. Specifically, we will discuss the applicability of the proposed method to several typical scenarios for dynamical systems: identification and estimation with an affine parametrization, nonlinear parametric approximation without prior knowledge, and general parameter estimation for a given dynamical system.