Approximation Capability Of Neural Networks Research Articles

Nonsingular fixed‐time adaptive neural terminal sliding mode control for a class of uncertain second‐order nonlinear systems

AbstractIn this article, the fixed‐time adaptive neural tracking control for a class of second‐order uncertain nonlinear systems is investigated. First, a novel fast fixed‐time stable (FTS) autonomous system with the switched nonlinear term is constructed. Compared with the traditional FTS system (FTSS), the state of the proposed fast FTS system has a faster convergence rate for arbitrary initial state. Then, a fixed‐time terminal sliding mode control (FTSMC) scheme is developed based on the novel fast FTSS, and the tracking error of the closed‐loop system is verified to converge to the origin within a fixed time. The proposed FTSMC scheme not only realizes the upper bound of the settling time independent of the initial conditions but also completely avoids the singularity problem by introducing the switching sliding mode surface. Moreover, an adaptive neural FTSMC scheme is developed for the general uncertain nonlinear systems with the help of the approximation capabilities of neural networks (NNs). The feasibility of the presented control strategy is examined by the simulation of a single‐link robot system.

Rigorous a Posteriori Error Bounds for PDE-Defined PINNs.

Prediction error quantification in machine learning has been left out of most methodological investigations of neural networks (NNs), for both purely data-driven and physics-informed approaches. Beyond statistical investigations and generic results on the approximation capabilities of NNs, we present a rigorous upper bound on the prediction error of physics-informed NNs (PINNs). This bound can be calculated without the knowledge of the true solution and only with a priori available information about the characteristics of the underlying dynamical system governed by a partial differential equation (PDE). We apply this a posteriori error bound exemplarily to four problems: the transport equation, the heat equation, the Navier-Stokes equation (NSE), and the Klein-Gordon equation.

High-Dimensional Uncertainty Quantification of High-Pressure Turbine Vane Based on Multifidelity Deep Neural Networks

Abstract In this work, a new multifidelity (MF) uncertainty quantification (UQ) framework is presented and applied to the LS89 nozzle modified by fouling. Geometrical uncertainties significantly influence the aerodynamic performance of gas turbines. One representative example is given by the airfoil shape modified by fouling deposition, as in turbine nozzle vanes, which generates high-dimensional input uncertainties. However, the traditional UQ approaches suffer from the curse of dimensionality phenomenon in predicting the influence of high-dimensional uncertainties. Thus, a new approach based on multifidelity deep neural networks (MF-DNN) was proposed in this paper to solve the high-dimensional UQ problem. The basic idea of MF-DNN is to ensure the approximation capability of neural networks based on abundant low-fidelity (LF) data and few high-fidelity (HF) data. The prediction accuracy of MF-DNN was first evaluated using a 15-dimensional benchmark function. An affordable turbomachinery UQ platform was then built based on key components including the MF-DNN model, the sampling module, the parameterization module and the statistical processing module. The impact of fouling deposition on LS89 nozzle vane flow was investigated using the proposed UQ framework. In detail, the MF-DNN was fine-tuned based on bi-level numerical simulation results: the 2D Euler flow field as low-fidelity data and the 3D Reynolds-averaged Navier–Stokes (RANS) flow field as high-fidelity data. The UQ results show that the total pressure loss of LS89 vane is increased by at most 17.1% or reduced by at most 4.3%, while the mean value of the loss is increased by 3.4% compared to the baseline. The main reason for relative changes in turbine nozzle performance is that the geometric uncertainties induced by fouling deposition significantly alter the intensity of shock waves near the throat area and trailing edge. The developed UQ platform could provide a useful tool in the design and optimization of advanced turbomachinery considering high-dimensional input uncertainties.

Nonparametric Multivariate Probability Density Forecast in Smart Grids With Deep Learning

This paper proposes a nonparametric multivariate density forecast model based on deep learning. It not only offers the whole marginal distribution of each random variable in forecasting targets, but also reveals the future correlation between them. Differing from existing multivariate density forecast models, the proposed method requires no a priori hypotheses on the forecasted joint probability distribution of forecasting targets. In addition, based on the universal approximation capability of neural networks, the real joint cumulative distribution functions of forecasting targets are well-approximated by a special positive-weighted deep neural network in the proposed method. Numerical tests from different scenarios were implemented under a comprehensive verification framework for evaluation, including the very short-term forecast of the wind speed, wind power, and the day-ahead forecast of the aggregated electricity load. Testing results corroborate the superiority of the proposed method over current multivariate density forecast models considering the accordance with reality, prediction interval width, and correlations between different random variables.

Deep learning algorithms for solving differential equations: a survey

ABSTRACT Differential equations (DEs) are widely employed in the mathematical modelling of a wide range of scientific and engineering problems. The analytical solution of these DEs is typically unknown for a variety of practical problems of relevance. Several numerical methods have been developed over time to find the solution to such DEs and numerous new approaches are still being proposed daily. In recent years, deep learning has emerged as a promising method for solving high-dimensional DEs. Due to the universal approximation capability of neural networks, there is no doubt that studies in this field will continue to grow in the near future. However, there is a need to understand the best-performing neural network architectures and algorithms that demonstrated their effectiveness and ability over traditional algorithms for solving various types of high-dimensional DEs. In this survey, we provide a review of deep learning algorithms classified as artificial neural networks (ANNs) and deep neural networks (DNNs) for solutions of DEs, that have been published in the last decade (between 2011 and 2022). The key purpose of this study is to explore the research papers published in the area of numerical solutions of DEs in order to get a deeper understanding of the current situation.

An artificial neural network approach to identify the parameter in a nonlinear subdiffusion model

In this paper, we propose an artificial neural network approach to identify the parameter in a non-linear subdiffusion model from additional data. Instead of determining the parameter in the time fractional diffusion model by its form itself, we approximate it in the form of an artificial neural network. The key point of this approach relies on the approximation capability of neural networks. We formulate this inverse problem as an optimal control one, and we demonstrate the existence of the solution for the control problem and provide a mathematical analysis and the derivation of optimal conditions. Moreover, various numerical tests of the regular and singular examples have shown that the artificial neural network method (ANN) is effective. This is reinforced by its numerical comparison with the gradient descent, the alternating direction multiplier method (ADMM), physics-informed neural network (PINN) and DeepONet method.

Observer-Based Dynamic Event-Triggered Control for Multiagent Systems With Time-Varying Delay

This article is concerned with the dynamic event-triggered-based adaptive output-feedback tracking control problem of nonlinear multiagent systems with time-varying input delay. By utilizing the approximation capability of neural network (NN), a low-gain nonlinear observer is first established to estimate the immeasurable states. To mitigate the effect of time-varying input delay, an auxiliary system with communication information is designed to generate the compensation signals. Then, a distributed adaptive composite NN dynamic surface control (DSC) strategy is proposed to acquire the satisfactory tracking accuracy, where the filter errors are compensated by the introduced serial-parallel estimation model. Moreover, an effective switching dynamic event-triggered mechanism is developed to determine the communication instants and reduce the update frequency of the controller. It is proven that the consensus tracking error converges to a residual set of the origin. Finally, simulation results are presented to demonstrate the effectiveness of the proposed composite NN DSC scheme.

ANN-Based DEM Fusion and DEM Improvement Frameworks in Regions of Assam and Meghalaya using Remote Sensing Datasets

The Digital Elevation Models (DEMs) are a key and primary input to a large number of modelling processes such as disaster risk monitoring, flood modelling, hydrology, geology, geomorphology, climatology, and environmental study applications. The DEM serves as an important source of topographic information representing the continuous surface of the earth in 3 dimensions with x, y and z coordinates of any point in a grided raster form or a vector TIN form. This study is based on developing a new method using the universal approximation capability of neural networks for the fusion and improvement of L-band and X-band SAR (Synthetic Aperture Radar) DEMs in the complex terrain of Assam and Meghalaya states of the Indian geographic region using the DEM fusion technique. The high-spatial-resolution ALOS PALSAR RTC HR 12.5 m DEM products are used in a fusion framework designed with neural network models. The network adaptively learns the terrain information to produce fused DEM products. The neural network models generate the relationship between the input elevation information from ALOS PALSAR DEMs and precise reference elevation from ICESat-2 spaceborne altimetry as target data. The training and testing data samples are prepared and filtered by checking the correct range of elevation from the toposheets of this region. Different models are used to separately train for the relatively plain valley portion of the Assam region and mountainous portions of the highland Shillong plateau. The obtained fused DEMs from the developed neural net structure are assessed for their quality and accuracy by estimating the RMSE parameter. The fused DEM attains an RMSE value of 7 meters for the complete region which is a significant improvement over the input DEM RMSE of approximately 11 meters. The plain area points and mountainous region points are assessed separately to analyze the predictions from neural nets in the two types of terrains observed in this study site. Moreover, TanDEM-X 90 m DEM is improved using the neural network modeling in the geometric distortions affected areas, which shows an improvement of around 33% overall at the study site. The assessment of plain and mountainous region points for near-ground points shows an improvement of 47% and 55% respectively. The fusion framework designed using the neural network models is an effective and efficient method for obtaining fused DEMs as well as for the improvement of the existing DEMs for complex terrain.

Data-driven passivity-based control of underactuated mechanical systems via interconnection and damping assignment

Since its introduction in the late 1980s, passivity-based control (PBC) has proven to be successful in controlling many robotic systems. The connection between stability and passivity theory is the most attractive feature of controllers designed using this methodology. However, the need to solve nonlinear partial differential equations (PDE) in closed-form has been a major challenge in applying PBC to general robotic systems. Here, we introduce a systematic approach to design controllers for a class of underactuated mechanical systems based on interconnection and damping assignment. Exploiting the universal approximation capability of neural networks, we formulate a data-driven optimisation problem that discovers solutions to the required PDEs automatically. Our approach does not destroy the passivity structure, preserving the inherent stability properties. We demonstrate the efficacy of our framework on two benchmark problems: the inertia wheel pendulum and the ball and beam system.

An online data driven actor-critic-disturbance guidance law for missile-target interception with input constraints

In this article, we develop an online robust actor-critic-disturbance guidance law for a missile-target interception system with limited normal acceleration capability. Firstly, the missile-target engagement is formulated as a zero-sum pursuit-evasion game problem. The key is to seek the saddle point solution of the Hamilton Jacobi Isaacs (HJI) equation, which is generally intractable due to the nonlinearity of the problem. Then, based on the universal approximation capability of Neural Networks (NNs), we construct the critic NN, the actor NN and the disturbance NN, respectively. The Bellman error is adjusted by the normalized-least square method. The proposed scheme is proved to be Uniformly Ultimately Bounded (UUB) stable by Lyapunov method. Finally, the effectiveness and robustness of the developed method are illustrated through numerical simulations against different types of non-stationary targets and initial conditions.

Approximation capabilities of neural networks on unbounded domains

There is limited study in the literature on the representability of neural networks on unbounded domains. For some application areas, results in this direction provide additional value in the design of learning systems. Motivated by an old option pricing problem, we are led to the study of this subject. For networks with a single hidden layer, we show that under suitable conditions they are capable of universal approximation in Lp(R×[0,1]n) but not in Lp(R2×[0,1]n). For deeper networks, we prove that the ReLU network with two hidden layers is a universal approximator in Lp(Rn).

Neural-Network Based Adaptive Self-Triggered Consensus of Nonlinear Multi-Agent Systems With Sensor Saturation

This paper aims to propose a self-triggered consensus control scheme for a class of nonlinear multi-agent systems with sensor saturation. Because of the existence of unknown nonlinear dynamics, this study borrows the approximation capability of neural networks to design the consensus control protocol. This paper adopts neural network to approximate the ideal controller, instead of using the combination of neural network and adaptive method to approximate the unknown system dynamics. Thus, the extended approximation property of neural network for event-based sampling can be beneficially introduced. Moreover, the designed controller only updates at discrete time, which enables that the system can be modeled as a hybrid system with impulsive dynamics. Thus, the stability theory of impulsive systems can be used to analyze the convergence of the system. It should be noted that this is the first time to propose an effective event-triggered consensus control algorithm based on neural network. Furthermore, this paper also considers a frequently encountered phenomenon of sensor saturation. The convex hull method is adopted to deal with sensor saturation problem, instead of the widely used sector condition method. Finally, the performance of the proposed neural-network based self-triggered consensus control algorithm is demonstrated by the numerical examples.

NNs-Based Adaptive Control for Genetic Regulatory Networks With Sensor Faults

This article considers an adaptive control problem of genetic regulatory networks, where unknown sensor faults are considered. By using the function approximation capability of neural networks, a neural-networks-based gene circuit control method is designed, where the unknown sensor faults are compensated. Comparing with the existing results where regulatory functions meet known SUM logic, the regulatory functions considered in this article are unknown and do not satisfy SUM logic. Furthermore, the fault negative influence on neural network function approximation, which is caused by state sensor faults, has been compensated. In sense of Lyapunov stability theory, the closed-loop system is asymptotically bounded and all the signals in the system converge to an adjustable neighborhood of the origin. Finally, some simulation results are given to show the effectiveness of the design method.

Convergence of the deep BSDE method for coupled FBSDEs

The recently proposed numerical algorithm, deep BSDE method, has shown remarkable performance in solving high-dimensional forward-backward stochastic differential equations (FBSDEs) and parabolic partial differential equations (PDEs). This article lays a theoretical foundation for the deep BSDE method in the general case of coupled FBSDEs. In particular, a posteriori error estimation of the solution is provided and it is proved that the error converges to zero given the universal approximation capability of neural networks. Numerical results are presented to demonstrate the accuracy of the analyzed algorithm in solving high-dimensional coupled FBSDEs.

Single-Layer Learning-Based Predictive Control With Echo State Network for Pneumatic-Muscle-Actuators-Driven Exoskeleton

This article presents a single-layer learning-based predictive control strategy for pneumatic muscle actuators (PMAs)-driven lower limb exoskeleton. Although PMAs are promising for rehabilitation robots, they suffer from nonlinearities, unmodeled uncertainties, hysteresis, etc. As a consequence, the mechanism actuated by PMAs rarely involves complex dynamics, and the related precise control remains a challenging problem. Hence, considering the global approximation capability of neural networks, we use an echo state network (ESN) to approximate the dynamics of the PMAs-driven exoskeleton with a nonlinear autoregressive exogenous model and forecast its behaviors by constructing training and testing data sets. Through the model predictions, the idea of single-layer learning solves a quadratic programming problem based on the principle of predictive control over a finite future horizon. After that, the control strategy turns out to be asymptotically stable when the ESN is capable of approximating the dynamics of the exoskeleton. Passive gait training experiments are conducted with six healthy subjects to verify the effectiveness of the proposed control strategy. Compared with the traditional strategies, the proposed control strategy achieves higher tracking accuracy for passive gait training tasks.

Neural-network-based adaptive quasi-consensus of nonlinear multi-agent systems with communication constrains and switching topologies

The quasi-consensus problem is investigated for a class of nonlinear multi-agent systems with communication constrains and switching topologies, where each agent is assumed to share information only with its neighbors on some disconnected time intervals, and the underlying topology is time-varying. Due to the approximation capability of neural networks, the uncertain nonlinear dynamics is compensated by the adaptive neural network scheme. A novel neural-network-based adaptive intermittent control protocol is proposed based on each agent maintaining a neural network parametric approximator. Some novel and simple quasi-consensus criteria are derived by the Lyapunov stability theory and matrix analysis, it is proved that quasi-consensus can be reached if the measure of communication is larger than a threshold value. By the theoretical analysis, the consensus error can be reduced as small as desired. Then, the proposed method is used to consensus of multi-agent systems with known nonlinear dynamics. Finally, two simulation examples are provided to demonstrate the effectiveness of the obtained theoretical results.

Estimating Predictive Rate-Distortion Curves via Neural Variational Inference.

The Predictive Rate–Distortion curve quantifies the trade-off between compressing information about the past of a stochastic process and predicting its future accurately. Existing estimation methods for this curve work by clustering finite sequences of observations or by utilizing analytically known causal states. Neither type of approach scales to processes such as natural languages, which have large alphabets and long dependencies, and where the causal states are not known analytically. We describe Neural Predictive Rate–Distortion (NPRD), an estimation method that scales to such processes, leveraging the universal approximation capabilities of neural networks. Taking only time series data as input, the method computes a variational bound on the Predictive Rate–Distortion curve. We validate the method on processes where Predictive Rate–Distortion is analytically known. As an application, we provide bounds on the Predictive Rate–Distortion of natural language, improving on bounds provided by clustering sequences. Based on the results, we argue that the Predictive Rate–Distortion curve is more useful than the usual notion of statistical complexity for characterizing highly complex processes such as natural language.

VAULT MODELING WITH NEURAL NETWORKS

Abstract. Nowadays, the digital reconstruction of vaults is carried out using photogrammetric and laser scanning techniques able to capture the visible surface with dense point clouds. Then, different modeling strategies allow the generation of 3D models in various formats, such as meshes that interpolates the acquired point cloud, NURBS-based reconstructions based on manual, semi-automated, or automated procedures, and parametric objects for Building Information Modeling. This paper proposes a novel method that reconstructs the visible surface of a vault using neural networks. It is based on the assumption that vaults are not irregular free-form objects, but they can be reconstructed by mathematical functions calculated from the acquired point clouds. The proposed approach uses the point cloud to train a neural network that approximates vault surface. The achieved solution is not only able to consider the basic geometry of the vault, but also its irregularities that cannot be neglected in the case of accurate and detailed modeling projects of historical vaults. Considerations on the approximation capabilities of neural networks are illustrated and discussed along with the advantages of creating a mathematical representation encapsulated into a function.

Adaptive neural integral sliding‐mode control for tracking control of fully actuated uncertain surface vessels

SummaryThis paper develops a novel adaptive neural integral sliding‐mode control to enhance the tracking performance of fully actuated uncertain surface vessels. The proposed method is built based on an integrating between the benefits of the approximation capability of neural network (NN) and the high robustness and precision of the integral sliding‐mode control (ISMC). In this paper, the design of NN, which is used to approximate the unknown dynamics, is simplified such that just only one simple adaptive rule is needed. The ISMC, which can eliminate the reaching phase and offer higher tracking performance compared to the conventional sliding‐mode control, is designed such that the system robust against the approximation error and stabilize the whole system. The design procedure of the proposed controller is constructed according to the backstepping control technique so that the stability of the closed‐loop system is guaranteed based on Lyapunov criteria. The proposed method is then tested on a simulated vessel system using computer simulation and compared with other state‐of‐the‐art methods. The comparison results demonstrate the superior performance of the proposed approach.

Distributed adaptive formation control for non‐identical non‐linear multi‐agents systems based on sliding mode

This study investigates the distributed control strategy for handling the time-invariant and time-varying formation tracking problem of a class of non-linear nonidentical leader-follower multi-agent systems with uncertainties and unknown external disturbances. Firstly, the neighbour-based distributed finite-time observers are proposed for the followers to estimate the position and velocity of the leader. Then, two novel distributed adaptive formation control algorithms based on the approximation capability of neural networks and sliding mode are developed. One can prove that, with the proposed observer and the control laws, all the followers will reach the desired time-invariant or time-varying formation tracking, and achieve the consistent velocity with the leader in spite of uncertainties and disturbances. Finally, illustrative simulation examples are given to verify the effectiveness of the obtained theoretical results.