Standard Numerical Solvers Research Articles

Analysis of Numerical Integration in RNN-Based Residuals for Fault Diagnosis of Dynamic Systems

Data-driven modeling and machine learning are widely used to model the behavior of dynamic systems. One application is the residual evaluation of technical systems where model predictions are compared with measurement data to create residuals for fault diagnosis applications. While recurrent neural network models have been shown capable of modeling complex non-linear dynamic systems, they are limited to fixed steps discrete-time simulation. Modeling using neural ordinary differential equations, however, make it possible to evaluate the state variables at specific times, compute gradients when training the model and use standard numerical solvers to explicitly model the underlying dynamic of the time-series data. Here, the effect of solver selection on the performance of neural ordinary differential equation residuals during training and evaluation is investigated. The paper includes a case study of a heavy-duty truck's after-treatment system to highlight the potential of these techniques for improving fault diagnosis performance.

Reduced order modeling for flow and transport problems with Barlow Twins self-supervised learning

We propose a unified data-driven reduced order model (ROM) that bridges the performance gap between linear and nonlinear manifold approaches. Deep learning ROM (DL-ROM) using deep-convolutional autoencoders (DC–AE) has been shown to capture nonlinear solution manifolds but fails to perform adequately when linear subspace approaches such as proper orthogonal decomposition (POD) would be optimal. Besides, most DL-ROM models rely on convolutional layers, which might limit its application to only a structured mesh. The proposed framework in this study relies on the combination of an autoencoder (AE) and Barlow Twins (BT) self-supervised learning, where BT maximizes the information content of the embedding with the latent space through a joint embedding architecture. Through a series of benchmark problems of natural convection in porous media, BT–AE performs better than the previous DL-ROM framework by providing comparable results to POD-based approaches for problems where the solution lies within a linear subspace as well as DL-ROM autoencoder-based techniques where the solution lies on a nonlinear manifold; consequently, bridges the gap between linear and nonlinear reduced manifolds. We illustrate that a proficient construction of the latent space is key to achieving these results, enabling us to map these latent spaces using regression models. The proposed framework achieves a relative error of 2% on average and 12% in the worst-case scenario (i.e., the training data is small, but the parameter space is large.). We also show that our framework provides a speed-up of 7 times 10^{6} times, in the best case, and 7 times 10^{3} times on average compared to a finite element solver. Furthermore, this BT–AE framework can operate on unstructured meshes, which provides flexibility in its application to standard numerical solvers, on-site measurements, experimental data, or a combination of these sources.

Deep learning solver for solving advection–diffusion​ equation in comparison to finite difference methods

In numerical modeling, the advection–diffusion equation describes the long-range transport of atmospheric pollutants. Most numerical models in the atmospheric science community are based on finite difference methods (FDM). In this study, we conduct a comprehensive comparative analysis of standard FDM-based numerical solvers with a deep learning-based solver, the objective of which is to solve the 2D unsteady advection–diffusion equation. The performance is compared on key performance aspects accuracy, stability, and interpolation. In the analysis, we find that despite being trained with a coarse resolution, the DNN solver is the most accurate among all the solvers. For the DNN solver, the mean absolute error and maximum absolute error of fluid concentration are lowered up to 2 orders of magnitude than the FDM-based method, which corresponds to 95% and 97% relative error reduction, respectively. The analysis also shows that the DNN solver is more stable in coarse spatial–temporal domains. Owing to its continuous nature, the DNN can interpolate a solution with consistent accuracy in a resampled spatial and temporal domain magnified up to 5 and 16 times, respectively. This study highlights the fundamental differences in the partial differential equation solving methods by comparing the DNN and FDM-based solvers and presents the DNN solver as a potential alternative to the FDM-based solvers in atmospheric numerical modeling.

NeuralSound

We present a novel learning-based modal sound synthesis approach that includes a mixed vibration solver for modal analysis and a radiation network for acoustic transfer. Our mixed vibration solver consists of a 3D sparse convolution network and a Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) module for iterative optimization. Moreover, we highlight the correlation between a standard numerical vibration solver and our network architecture. Our radiation network predicts the Far-Field Acoustic Transfer maps (FFAT Maps) from the surface vibration of the object. The overall running time of our learning-based approach for most new objects is less than one second on a RTX 3080 Ti GPU while maintaining a high sound quality close to the ground truth solved by standard numerical methods. We also evaluate the numerical and perceptual accuracy of our approach on different objects with various shapes and materials.

An extended multi-model regression approach for compressive strength prediction and optimization of a concrete mixture

Due to the significant delay and cost associated with experimental tests, a model-based evaluation of concrete compressive strength is of high value, both for the purpose of strength prediction as well as the mixture optimization. In contrast to the prior recent studies employing a single regression model, in this paper, we present a combined multi-model framework where the regression methods based on artificial neural network, random forest regression and polynomial regression are jointly implemented for compressive strength prediction with a higher accuracy. The outcomes of the individual regression models are combined via a linear weighting strategy and optimized over the training data set as a quadratic convex optimization problem. It is worth mentioning that due to the convexity of the formulated problem, the globally optimum weighting strategy is obtained via standard numerical solvers. Afterwards, employing the obtained regression model, a multi-objective genetic algorithm-based method is proposed for mixture optimization under practical constraints, where a Pareto front of the cost-CS trade-off has been obtained employing the available data set. Numerical evaluations show that the proposed multi-model regression achieves a significantly higher prediction accuracy, i.e., approximately 18% reduction in the obtained prediction mean squared error, without weight optimization, and roughly 30% reduction in the obtained prediction mean squared error with an optimized combination following a convex quadratic optimization, compared to the best single model regression method employing a multi-layered artificial neural network.

Stochastic mixed-integer programming for a spare parts inventory management problem

The German Armed Forces provide an operation contingent to support the North Atlantic Treaty Organization (NATO) Response Force (NRF). To fulfill a mission, the NRF operates a number of technical systems, mostly vehicles. Each system is composed of several parts which might fail over time, and it can only be used again in the mission if all broken parts are replaced. For short deployments (e.g., one month), the NRF troops bring with them a tightly constrained “warehouse” of spare parts. To ensure an optimal use of the space, we present a two-stage stochastic programming model where in the first stage spare parts are chosen, then failures occur at random, and in the second stage the parts are assigned to the broken systems. We carry out a scenario-based approach, where the failures are simulated by a Monte-Carlo approach. We demonstrate that the resulting mixed-integer linear program can be solved using standard numerical solvers. Using real-world input data provided by the German Armed Forces and simulated data, we analyze the sensitivity of the solutions with respect to the size of the warehouse, the service level, and the number of scenarios, and compare our approach with simpler, doctrine based warehousing strategies.

Robust observer-based tracking control under actuator constraints for power-assisted wheelchairs

Power-assisted wheelchairs (PWA) is an important growing market. The goal is to provide electrical assistive kits that are able to cope with a large family of disabled people and to equip a large variety of wheelchairs. This work is made in collaboration with Autonomad Mobility, a company that designs the hardware and sells Power-Assistance kits for wheelchairs. Several crucial issues arise, e.g. how to assist any Person with Reduced Mobility (PRM)? How to detect user’s intentions? how to cope with the lack of system information due to excessive sensor costs. Effectively, due to the variety of wheelchairs and the different unknown PRM characteristics (mass, height, force, etc.) and pathologies, it is unrealistic to provide a solution using a precise modeling of the whole system including the wheelchair, the PRM and the ground conditions. However, proposing a safe and secure solution is obviously mandatory for this application. In particular, an on-the-market solution should be also smooth and friendly for the end-user. Estimation of the human torques is a first key point to achieve such a solution, which has been already studied in our previous works. This paper exploits these estimation results to propose a robust control law for PWA systems under saturation constraints. These constraints are unavoidable due to regulations on maximum authorized speed. From a control point of view, it resumes to an output feedback control with partially unknown references (desired speed, direction), unknown parameters (wheelchair and PRM masses, available force, ground characteristics) and input constraints. Finding an effective solution for this constrained output feedback tracking control still remains open. In this paper, we propose a two-step control design using quasi Linear Parameter Varying (q-LPV) formulation to solve this challenging control problem, i.e., first design an observer for state and unknown input estimation, and second propose a robust control scheme under parameter variations and input saturations. The control procedure is reformulated as convex optimization problems involving linear matrix inequality (LMI) constraints that can be efficiently solved with standard numerical solvers. Simulations and real-time experiments are proposed to show the effectiveness of the solution.

Takagi–Sugeno Fuzzy Unknown Input Observers to Estimate Nonlinear Dynamics of Autonomous Ground Vehicles: Theory and Real-Time Verification

In this article, we address the simultaneous estimation problem of the lateral speed, the steering input, and the effective engine torque, which play a fundamental role in vehicle handling, stability control, and fault diagnosis of autonomous ground vehicles. Due to the involved longitudinal-lateral coupling dynamics and the presence of unknown inputs (UIs), a new nonlinear observer design technique is proposed to guarantee the asymptotic estimation performance. To this end, we make use of a specific Takagi-Sugeno (TS) fuzzy representation with nonlinear consequents to exactly model the nonlinear vehicle dynamics within a compact set of the vehicle state. This TS fuzzy modeling not only allows reducing significantly the real-time computational effort in estimating the vehicle variables but also enables an effective way to deal with unmeasured nonlinearities. Moreover, via a generalized Luenberger observer structure, the UI decoupling can be achieved without requiring a priori UI information. Using Lyapunov stability arguments, the UI observer design is reformulated as an optimization problem under linear matrix inequalities, which can be effectively solved with standard numerical solvers. The effectiveness of the proposed TS fuzzy UI observer design is demonstrated with real-time hardware-in-the-loop experiments.

Takagi-Sugeno Fuzzy Fault Detector Design with Finite-Frequency Specifications for Autonomous Ground Vehicles

This paper investigates the steering fault detection problem for autonomous ground vehicles (AGVs). Using an observer-based approach, a new fuzzy fault detector for steering actuator is designed for safety concern. To this end, a two degrees-of-freedom (2-DOF) nonlinear vehicle model is adopted to represent the nonlinear dynamics of AGVs. Since it is not easy to measure the lateral velocity in practice, this model is then represented in a specific Takagi-Sugeno (TS) fuzzy form with nonlinear consequents. In contrast to the conventional TS fuzzy modeling, it allows separating the unmeasured premise variables in the local nonlinear consequent, which enables a more effective way to deal with the challenging issue of unmeasured premise variables. Moreover, to minimize the effect of disturbances on system performance and maximize that of actuator faults on the generated residual, both H∞ disturbance attenuation index and H_fault sensitivity index are taken into account in a finite-frequency domain. The conditions to design fault detection TS fuzzy observer are derived using Lyapunov stability method. The design procedure can be reformulated as an optimization problem under linear matrix inequalities, efficiently solved by standard numerical solvers. Simulation results are given to verify the fault detection performance of the proposed method.

Dissipative stabilization of linear systems with time-varying general distributed delays

New methods are developed for the stabilization of a linear system with general time-varying distributed delays existing at the system’s states, inputs and outputs. In contrast to most existing literature where the function of time-varying delay is continuous and bounded, we assume it to be bounded and measurable. Furthermore, the distributed delay kernels can be any square-integrable function over a bounded interval, where the kernels are handled directly by using a decomposition scenario without using approximations. By constructing a Krasovskiĭ functional via the application of a novel integral inequality, sufficient conditions for the existence of a dissipative state feedback controller are derived in terms of matrix inequalities without utilizing the existing reciprocally convex combination lemmas. The proposed synthesis (stability) conditions, which take dissipativity into account, can be either solved directly by a standard numerical solver of semidefinite programming if they are convex, or reshaped into linear matrix inequalities, or solved via a proposed iterative algorithm. To the best of our knowledge, no existing methods can handle the synthesis problem investigated in this paper. Finally, numerical examples are presented to demonstrate the effectiveness of the proposed methodologies.

Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks

In recent years, deep learning has proven to be a viable methodology for surrogate modeling and uncertainty quantification for a vast number of physical systems. However, in their traditional form, such models can require a large amount of training data. This is of particular importance for various engineering and scientific applications where data may be extremely expensive to obtain. To overcome this shortcoming, physics-constrained deep learning provides a promising methodology as it only utilizes the governing equations. In this work, we propose a novel auto-regressive dense encoder-decoder convolutional neural network to solve and model non-linear dynamical systems without training data at a computational cost that is potentially magnitudes lower than standard numerical solvers. This model includes a Bayesian framework that allows for uncertainty quantification of the predicted quantities of interest at each time-step. We rigorously test this model on several non-linear transient partial differential equation systems including the turbulence of the Kuramoto-Sivashinsky equation, multi-shock formation and interaction with 1D Burgers' equation and 2D wave dynamics with coupled Burgers' equations. For each system, the predictive results and uncertainty are presented and discussed together with comparisons to the results obtained from traditional numerical analysis methods.

Analysis and simulation of a new multi-component two-phase flow model with phase transitions and chemical reactions

A class-II-model for multi-component mixtures recently introduced in D. Bothe and W. Dreyer, Continuum thermodynamics of chemically reacting fluid mixtures, Acta Mech., 226 (2015), 1757–1805, is investigated for simple mixtures. Bothe and Dreyer were aiming at deriving physically admissible closure conditions. Here the focus is on mathematical properties of this model. In particular, hyperbolicity of the inviscid flux Jacobian is verified for non-resonance states. Although the eigenvalues cannot be determined explicitly but have to be computed numerically an eigenvector basis is constructed depending on the eigenvalues. This basis is helpful to apply standard numerical solvers for the discretization of the model. This is verified by numerical computations for two- and three-component mixtures with and without phase transition and chemical reactions.

Data-driven non-linear elasticity: constitutive manifold construction and problem discretization

The use of constitutive equations calibrated from data has been implemented into standard numerical solvers for successfully addressing a variety problems encountered in simulation-based engineering sciences (SBES). However, the complexity remains constantly increasing due to the need of increasingly detailed models as well as the use of engineered materials. Data-Driven simulation constitutes a potential change of paradigm in SBES. Standard simulation in computational mechanics is based on the use of two very different types of equations. The first one, of axiomatic character, is related to balance laws (momentum, mass, energy, $$\ldots $$ ), whereas the second one consists of models that scientists have extracted from collected, either natural or synthetic, data. Data-driven (or data-intensive) simulation consists of directly linking experimental data to computers in order to perform numerical simulations. These simulations will employ laws, universally recognized as epistemic, while minimizing the need of explicit, often phenomenological, models. The main drawback of such an approach is the large amount of required data, some of them inaccessible from the nowadays testing facilities. Such difficulty can be circumvented in many cases, and in any case alleviated, by considering complex tests, collecting as many data as possible and then using a data-driven inverse approach in order to generate the whole constitutive manifold from few complex experimental tests, as discussed in the present work.

Intelligent computing approach to analyze the dynamics of wire coating with Oldroyd 8-constant fluid

In the study, intelligent computing technique is developed for solving the nonlinear system for wire coating analysis with the bath of Oldroyd 8-constant fluid having pressure gradient using feedforward artificial neural networks (ANNs), evolutionary computing, active-set algorithm (ASA) and their hybrid. Original partial differential equations of wire coating process are converted to nonlinear ordinary differential equation (NL-ODEs) in dimensionless form using similarity transformation. Strength of ANNs is exploited to develop mathematical model of the transformed equations by defining an unsupervised error. Training of design variables of the network is carried out globally using evolutionary computing techniques based on genetic algorithms (GAs) hybrid with ASA for rapid local convergence. Design scheme is applied to analyze the dynamics of the problem for number of variants based on dilatant constant, the pseudoplastic constant, the pressure gradient, shear stress under the effect of viscosity parameter and varying the coating thickness of the polymer. Results of the proposed method are compared with standard numerical solver for NL-ODEs based of Adams method to establish its correctness. Reliability of the method is further validated through the results of statistics based on different performance measures for accuracy and computational complexity.

Joint Transceiver Design for Linear MMSE Data Fusion in Coherent MAC Wireless Sensor Networks

This paper considers the design of minimal mean square error (MMSE) transceivers in a wireless sensor network. The problem is nonconvex and challenging, and previous results (with partial solutions and/or with convergence unproved) left much to be desired. Here we propose several approaches – 2 block coordinate descent (BCD), essentially cyclic multi-block method and its variants and distributive method to solve this problem. The proposed 2-BCD approach formulates the subproblem of joint beamformer optimization as a general second-order cone programming problem, which lends itself to standard numerical solvers and which requires no extra assumptions like previous works do. The proposed essentially cyclic multi-block approach further decomposes the joint beamformer design subproblem into multiple blocks, and rigorously solves each with semi-closed-form solution. The distributive algorithm optimizes transmitters in a decentralized manner and has never been considered in existing literature. The distributive algorithm has time complexity independent of number of sensors and is especially suitable for large-scale networks. All the previous BCD-based approaches left some singularity issue unattended as well as the convergence property unaddressed, our proposals are the first to provide a complete and provably-converging analytical solution. Extensive analysis and simulations demonstrate the merits of the novel approaches relative to existing alternatives.

Computationally efficient simulation of unsteady aerodynamics using POD on the fly

Modern industrial aircraft design requires a large amount of sufficiently accurate aerodynamic and aeroelastic simulations. Current computational fluid dynamics (CFD) solvers with aeroelastic capabilities, such as the NASA URANS unstructured solver FUN3D, require very large computational resources. Since a very large amount of simulation is necessary, the CFD cost is just unaffordable in an industrial production environment and must be significantly reduced. Thus, a more inexpensive, yet sufficiently precise solver is strongly needed. An opportunity to approach this goal could follow some recent results (Terragni and Vega 2014 SIAM J. Appl. Dyn. Syst. 13 330–65; Rapun et al 2015 Int. J. Numer. Meth. Eng. 104 844–68) on an adaptive reduced order model that combines ‘on the fly’ a standard numerical solver (to compute some representative snapshots), proper orthogonal decomposition (POD) (to extract modes from the snapshots), Galerkin projection (onto the set of POD modes), and several additional ingredients such as projecting the equations using a limited amount of points and fairly generic mode libraries. When applied to the complex Ginzburg–Landau equation, the method produces acceleration factors (comparing with standard numerical solvers) of the order of 20 and 300 in one and two space dimensions, respectively. Unfortunately, the extension of the method to unsteady, compressible flows around deformable geometries requires new approaches to deal with deformable meshes, high-Reynolds numbers, and compressibility. A first step in this direction is presented considering the unsteady compressible, two-dimensional flow around an oscillating airfoil using a CFD solver in a rigidly moving mesh. POD on the Fly gives results whose accuracy is comparable to that of the CFD solver used to compute the snapshots.

Preconditioning for Underdetermined Linear Systems with Sparse Solutions

Performance guarantees for the algorithms deployed to solve underdetermined linear systems with sparse solutions are based on the assumption that the involved system matrix has the form of an incoherent unit norm tight frame. Learned dictionaries, which are popular in sparse representations, often do not meet the necessary conditions for signal recovery. In compressed sensing (CS), recovery rates have been improved substantially with optimized projections; however, these techniques do not produce binary matrices, which are more suitable for hardware implementation. In this paper, we consider an underdetermined linear system with sparse solutions and propose a preconditioning technique that yields a system matrix having the properties of an incoherent unit norm tight frame. While existing work in preconditioning concerns greedy algorithms, the proposed technique is based on recent theoretical results for standard numerical solvers such as BP and OMP. Our simulations show that the proposed preconditioning improves the recovery rates both in sparse representations and CS; the results for CS are comparable to optimized projections.

Adaptive POD‐based low‐dimensional modeling supported by residual estimates

SummaryAn adaptive low‐dimensional model is considered to simulate time‐dependent dynamics in nonlinear dissipative systems governed by PDEs. The method combines an inexpensive POD‐based Galerkin system with short runs of a standard numerical solver that provides the snapshots necessary to first construct and then update the POD modes. Switching between the numerical solver and the Galerkin system is decided ‘on the fly’ by monitoring (i) a truncation error estimate and (ii) a residual estimate. The latter estimate is used to control the mode truncation instability and highly improves former adaptive strategies that detected this instability by monitoring consistency with a second instrumental Galerkin system based on a larger number of POD modes. The most computationally expensive run of the numerical solver occurs at the outset, when the whole set of POD modes is calculated. This step is improved by using mode libraries, which may either be generic or result from former applications of the method. The outcome is a flexible, robust, computationally inexpensive procedure that adapts itself to the local dynamics by using the faster Galerkin system for the majority of the time and few, on demand, short runs of a numerical solver. The method is illustrated considering the complex Ginzburg–Landau equation in one and two space dimensions. Copyright © 2015 John Wiley & Sons, Ltd.

Interval observers for continuous-time LPV systems with [formula omitted]/[formula omitted] performance

An approach to interval observer design for Linear Parameter-Varying (LPV) systems is proposed. It is assumed that the vector of scheduling parameters in LPV models is not available for measurement. Two different interval observers are constructed for nonnegative systems and for a generic case. Stability conditions are expressed in terms of matrix inequalities, which can be solved with respect to the observer gains using standard numerical solvers. Applying L1/L2 framework the robustness and estimation accuracy with respect to model uncertainty are analyzed. The efficiency of interval estimation for LPV models is demonstrated through numerical experiments for a microfluidic system and an academic example.

An Approximation of the Riccati Equation in Large-Scale Systems With Application to Adaptive Optics

The problem of finding linear optimal controllers and estimators, like linear quadratic regulators or Kalman filters (KFs), is solved by means of a matrix Riccati equation. A bottleneck of such an approach is that the numerical solvers for this equation are computationally intensive for systems with a high number of states, making it difficult if not impossible to apply optimal (minimum-variance) control and/or estimation methods to large-scale systems. A specific example is adaptive optics (AO) system for the next generation of extremely large telescopes, for which the number of states to be estimated by a KF is in the order of the tens of thousands, making the numerical solution of the Riccati equations problematic. In this paper, we show that for a special class of state-space systems, the discrete-time algebraic Riccati equation can be simplified with an approximation which leads to a closed-form solution, which can be computed more quickly and used as an alternative to standard numerical solvers. The class of systems for which this approximation holds includes a class of models widely employed in AO, namely autoregressive (AR) models of order 1 or 2 (AR1 and AR2). We verify a posteriori the accuracy and applicability of the proposed solution.