Parametric Partial Differential Equations Research Articles

Reinforcement learning-based estimation for spatio-temporal systems

State estimators such as Kalman filters compute an estimate of the instantaneous state of a dynamical system from sparse sensor measurements. For spatio-temporal systems, whose dynamics are governed by partial differential equations (PDEs), state estimators are typically designed based on a reduced-order model (ROM) that projects the original high-dimensional PDE onto a computationally tractable low-dimensional space. However, ROMs are prone to large errors, which negatively affects the performance of the estimator. Here, we introduce the reinforcement learning reduced-order estimator (RL-ROE), a ROM-based estimator in which the correction term that takes in the measurements is given by a nonlinear policy trained through reinforcement learning. The nonlinearity of the policy enables the RL-ROE to compensate efficiently for errors of the ROM, while still taking advantage of the imperfect knowledge of the dynamics. Using examples involving the Burgers and Navier-Stokes equations with parametric uncertainties, we show that in the limit of very few sensors, the trained RL-ROE outperforms a Kalman filter designed using the same ROM and yields accurate instantaneous estimates of high-dimensional states corresponding to unknown initial conditions and physical parameter values. The RL-ROE opens the door to lightweight real-time sensing of systems governed by parametric PDEs.

Exploiting locality in sparse polynomial approximation of parametric elliptic PDEs and application to parameterized domains

This work studies how the choice of the representation for parametric, spatially distributed inputs to elliptic partial differential equations (PDEs) affects the efficiency of a polynomial surrogate, based on Taylor expansion, for the parameter-to-solution map. In particular, we show potential advantages of representations using functions with localized supports. As model problem, we consider the steady-state diffusion equation, where the diffusion coefficient and right-hand side depend smoothly but potentially in a highly nonlinear way on a parameter y ∈ [−1, 1]N. Following previous work for affine parameter dependence and for the lognormal case, we use pointwise instead of norm-wise bounds to prove ℓp-summability of the Taylor coefficients of the solution. As application, we consider surrogates for solutions to elliptic PDEs on parametric domains. Using a mapping to a nominal configuration, this case fits in the general framework, and higher convergence rates can be attained when modeling the parametric boundary via spatially localized functions. The theoretical results are supported by numerical experiments for the parametric domain problem, illustrating the efficiency of the proposed approach and providing further insight on numerical aspects. Although the methods and ideas are carried out for the steady-state diffusion equation, they extend easily to other elliptic and parabolic PDEs.

Parametric Neural Networks as Full-Field Surrogates for Material Model Calibration

The identification of material parameters occurring in material models is essential for structural health monitoring. Due to chemical and physical processes, building structures and materials age during their service life. This, in turn, leads to a deterioration in both the reliability and quality of the structures. Knowing the current condition of the building structures can help prevent disasters and extend service life. We developed a physics-informed neural network (PINN) [1] for the calibration of the linear-elastic material model from full-field displacement data measured by digital image correlation. In an offline-phase, the PINN is trained to learn a parameterized solution of the underlying parametric partial differential equation without the need for training data [2]. We demonstrate the ability of the parametric PINN to act as a surrogate in a least-squares based material model calibration. In order to quantify the uncertainty, we further use the parametric PINN with Markov Chain Monte Carlo based Bayesian inference. Even with artificially noisy data, the calibration produces good results for reasonable material parameter ranges. Especially in sampling based methods, parametric PINNs have the advantage that model evaluation is very cheap compared to, e.g., the Finite Element Method. Thus, information on the material condition can be provided in near real-time in the online-phase. Moreover, PINNs use a continuous ansatz and thereby avoid the need to interpolate sensor locations to the simulation domain. In our ongoing work, we plan to apply the parametric PINN to more complex material models, such as those for elasto-plastic materials. We also investigate the extension of the proposed method to more complex geometries. [1] M. Raissi et al.: Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378, 686-707 (2019). [2] A. Beltrán-Pulido et al.: Physics-Informed Neural Networks for Solving Parametric Magnetostatic Problems, IEEE Transactions on Energy Conversion 37(4), 2678-2689 (2022).

Porous-DeepONet: Learning the Solution Operators of Parametric Reactive Transport Equations in Porous Media

Reactive transport equations in porous media are critical in various scientific and engineering disciplines, but solving these equations can be computationally expensive when exploring different scenarios, such as varying porous structures and initial or boundary conditions. The deep operator network (DeepONet) has emerged as a popular deep learning framework for solving parametric partial differential equations. However, applying the DeepONet to porous media presents significant challenges due to its limited capability to extract representative features from intricate structures. To address this issue, we propose the Porous-DeepONet, a simple yet highly effective extension of the DeepONet framework that leverages convolutional neural networks (CNNs) to learn the solution operators of parametric reactive transport equations in porous media. By incorporating CNNs, we can effectively capture the intricate features of porous media, enabling accurate and efficient learning of the solution operators. We demonstrate the effectiveness of the Porous-DeepONet in accurately and rapidly learning the solution operators of parametric reactive transport equations with various boundary conditions, multiple phases, and multi-physical fields through five examples. This approach offers significant computational savings, potentially reducing the computation time by 50–1000 times compared with the finite-element method. Our work may provide a robust alternative for solving parametric reactive transport equations in porous media, paving the way for exploring complex phenomena in porous media.

Tensor-Based Data-Driven Identification of Partial Differential Equations

Abstract We present a tensor-based method for model selection which identifies the unknown partial differential equation that governs a dynamical system using only spatiotemporal measurements. The method circumvents a disadvantage of standard matrix-based methods which typically have large storage consumption. Using a recently developed multidimensional approximation of nonlinear dynamical systems (MANDy), we collect the nonlinear and partial derivative terms of the measured data and construct a low-rank dictionary tensor in the tensor-train (TT) format. A tensor-based linear regression problem is then built, which balances the learning accuracy, model complexity, and computational efficiency. An algebraic expression of the unknown equations can be extracted. Numerical results are demonstrated on datasets generated by the wave equation, the Burgers' equation, and a few parametric partial differential equations (PDEs).

Reduced Geostatistical Approach With a Fourier Neural Operator Surrogate Model for Inverse Modeling of Hydraulic Tomography

AbstractIn this study, we propose a new approach for inverse modeling of hydraulic tomography (HT) using the Fourier neural operator (FNO) as a surrogate forward model. FNO is a deep learning model that directly parameterizes the integral kernel in Fourier space to learn solution operators for parametric partial differential equations (PDEs). We trained a highly accurate FNO to learn the solution operators of PDEs in HT, which can efficiently generate multiple channels of hydraulic head fields corresponding to specific pumping tests in new parameter fields through a single forward pass. The trained FNO surrogate model is integrated with the reduced geostatistical approach (RGA) to provide a powerful tool, named RGA‐FNO, for inverse modeling of Gaussian random fields. RGA utilizes principal components to effectively reduce problem dimensions and encode geostatistical information. FNO benefits from deep learning automatic differentiation, enabling efficient backpropagation of gradients during inverse modeling. Our results show that RGA‐FNO can efficiently produce satisfactory estimations for random transmissivity fields with an exponential covariance function, which is non‐differentiable and visually non‐smooth. The RGA‐FNO application on steady‐state multi‐well HT is better than the single‐well transient pumping test for inverse modeling. The data required for inverse modeling applications of RGA‐FNO increases with the variance of the random transmissivity fields. FNO and other neural operator models have the potential to play an important role in groundwater modeling, especially in inverse modeling and experimental design, which often requires a large number of forward simulations.

Physics-Informed Neural Operator for Learning Partial Differential Equations

In this article, we propose physics-informed neural operators (PINO) that combine training data and physics constraints to learn the solution operator of a given family of parametric Partial Differential Equations (PDE). PINO is the first hybrid approach incorporating data and PDE constraints at different resolutions to learn the operator. Specifically, in PINO, we combine coarse-resolution training data with PDE constraints imposed at a higher resolution. The resulting PINO model can accurately approximate the ground-truth solution operator for many popular PDE families and shows no degradation in accuracy even under zero-shot super-resolution, that is, being able to predict beyond the resolution of training data. PINO uses the Fourier neural operator (FNO) framework that is guaranteed to be a universal approximator for any continuous operator and discretization convergent in the limit of mesh refinement. By adding PDE constraints to FNO at a higher resolution, we obtain a high-fidelity reconstruction of the ground-truth operator. Moreover, PINO succeeds in settings where no training data is available and only PDE constraints are imposed, while previous approaches, such as the Physics-Informed Neural Network (PINN), fail due to optimization challenges, for example, in multi-scale dynamic systems such as Kolmogorov flows.

A certified wavelet-based physics-informed neural network for the solution of parameterized partial differential equations

Abstract Physics Informed Neural Networks (PINNs) have frequently been used for the numerical approximation of Partial Differential Equations (PDEs). The goal of this paper is to construct PINNs along with a computable upper bound of the error, which is particularly relevant for model reduction of Parameterized PDEs (PPDEs). To this end, we suggest to use a weighted sum of expansion coefficients of the residual in terms of an adaptive wavelet expansion both for the loss function and an error bound. This approach is shown here for elliptic PPDEs using both the standard variational and an optimally stable ultra-weak formulation. Numerical examples show a very good quantitative effectivity of the wavelet-based error bound.

LordNet: An efficient neural network for learning to solve parametric partial differential equations without simulated data

Neural operators, as a powerful approximation to the non-linear operators between infinite-dimensional function spaces, have proved to be promising in accelerating the solution of partial differential equations (PDE). However, it requires a large amount of simulated data, which can be costly to collect. This can be avoided by learning physics from the physics-constrained loss, which we refer to it as mean squared residual (MSR) loss constructed by the discretized PDE. We investigate the physical information in the MSR loss, which we called long-range entanglements, and identify the challenge that the neural network requires the capacity to model the long-range entanglements in the spatial domain of the PDE, whose patterns vary in different PDEs. To tackle the challenge, we propose LordNet, a tunable and efficient neural network for modeling various entanglements. Inspired by the traditional solvers, LordNet models the long-range entanglements with a series of matrix multiplications, which can be seen as the low-rank approximation to the general fully-connected layers and extracts the dominant pattern with reduced computational cost. The experiments on solving Poisson’s equation and (2D and 3D) Navier–Stokes equation demonstrate that the long-range entanglements from the MSR loss can be well modeled by the LordNet, yielding better accuracy and generalization ability than other neural networks. The results show that the Lordnet can be 40× faster than traditional PDE solvers. In addition, LordNet outperforms other modern neural network architectures in accuracy and efficiency with the smallest parameter size.

Error estimates for POD-DL-ROMs: a deep learning framework for reduced order modeling of nonlinear parametrized PDEs enhanced by proper orthogonal decomposition

POD-DL-ROMs have been recently proposed as an extremely versatile strategy to build accurate and reliable reduced order models (ROMs) for nonlinear parametrized partial differential equations, combining (i) a preliminary dimensionality reduction obtained through proper orthogonal decomposition (POD) for the sake of efficiency, (ii) an autoencoder architecture that further reduces the dimensionality of the POD space to a handful of latent coordinates, and (iii) a dense neural network to learn the map that describes the dynamics of the latent coordinates as a function of the input parameters and the time variable. Within this work, we aim at justifying the outstanding approximation capabilities of POD-DL-ROMs by means of a thorough error analysis, showing how the sampling required to generate training data, the dimension of the POD space, and the complexity of the underlying neural networks, impact on the solutions us to formulate practical criteria to control the relative error in the approximation of the solution field of interest, and derive general error estimates. Furthermore, we show that, from a theoretical point of view, POD-DL-ROMs outperform several deep learning-based techniques in terms of model complexity. Finally, we validate our findings by means of suitable numerical experiments, ranging from parameter-dependent operators analytically defined to several parametrized PDEs.

Positional embeddings for solving PDEs with evolutional deep neural networks

This work extends the paradigm of evolutional deep neural networks (EDNNs) to solving parametric time-dependent partial differential equations (PDEs) on domains with geometric structure. By introducing positional embeddings based on eigenfunctions of the Laplace-Beltrami operator, geometric properties are encoded intrinsically and Dirichlet, Neumann and periodic boundary conditions of the PDE solution are enforced directly through the neural network architecture. The proposed embeddings lead to improved error convergence for static PDEs and extend EDNNs towards computational domains of realistic complexity.Several steps are taken to improve performance of EDNNs: Solving the EDNN update equation with a Krylov solver avoids the explicit assembly of Jacobians and enables scaling to larger neural networks. Computational efficiency is further improved by an ad-hoc active sampling scheme that uses the PDE dynamics to effectively sample collocation points. A modified linearly implicit Rosenbrock method is proposed to alleviate the time step requirements of stiff PDEs. Lastly, a completely training-free approach, which automatically enforces initial conditions and only requires time integration, is compared against EDNNs that are trained on the initial conditions. We report results for the Korteweg-de Vries equation, a nonlinear heat equation and (nonlinear) advection-diffusion problems on domains with and without holes and various boundary conditions, to demonstrate the effectiveness of the method. The numerical results highlight EDNNs as a promising surrogate model for parametrized PDEs with slow decaying Kolmogorov n-width.

Parametric learning of time-advancement operators for unstable flame evolution

This study investigates the application of machine learning, specifically Fourier neural operator (FNO) and convolutional neural network (CNN), to learn time-advancement operators for parametric partial differential equations (PDEs). Our focus is on extending existing operator learning methods to handle additional inputs representing PDE parameters. The goal is to create a unified learning approach that accurately predicts short-term solutions and provides robust long-term statistics under diverse parameter conditions, facilitating computational cost savings and accelerating development in engineering simulations. We develop and compare parametric learning methods based on FNO and CNN, evaluating their effectiveness in learning parametric-dependent solution time-advancement operators for one-dimensional PDEs and realistic flame front evolution data obtained from direct numerical simulations of the Navier–Stokes equations.

A MCMC method based on surrogate model and Gaussian process parameterization for infinite Bayesian PDE inversion

This work focuses on an inversion problem derived from parametric partial differential equations (PDEs) with an infinite-dimensional parameter, represented as a coefficient function. The objective is to estimate this coefficient function accurately despite having only noisy measurements of the PDE solution at sparse input points. Conventional methods for inversion require numerous calls to a refined PDE solver, resulting in significant computational complexity, especially for challenging PDE problems, making the inference of the coefficient function practically infeasible. To address this issue, we propose a MCMC method that combines an deep learning-based surrogate and Gaussian process parameterization to efficiently infer the posterior of the coefficient function. The surrogate model is a combination of a cost-effective coarse PDE solver and a neural network-based transformation which provides an approximate solution derived from the refined PDE solver but based on the coarse PDE solution. The coefficient function is represented by Gaussian process with finite number of spatially dependent parameters and this parametric representation will be beneficial for the preconditioned Crank-Nicolson (pCN) Markov chain Monte Carlo method to sample efficiently from the posterior distribution. Approximate Bayesian computation method is used for constructing an informative dataset for the transformation learning. Our numerical examples demonstrate the effectiveness of this approach in accurately estimating the coefficient function while significantly reducing the computational burden associated with traditional inversion techniques.

Mesh-Clustered Gaussian Process Emulator for Partial Differential Equation Boundary Value Problems

Partial differential equations (PDEs) have become an essential tool for modeling complex physical systems. Such equations are typically solved numerically via mesh-based methods, such as finite element methods, with solutions over the spatial domain. However, obtaining these solutions are often prohibitively costly, limiting the feasibility of exploring parameters in PDEs. In this article, we propose an efficient emulator that simultaneously predicts the solutions over the spatial domain, with theoretical justification of its uncertainty quantification. The novelty of the proposed method lies in the incorporation of the mesh node coordinates into the statistical model. In particular, the proposed method segments the mesh nodes into multiple clusters via a Dirichlet process prior and fits Gaussian process models with the same hyperparameters in each of them. Most importantly, by revealing the underlying clustering structures, the proposed method can provide valuable insights into qualitative features of the resulting dynamics that can be used to guide further investigations. Real examples are demonstrated to show that our proposed method has smaller prediction errors than its main competitors, with competitive computation time, and identifies interesting clusters of mesh nodes that possess physical significance, such as satisfying boundary conditions. An R package for the proposed methodology is provided in an open repository.

A multi-fidelity transfer learning strategy based on multi-channel fusion

Multi-fidelity strategies leverage a large amount of low-fidelity data combined with a smaller set of high-fidelity data, thereby achieving satisfactory results at a reasonable cost. In our research, we introduce an innovative multi-fidelity strategy that integrates the concepts of multi-fidelity data fusion and transfer learning. In the proposed framework, we incorporate auto-encoders and a multi-channel transfer learning strategy, enabling the network model to comprehend the relationship between the low-fidelity and high-fidelity models in both explicit and implicit manners. This approach not only enhances prediction accuracy but also mitigates issues such as overfitting and negative transfer, which may arise in scenarios with sparse samples. Additionally, Bayesian optimization is employed for effective hyperparameter selection. To evaluate and analyze the performance of our proposed method, we present a series of benchmark test cases. Furthermore, we also show the application of the proposed method to engineering problems. Firstly, we consider a parametrized partial differential equation problem, where high-fidelity and low-fidelity data are obtained using exact methods and simplified algorithms, respectively. Subsequently, we extend this strategy to convolutional neural network architectures, specifically addressing a pressure Poisson equation problem. We also explore the effect of the reliability of the low-fidelity data and the number of high-fidelity data on the results. The results show that the proposed method exhibits low requirements in terms of both the reliability of the low-fidelity data and the number of high-fidelity data while maintaining satisfactory accuracy metrics.

Nonlinear approximation of high-dimensional anisotropic analytic functions

Motivated by nonlinear approximation results for classes of parametric partial differential equations (PDEs), we seek to better understand so-called library approximations to analytic functions of countably infinite number of variables. Rather than approximating a function of interest by a single space, a library approximation uses a collection of spaces and the best space may be chosen for any point in the domain. In the setting of this paper, we use a specific library which consists of local Taylor approximations on sufficiently small rectangular subdomains of the (rescaled) parameter domain Y≔[−1,1]N. When the function of interest is the solution of a certain type of parametric PDE, recent results (Bonito et al., 2021 [4]) prove an upper bound on the number of spaces required to achieve a desired target accuracy. In this work, we prove a similar result for a more general class of functions with anisotropic analyticity, namely the class introduced in Bonito et al. (2021) [5]. In this way we show both where the theory developed in Bonito et al. (2021) [4] depends on being in the setting of parametric PDEs with affine diffusion coefficients, and prove a more general result outside of this setting.

A non-intrusive bi-fidelity reduced basis method for time-independent problems

Scientific and engineering problems often involve parametric partial differential equations (PDEs), such as uncertainty quantification, optimizations, and inverse problems. However, solving these PDEs repeatedly can be prohibitively expensive, especially for large-scale complex applications. To address this issue, reduced order modeling (ROM) has emerged as an effective method to reduce computational costs. However, ROM often requires significant modifications to the existing code, which can be time-consuming and complex, particularly for large-scale legacy codes. Non-intrusive methods have gained attention as an alternative approach. However, most existing non-intrusive approaches are purely data-driven and may not respect the underlying physics laws during the online stage, resulting in less accurate approximations of the reduced solution. In this study, we propose a new non-intrusive bi-fidelity reduced basis method for time-independent parametric PDEs. Our algorithm utilizes the discrete operator, solutions, and right-hand sides obtained from the high-fidelity legacy solver. By leveraging a low-fidelity model, we efficiently construct the reduced operator and right-hand side for new parameter values during the online stage. Unlike other non-intrusive ROM methods, we enforce the reduced equation during the online stage. In addition, the non-intrusive nature of our algorithm makes it straightforward and applicable to general nonlinear time-independent problems. We demonstrate its performance through several benchmark examples, including nonlinear and multiscale PDEs.

A graph convolutional autoencoder approach to model order reduction for parametrized PDEs

The present work proposes a framework for nonlinear model order reduction based on a Graph Convolutional Autoencoder (GCA-ROM). In the reduced order modeling (ROM) context, one is interested in obtaining real-time and many-query evaluations of parametric Partial Differential Equations (PDEs). Linear techniques such as Proper Orthogonal Decomposition (POD) and Greedy algorithms have been analyzed thoroughly, but they are more suitable when dealing with linear and affine models showing a fast decay of the Kolmogorov n-width. On one hand, the autoencoder architecture represents a nonlinear generalization of the POD compression procedure, allowing one to encode the main information in a latent set of variables while extracting their main features. On the other hand, Graph Neural Networks (GNNs) constitute a natural framework for studying PDE solutions defined on unstructured meshes. Here, we develop a non-intrusive and data-driven nonlinear reduction approach, exploiting GNNs to encode the reduced manifold and enable fast evaluations of parametrized PDEs. We show the capabilities of the methodology for several models: linear/nonlinear and scalar/vector problems with fast/slow decay in the physically and geometrically parametrized setting. The main properties of our approach consist of (i) high generalizability in the low-data regime even for complex behaviors, (ii) physical compliance with general unstructured grids, and (iii) exploitation of pooling and un-pooling operations to learn from scattered data.

Spectral operator learning for parametric PDEs without data reliance

In this paper, we introduce the Spectral Coefficient Learning via Operator Network (SCLON), a novel operator learning-based approach for solving parametric partial differential equations (PDEs) without the need for data harnessing. The cornerstone of our method is the spectral methodology that employs expansions using orthogonal functions, such as Fourier series and Legendre polynomials, enabling accurate PDE solutions with fewer grid points. By merging the merits of spectral methods – encompassing high accuracy, efficiency, generalization, and the exact fulfillment of boundary conditions – with the prowess of deep neural networks, SCLON offers a transformative strategy. Our approach not only eliminates the need for paired input–output training data, which typically requires extensive numerical computations, but also effectively learns and predicts solutions of complex parametric PDEs, ranging from singularly perturbed convection–diffusion equations to the Navier–Stokes equations. The proposed framework demonstrates superior performance compared to existing scientific machine learning techniques, offering solutions for multiple instances of parametric PDEs without harnessing data. The mathematical framework is robust and reliable, with a well-developed loss function derived from the weak formulation, ensuring accurate approximation of solutions while exactly satisfying boundary conditions. The method’s efficacy is further illustrated through its ability to accurately predict intricate natural behaviors like the Kolmogorov flow and boundary layers. In essence, our work pioneers a compelling avenue for parametric PDE solutions, serving as a bridge between traditional numerical methodologies and cutting-edge machine learning techniques in the realm of scientific computation.

Deep learning-based surrogate models for parametrized PDEs: Handling geometric variability through graph neural networks

Mesh-based simulations play a key role when modeling complex physical systems that, in many disciplines across science and engineering, require the solution to parametrized time-dependent nonlinear partial differential equations (PDEs). In this context, full order models (FOMs), such as those relying on the finite element method, can reach high levels of accuracy, however often yielding intensive simulations to run. For this reason, surrogate models are developed to replace computationally expensive solvers with more efficient ones, which can strike favorable trade-offs between accuracy and efficiency. This work explores the potential usage of graph neural networks (GNNs) for the simulation of time-dependent PDEs in the presence of geometrical variability. In particular, we propose a systematic strategy to build surrogate models based on a data-driven time-stepping scheme where a GNN architecture is used to efficiently evolve the system. With respect to the majority of surrogate models, the proposed approach stands out for its ability of tackling problems with parameter-dependent spatial domains, while simultaneously generalizing to different geometries and mesh resolutions. We assess the effectiveness of the proposed approach through a series of numerical experiments, involving both two- and three-dimensional problems, showing that GNNs can provide a valid alternative to traditional surrogate models in terms of computational efficiency and generalization to new scenarios.