PDE Solver Research Articles

Generative Downscaling of PDE Solvers with Physics-Guided Diffusion Models

Generative Downscaling of PDE Solvers with Physics-Guided Diffusion Models

Thermal analysis of nonlinear mixed convection effects within wavy enclosure

The analysis in this study focusses on the transfer of heat within a wavy cavity in the presence of nonlinear mixed convection and MHD. We conduct the investigation under conditions where the upper wall of the cavity remains cold, the lower wall receives heat, and the left and right wavy walls remain adiabatic. Using the PDE solver in COMSOL, the non-dimensional system is simplified, employing the finite element method. The study aims to explore the impact of parameters such as nanoparticle concentration, Hartmann number, Reynolds number, Richardson number, and nonlinear mixed convection parameters on fluid flow and heat transfer. An artificial neural network has been employed to examine the behaviour of the Nusselt number within a cavity. The isotherms reveal that the temperature is higher near the lower wall for all parameters due to the heated source. Analysis of the streamline plots indicates that heat increases as it moves towards the upper wall, influenced by the Hartman number and volume fraction parameter. The use of ANN in the study forecasts the accuracy, smoothness, and convergence of the data through plots of mean squared error (MSE), error histogram, regression analysis, and transition state plot. The authors believe that using artificial neural networks to explore the heat transfer rate under the assumptions of MHD and nonlinear mixed convection is a novel approach that has not been previously investigated. Additionally, researchers can utilise ANN concepts to predict intricate cavity formations through multiphysics. This approach will require less human effort and computing time to analyse the heat response of any dataset.

Simulation of Bingham–Papanastasiou model within trapezoidal cavity with mixed convection effects

The study of Bingham–Papanastasiou fluids is conducted in lid-driven cavity with consideration of viscous dissipation. The upper and left wall of the cavity is cold while other walls are insulated. Numerical simulations are conducted to study the isotherms, temperature profile, local and average Nusselt number. The main focus of work is to analyse the behaviour of heat transfer within trapezoidal cavity. The governing system of nonlinear dimensionless partial differential equations is analysed by using PDE solver of finite element method in COMSOL. The analysis is carried out for different parameters like Reynolds number, Bingham parameter, stress growth parameter, Eckert number and Prandtl number. It is observed that impact of Bingham parameter on temperature variation is negligible while the impact of stress parameter leads to the reduction in temperature within cavity. The novelty of this work is that no work is done for the case of trapezoidal cavity where Bingham–Papanastasiou fluid behaviour is observed under the consideration of viscous dissipation and mixed convection.

Physics-informed multi-grid neural operator: Theory and an application to porous flow simulation

We propose a physics-informed multi-grid finite neural operator as an efficient alternative for PDE solvers. The operator aims to map a random parameter function to the corresponding solution function of a linear PDE, both distributed on a fine grid (called original grid). The training of the operator involves using the physics-informed method to train a neural network on the original grid to approximate the PDE solution. The grid is then coarsened, and a new neural network is trained on the coarse grid to approximate the prediction error of the previous network. This process is iteratively repeated until the final grid's neural network achieves adequate accuracy. The combination of all neural networks from coarse to fine grids forms the multi-grid neural operator capable of mapping random parameters to solutions on the original grid. The above process involving fine-coarse-fine grid is called one V cycle (following the traditional multi-grid PDE literature), and multiple V cycles can be included into the operator to enhance its performance. Additionally, the operator integrates the idea of using coarse-grid predictions as initial guesses for fine-grid predictions. Validation is performed using three porous flow examples of varying dimensions. The operator's prediction accuracy improves with more V cycles included, and its predictions closely match results obtained from a numerical solver while executing 10 to 100 times faster.

A High-Order Fast Direct Solver for Surface PDEs

A High-Order Fast Direct Solver for Surface PDEs

SlabLU: a two-level sparse direct solver for elliptic PDEs

SlabLU: a two-level sparse direct solver for elliptic PDEs

Very High-Order Accurate Discontinuous Galerkin Method for Curved Boundaries with Polygonal Meshes

Preserving the optimal convergence order of discontinuous Galerkin (DG) discretisations in curved domains is a critical and well-known issue. The proposed approach relies on the reconstruction for off-site data (ROD) method developed originally within the finite volume framework. The main advantages are simplicity, since the PDE solver only considers polygonal domains, and versatility, since any type of boundary condition can be imposed. The developed DG–ROD method consists in splitting the boundary conditions treatment and the leading discrete equations from a classical DG formulation into two independent solvers coupled in a simple and efficient iterative procedure. A numerical benchmark is provided to assess the capability of the method with Dirichlet and Neumann boundary conditions prescribed on curved boundaries, demonstrating that the optimal convergence order is effectively achieved.

Shape deformation, disintegration, and coalescence of suspension drops: Efficient simulation enabled by graph neural networks

Understanding the behaviors of suspension drops (particle swarms) as they settle in a viscous fluid holds significant importance across various applications. Due to hydrodynamic interactions (HIs), suspension drops would undergo a series of intricate behaviors, including shape deformation, disintegration, and coalescence. This work presents the hydrodynamic interaction graph neural network (HIGNN), developed in our prior work (Ma et al., 2022), as an efficient and accurate modeling framework for simulating the dynamics of suspension drops and investigating the various behaviors they exhibit. The HIGNN effectively incorporates the many-body nature of HIs, a feature lacking in most previous simulations that employ the Stokeslet assumption. In the meanwhile, the HIGNN achieves superior computational efficiency compared to traditional, high-fidelity numerical tools such as Stokesian dynamics and PDE solvers. Moreover, the HIGNN, once trained, is applicable to predicting suspension drops across a range of particle concentrations and under diverse forces (such as gravity and Coulombic interactions). Training the HIGNN only requires the data containing a small number of particles, leading to low training cost. Our results demonstrate that the HIGNN can effectively reproduce the various behaviors of suspension drops that were previously reported in literature. More specifically, a single, initially spherical drop slowly evolves into a torus-shaped drop, as particles escape from its rear and form a tail along the sedimenting direction. Subsequently, the torus breaks into secondary droplets, each undergoing a similar transition (deformation into a torus followed by disintegration), thereby leading to a repeating cascade. Further, we quantitatively analyze the correlation between the drop’s sedimentation velocity and volume fraction. We also propose new scaling laws for evaluating both the leakage rate of particles and the expansion rate of the horizontal radius of a suspension drop. For single suspension drops, we also systematically investigate how their dynamics is affected by their initial shapes and the formed tori’s aspect ratios, as well as with or without Coulombic interactions between particles. For a pair of suspension drops, we study the process of coalescence of two vertically aligned particles and examine the effect of introducing a horizontal offset on the subsequent breakup of the coalesced drop. All simulations were executed on a single GPU, with the computation of velocities for several thousand particles requiring less than five seconds per time step. This computational efficiency enables fast and resource-saving simulations of large suspension drops over extended time scales.

Drift-Correcting Multiphysics Informed Neural Network Coupled PDE Solver

Drift-Correcting Multiphysics Informed Neural Network Coupled PDE Solver

A deep branching solver for fully nonlinear partial differential equations

We present a multidimensional deep learning implementation of a stochastic branching algorithm for the numerical solution of fully nonlinear PDEs. This approach is designed to tackle functional nonlinearities involving gradient terms of any orders, by combining the use of neural networks with a Monte Carlo branching algorithm. In comparison with other deep learning PDE solvers, it also allows us to check the consistency of the learned neural network function. Numerical experiments presented show that this algorithm can outperform deep learning approaches based on backward stochastic differential equations or the Galerkin method, and provide solution estimates that are not obtained by those methods in fully nonlinear examples.

RoseNNa: A performant, portable library for neural network inference with application to computational fluid dynamics

The rise of neural network-based machine learning ushered in high-level libraries, including TensorFlow and PyTorch, to support their functionality. Computational fluid dynamics (CFD) researchers have benefited from this trend and produced powerful neural networks that promise shorter simulation times. For example, multilayer perceptrons (MLPs) and Long Short Term Memory (LSTM) recurrent-based (RNN) architectures can represent sub-grid physical effects, like turbulence. Implementing neural networks in CFD solvers is challenging because the programming languages used for machine learning and CFD are mostly non-overlapping, We present the roseNNa library, which bridges the gap between neural network inference and CFD. RoseNNa is a non-invasive, lightweight (1000 lines), and performant tool for neural network inference, with focus on the smaller networks used to augment PDE solvers, like those of CFD, which are typically written in C/C++ or Fortran. RoseNNa accomplishes this by automatically converting trained models from typical neural network training packages into a high-performance Fortran library with C and Fortran APIs. This reduces the effort needed to access trained neural networks and maintains performance in the PDE solvers that CFD researchers build and rely upon. Results show that RoseNNa reliably outperforms PyTorch (Python) and libtorch (C++) on MLPs and LSTM RNNs with less than 100 hidden layers and 100 neurons per layer, even after removing the overhead cost of API calls. Speedups range from a factor of about 10 and 2 faster than these established libraries for the smaller and larger ends of the neural network size ranges tested. Program summaryProgram Title: RoseNNaCPC Library link to program files:https://doi.org/10.17632/srbrfx8k74.1Developer's repository link:https://github.com/comp-physics/roseNNaLicensing provisions: MITProgramming language: Fortran90 and PythonNature of problem: Neural network applications in computational fluid dynamics exhibit high promise. However, they are often not deployed on HPC systems due to limited deep-learning support in Fortran or C/C++. Previous attempts to solve this issue rely on significant manual intervention. This leaves researchers with no practical solution for conveniently porting their trained models for use in performant HPC codebases.Solution method: RoseNNa converts trained neural networks from popular machine learning frameworks to Fortran/C code for inference, requiring minimal manual intervention. Metaprogramming enables automatic code creation, and ONNX ensures flexible machine learning library support.

An incremental singular value decomposition approach for large-scale spatially parallel & distributed but temporally serial data – applied to technical flows

The article presents a strategy and its algorithm to compile a simulation-accompanying, incremental Singular Value Decomposition (SVD) for time-evolving, spatially parallel discrete data sets. The framework addresses state-of-the-art PDE solvers for computational science and engineering applications. An important characteristic of such applications is that the spatial size of the data is often time-invariant and significantly exceeds the temporal size due to the large computational grid in 3D applications. Typical examples, which are also considered in this article, relate to results extracted from unsteady flow simulations. Herein, the flow data, which progresses over time, is frequently calculated spatially parallel based on domain decomposition strategies, which allow to parallelize the simulation on distributed memory machines following a Single Instruction Multiple Data (SIMD) concept.With a view to the memory-efficient reuse of (compressed) simulation results and their CPU time-saving, sufficiently accurate generation, the paper scrutinizes the efficiency of incremental/parallel SVD approaches for such simulation examples. To improve the computational efficiency, the introduction of a bunch matrix is proposed, which enables the aggregation of multiple time steps and SVD updates, and significantly increases the efficiency. The suggested strategy is verified and validated by simple 2D laminar single-phase flows and subsequently applied to more complex 2D and 3D turbulent two-phase flows. Emphasis is given to (a) the accuracy of SVD-based reconstruction, (b) the physical realizability of the reconstructed quantities, (c) the independence of domain partitioning, (d) an efficient snapshot bunching, and (e) related implementation aspects. In addition, the influence of lower and (adaptive) upper rank thresholds on the effort and accuracy is evaluated. A final application renders the practical benefits of the approach and refers to a merchant ship in head waves at Re = 1.4×107 and Fn = 0.26. The simulation involves 2880 processor cores and the related full-rank snapshot matrix has (108×104) entries. With a numerical overhead of O(10%), this snapshot matrix can be incrementally generated and compressed by O(95%). The compression is accompanied by only small errors in the integral force and local wave elevation of O(10−2%). This qualifies the method for an efficient subsequent data processing.

JAX-DIPS: Neural bootstrapping of finite discretization methods and application to elliptic problems with discontinuities

We present a scalable strategy for development of mesh-free hybrid neuro-symbolic partial differential equation solvers based on existing mesh-based numerical discretization methods. Particularly, this strategy can be used to efficiently train neural network surrogate models of partial differential equations by (i) leveraging the accuracy and convergence properties of advanced numerical methods, solvers, and preconditioners, as well as (ii) better scalability to higher order PDEs by strictly limiting optimization to first order automatic differentiation. The presented neural bootstrapping method (hereby dubbed NBM) is based on evaluation of the finite discretization residuals of the PDE system obtained on implicit Cartesian cells centered on a set of random collocation points with respect to trainable parameters of the neural network. Importantly, the conservation laws and symmetries present in the bootstrapped finite discretization equations inform the neural network about solution regularities within local neighborhoods of training points. We apply NBM to the important class of elliptic problems with jump conditions across irregular interfaces in three spatial dimensions. We show the method is convergent such that model accuracy improves by increasing number of collocation points in the domain and preconditioning the residuals. We show NBM is competitive in terms of memory and training speed with other PINN-type frameworks. The algorithms presented here are implemented using JAX in a software package named JAX-DIPS (https://github.com/JAX-DIPS/JAX-DIPS), standing for differentiable interfacial PDE solver. We open sourced JAX-DIPS to facilitate research into use of differentiable algorithms for developing hybrid PDE solvers.

DOSnet as a non-black-box PDE solver: When deep learning meets operator splitting

Deep neural networks (DNNs) recently emerged as a promising tool for analyzing and solving complex differential equations arising in science and engineering applications. Alternative to traditional numerical schemes, learning-based solvers utilize the representation power of DNNs to approximate the input-output relations in an automated manner. However, the lack of physics-in-the-loop often makes it difficult to construct a neural network solver that simultaneously achieves high accuracy, low computational burden, and interpretability. In this work, focusing on a class of evolutionary PDEs characterized by decomposable operators, we show that the classical “operator splitting” technique can be adapted to design neural network architectures. This gives rise to a learning-based PDE solver, which we name Deep Operator-Splitting Network (DOSnet). Such non-black-box network design is constructed from the physical rules and operators governing the underlying dynamics, and is more efficient and flexible than the classical numerical schemes and standard DNNs. To demonstrate the advantages of our new AI-enhanced PDE solver, we train and validate it on several types of operator-decomposable differential equations. We also apply DOSnet to nonlinear Schrödinger equations which have important applications in the signal processing for modern optical fiber transmission systems, and experimental results show that our model has better accuracy and lower computational complexity than numerical schemes and the baseline DNNs.

PIXEL: Physics-Informed Cell Representations for Fast and Accurate PDE Solvers

With the increases in computational power and advances in machine learning, data-driven learning-based methods have gained significant attention in solving PDEs. Physics-informed neural networks (PINNs) have recently emerged and succeeded in various forward and inverse PDE problems thanks to their excellent properties, such as flexibility, mesh-free solutions, and unsupervised training. However, their slower convergence speed and relatively inaccurate solutions often limit their broader applicability in many science and engineering domains. This paper proposes a new kind of data-driven PDEs solver, physics-informed cell representations (PIXEL), elegantly combining classical numerical methods and learning-based approaches. We adopt a grid structure from the numerical methods to improve accuracy and convergence speed and overcome the spectral bias presented in PINNs. Moreover, the proposed method enjoys the same benefits in PINNs, e.g., using the same optimization frameworks to solve both forward and inverse PDE problems and readily enforcing PDE constraints with modern automatic differentiation techniques. We provide experimental results on various challenging PDEs that the original PINNs have struggled with and show that PIXEL achieves fast convergence speed and high accuracy. Project page: https://namgyukang.github.io/PIXEL/

Learning Similarity Metrics for Volumetric Simulations with Multiscale CNNs

Simulations that produce three-dimensional data are ubiquitous in science, ranging from fluid flows to plasma physics. We propose a similarity model based on entropy, which allows for the creation of physically meaningful ground truth distances for the similarity assessment of scalar and vectorial data, produced from transport and motion-based simulations. Utilizing two data acquisition methods derived from this model, we create collections of fields from numerical PDE solvers and existing simulation data repositories. Furthermore, a multiscale CNN architecture that computes a volumetric similarity metric (VolSiM) is proposed. To the best of our knowledge this is the first learning method inherently designed to address the challenges arising for the similarity assessment of high-dimensional simulation data. Additionally, the tradeoff between a large batch size and an accurate correlation computation for correlation-based loss functions is investigated, and the metric's invariance with respect to rotation and scale operations is analyzed. Finally, the robustness and generalization of VolSiM is evaluated on a large range of test data, as well as a particularly challenging turbulence case study, that is close to potential real-world applications.

Spectral methods for solving elliptic PDEs on unknown manifolds

In this paper, we propose a mesh-free numerical method for solving elliptic PDEs on unknown manifolds, identified with randomly sampled point cloud data. The PDE solver is formulated as a spectral method where the test function space is the span of the leading eigenfunctions of the Laplacian operator, which are approximated from the point cloud data. While the framework is flexible for any test functional space, we will consider the eigensolutions of a weighted Laplacian obtained from a symmetric Radial Basis Function (RBF) method induced by a weak approximation of a weighted Laplacian on an appropriate Hilbert space. In this paper, we consider a test function space that encodes the geometry of the data yet does not require us to identify and use the sampling density of the point cloud. To attain a more accurate approximation of the expansion coefficients, we adopt a second-order tangent space estimation method to improve the RBF interpolation accuracy in estimating the tangential derivatives. This spectral framework allows us to efficiently solve the PDE many times subjected to different parameters, which reduces the computational cost in the related inverse problem applications. In a well-posed elliptic PDE setting with randomly sampled point cloud data, we provide a theoretical analysis to demonstrate the convergence of the proposed solver as the sample size increases. We also report some numerical studies that show the convergence of the spectral solver on simple manifolds and unknown, rough surfaces. Our numerical results suggest that the proposed method is more accurate than a graph Laplacian-based solver on smooth manifolds. On rough manifolds, these two approaches are comparable. Due to the flexibility of the framework, we empirically found improved accuracies in both smoothed and unsmoothed Stanford bunny domains by blending the graph Laplacian eigensolutions and RBF interpolator.

Backward Deep BSDE Methods and Applications to Nonlinear Problems

We present a pathwise deep Backward Stochastic Differential Equation (BSDE) method for Forward Backward Stochastic Differential Equations with terminal conditions that time-steps the BSDE backwards and apply it to the differential rates problem as a prototypical nonlinear problem of independent financial interest. The nonlinear equation for the backward time-step is solved exactly or by a Taylor-based approximation. This is the first application of such a pathwise backward time-stepping deep BSDE approach for problems with nonlinear generators. We extend the method to the case when the initial value of the forward components X can be a parameter rather than fixed and similarly to also learn values at intermediate times. We present numerical results for a call combination and for a straddle, the latter comparing well to those obtained by Forsyth and Labahn with a specialized PDE solver.

Comparison of convective heat transfer in metal foam-filled channels of three different cross-sections

This work introduces a new approach of analyzing convective heat transfer in porous medium by considering the foam structure as a type of fin. It provides the resulting heat transfer characteristics for the design of a longitudinally flowed tube bundle reformer used for the Micro Gas Turbine Solid Oxide Fuel Cell (MGT-SOFC) hybrid process. Owing to a limited experimental database available in literature for the above-mentioned situation, a physical model is initially introduced for a channel flow configuration between two large flat plates using a commercial PDE solver. This model is then validated with experimental results available in literature. A comparison with theoretical solutions is also conducted. Later, this model is modified/adapted for a pipe flow configuration. The physical model for a channel with representative cross-section shape of a longitudinally flowed tube bundle is more complex and is therefore built in a commercial CFD-Solver. A comparative study of the heat transfer behavior in channels of different cross-sections is performed based on a new dimensionless correlation, whose physical coherence with fin efficiency is explained and mathematically proved. The applicability of the heat transfer correlation from one cross-sectional shape to the other are discussed. The proposed new treatment of the porous medium as a fin structure considerably simplifies the heat transfer analysis in porous medium by the clear physical meaning behind fin efficiency and Biot number. This relationship contributes to a better understanding of the heat heat transfer characteristics in porous media in contrast to the correlation between Nusselt number and Reynolds number. Furthermore, this correlation enables a direct comparison between foam structures of different parameters because the fin efficiency is always between 0 and 1. The strong physical background of new correlations also enhances the reliability and plausibility at characterizing and designing the metal foam for heat transfer enhancement.

Stationary Density Estimation of Itô Diffusions Using Deep Learning

In this paper, we consider the density estimation problem associated with the stationary measure of ergodic Itô diffusions from a discrete-time series that approximate the solutions of the stochastic differential equations. To take advantage of the characterization of density function through the stationary solution of a parabolic-type Fokker–Planck PDE, we proceed as follows: First, we employ deep neural networks to approximate the drift and diffusion terms of the SDE by solving appropriate supervised learning tasks. Subsequently, we solve a steady-state Fokker–Planck equation associated with the estimated drift and diffusion coefficients with a neural-network–based least squares method. We establish the convergence of the proposed scheme under appropriate mathematical assumptions, accounting for the generalization errors induced by regressing the drift and diffusion coefficients and the PDE solvers. This theoretical study relies on a recent perturbation theory of Markov chain result that shows a linear dependence of the density estimation to the error in estimating the drift term and generalization error results of nonparametric regression and PDE regression solution obtained with neural-network models. We demonstrate the effectiveness of this method by numerical simulations of a two-dimensional Student t-distribution and a 20-dimensional Langevin dynamics.