Partial Differential Equation Solver Research Articles

Simultaneous neural network approximation for smooth functions

We establish in this work approximation results of deep neural networks for smooth functions measured in Sobolev norms, motivated by recent development of numerical solvers for partial differential equations using deep neural networks. Our approximation results are nonasymptotic in the sense that the error bounds are explicitly characterized in terms of both the width and depth of the networks simultaneously with all involved constants explicitly determined. Namely, for f∈Cs([0,1]d), we show that deep ReLU networks of width O(NlogN) and of depth O(LlogL) can achieve a nonasymptotic approximation rate of O(N−2(s−1)/dL−2(s−1)/d) with respect to the W1,p([0,1]d) norm for p∈[1,∞). If either the ReLU function or its square is applied as activation functions to construct deep neural networks of width O(NlogN) and of depth O(LlogL) to approximate f∈Cs([0,1]d), the approximation rate is O(N−2(s−n)/dL−2(s−n)/d) with respect to the Wn,p([0,1]d) norm for p∈[1,∞).

Mixed-Precision Partial Differential Equation Solver Design Based on Nonvolatile Memory

Mixed-Precision Partial Differential Equation Solver Design Based on Nonvolatile Memory

Physics‐Informed Neural Network for Nonlinear Dynamics in Fiber Optics

AbstractA physics‐informed neural network (PINN) that combines deep learning with physics is studied to solve the nonlinear Schrödinger equation for learning nonlinear dynamics in fiber optics. A systematic investigation and comprehensive verification on PINN for multiple physical effects in optical fibers is carried out, including dispersion, self‐phase modulation, and higher‐order nonlinear effects. Moreover, both the special case (soliton propagation) and general case (multipulse propagation) are investigated and realized with PINN. In existing studies, PINN is mainly effective for a single scenario. To overcome this problem, the physical parameters (i.e., pulse peak power and amplitudes of subpulses) are hereby embedded as additional input parameter controllers, which allow PINN to learn the physical constraints of different scenarios and perform good generalizability. Furthermore, PINN exhibits better performance than the data‐driven neural network using much less data, and its computational complexity (in terms of number of multiplications) is much lower than that of the split‐step Fourier method. The results show that PINN is not only an effective partial differential equation solver, but also a prospective technique to advance the scientific computing and automatic modeling in fiber optics.

PythonFOAM: In-situ data analyses with OpenFOAM and Python

We outline the development of a general-purpose Python-based data analysis tool for OpenFOAM. Our implementation relies on the construction of OpenFOAM applications that have bindings to data analysis libraries in Python. Double precision data in OpenFOAM is cast to a NumPy array using the NumPy C-API and Python modules may then be used for arbitrary data analysis and manipulation on flow-field information. We highlight how the proposed wrapper may be used for an in-situ online singular value decomposition (SVD) implemented in Python and accessed from the OpenFOAM solver PimpleFOAM. Here, ‘in-situ’ refers to a programming paradigm that allows for a concurrent computation of the data analysis on the same computational resources utilized for the partial differential equation solver. In addition, to demonstrate parallel deployments, we deploy a distributed SVD, which collects snapshot data across the ranks of a distributed simulation to compute the global left singular vectors. Crucially, both OpenFOAM and Python share the same message passing interface (MPI) communicator for this deployment which allows Python objects and functions to exchange NumPy arrays across ranks. Subsequently, we provide scaling assessments of this distributed SVD on multiple nodes of Intel Broadwell and KNL architectures for canonical test cases such as the large eddy simulations of a backward facing step and a channel flow at friction Reynolds number of 395. Finally, we demonstrate the deployment of a deep neural network for compressing the flow-field information using an autoencoder to demonstrate an ability to use state-of-the-art machine learning tools in the Python ecosystem.

A moving finite element method for solving two-dimensional coupled Burgers’ equations at high Reynolds numbers

The coupled Burgers’ equations at high Reynolds numbers usually have sharp gradients or are discontinuous in the solution. Therefore, it is difficult to obtain analytical solutions. This paper aims to use the moving finite element method proposed by Li et al. (2001) to get stable and high-precision numerical solutions for the coupled Burgers’ equations at high Reynolds numbers. The method decouples the mesh equation and partial differential equation (PDE) into two unrelated parts, mesh reconstruction and PDE solver. The mesh reconstruction constructs the harmonic mapping between the physical and logical domains through an iterative method so that the mesh structure maintains harmonics after multiple numerical integrations. We compute three classic numerical examples. Numerical results show that the moving finite element method effectively solve the coupled Burgers’ equations at high Reynolds numbers, obtain stable numerical results, and achieve higher numerical accuracy. During the evolution of the solution, the mesh is always concentrated in the position where the solution has sharp gradients.

An innovative application of deep learning in multiscale modeling of subsurface fluid flow: Reconstructing the basis functions of the mixed GMsFEM

In multiscale modeling of subsurface fluid flow in heterogeneous porous media, standard polynomial basis functions are replaced by multiscale basis functions. For instance, to produce such functions in the mixed Generalized Multiscale Finite Element Method (mixed GMsFEM), a number of Partial Differential Equations (PDEs) must be solved, which requires a considerable overhead. Thus, it makes sense to replace PDE solvers with data-driven methods, given their great capabilities and general acceptance in the recent decades. Convolutional Neural Networks (CNNs) automatically perform feature engineering, and they also need fewer parameters via defining two-dimensional convolutional filters without reducing the quality of models. This is why four distinct CNN models were developed to predict four different multiscale basis functions for the mixed GMsFEM in the present study. These models were applied to 249,375 samples, with the permeability field as the only input. The statistical results indicate that the AMSGrad optimization algorithm with a coefficient of determination (R2) of 0.8434–0.9165 and Mean Squared Error (MSE) of 0.0078–0.0206 performs slightly better than Adam with an R2 of 0.8328–0.9049 and MSE of 0.0109–0.0261. Graphically, all models precisely follow the observed trend in each coarse block. This work could contribute to the distribution of pressure and velocity in the development of oil/gas fields. Looking at this work as an image (matrix)-to-image (matrix) regression problem, the constructed data-driven-based models may have applications beyond reservoir engineering, such as hydrogeology and rock mechanics.

Numerical Simulation Approach for a Dynamically Operated Sorption-Enhanced Water-Gas Shift Reactor

A dynamically operated sorption-enhanced water–gas shift reactor is modelled to leverage its performance by means of model-based process design. This reactor shall provide CO2-free synthesis gas for e-fuel production from pure CO. The nonlinear model equations describing simultaneous adsorption and reaction are solved with three numerical approaches in MATLAB: a built-in solver for partial differential equations, a semi-discretization method in combination with an ordinary differential equation solver, and an advanced graphic implementation of the latter method in Simulink. The novel implementation in Simulink offers various advantages for dynamic simulations and is expanded to a process model with six reaction chambers. The continuous conditions in the reaction chambers and the discrete states of the valves, which enable switching between reactive adsorption and regeneration, lead to a hybrid system. Controlling the discrete states in a finite-state machine in Stateflow enables automated switching between reactive adsorption and regeneration depending on predefined conditions, such as a time span or a concentration threshold in the product gas. The established chemical reactor simulation approach features unique possibilities in terms of simulation-driven development of operating procedures for intensified reactor operation. In a base case simulation, the sorbent usage for serial operation with adjusted switching times is increased by almost 15%.

Frame Invariance and Scalability of Neural Operators for Partial Differential Equations

Partial differential equations (PDEs) play a dominant role in the mathematical modeling of many complex dynamical processes. Solving these PDEs often requires prohibitively high computational costs, especially when multiple evaluations must be made for different parameters or conditions. After training, neural operators can provide PDEs solutions significantly faster than traditional PDE solvers. In this work, invariance properties and computational complexity of two neural operators are examined for transport PDE of a scalar quantity. Neural operator based on graph kernel network (GKN) operates on graph-structured data to incorporate nonlocal dependencies. Here we propose a modified formulation of GKN to achieve frame invariance. Vector cloud neural network (VCNN) is an alternate neural operator with embedded frame invariance which operates on point cloud data. GKN-based neural operator demonstrates slightly better predictive performance compared to VCNN. However, GKN requires an excessively high computational cost that increases quadratically with the increasing number of discretized objects as compared to a linear increase for VCNN.

Flow induced bifurcation and phase-plane stability analysis of branched nanotubes resting on two parameter foundation in a magnetic environment

This paper explores the nonlinear partial differential equations (PDEs) for transverse and longitudinal vibrations of slightly curved fluid-conveying embedded branched nanotubes operating in a magnetic environment. Using Bernoulli–Euler, Nonlocal and Hamilton theories, equations of motion governing the vibrations of the nanotubes are developed. Thereafter, the developed inimitable equations are solved using the numerical PDE solver and PDE-tool in MATLAB. With the numerical solutions, visualizations and parametric studies were carried out. The results indicate that increasing the branch angle at the downstream decreases quantitative stability. Furthermore, the stability results acquired via simulation point out that the magnetic field has a damping or attenuating influence of about 20%. The solutions as presented in this study match with existing results from previous studies, hence the verifications and validations of this work. It is envisioned that the results obtained will provide better physical insights into the stability criteria for carbon nanotubes and will enhance the designs of nanomechanical carbon nanotubes-based devices that convey fluid and operate in elastic and magnetic environments.

Learning finite element convergence with the Multi-fidelity Graph Neural Network

Machine learning techniques have emerged as potential alternatives to traditional physics-based modeling and partial differential equation solvers. Among these machine learning techniques, Graph Neural Networks (GNNs) simulate physics via graph models; GNNs embed relevant physical features into graph data structures, perform message passing within the graphs, and produce new attributes based on the system’s relationships. Like many machine learning frameworks, GNNs are limited by excessive data generation costs and limited generalizability outside of a narrow training domain. To address these limitations, we introduce the Multi-Fidelity Graph Neural Network (MFGNN), a supervised machine learning framework that uses low-fidelity projections to inform high-fidelity modeling across arbitrary subdomains represented by subgraphs. We implement the MFGNN for two-dimensional elastostatic problems with finite element training data. The MFGNN is trained to produce accurate analysis given low-fidelity evaluations and emulate the convergence behavior of traditional finite element analysis (FEA). Through subdomain abstraction, we also extend the MFGNN as a general model for new boundary conditions and material domains outside of the training domain.

Compute-in-Memory for Numerical Computations.

In recent years, compute-in-memory (CIM) has been extensively studied to improve the energy efficiency of computing by reducing data movement. At present, CIM is frequently used in data-intensive computing. Data-intensive computing applications, such as all kinds of neural networks (NNs) in machine learning (ML), are regarded as ‘soft’ computing tasks. The ‘soft’ computing tasks are computations that can tolerate low computing precision with little accuracy degradation. However, ‘hard’ tasks aimed at numerical computations require high-precision computing and are also accompanied by energy efficiency problems. Numerical computations exist in lots of applications, including partial differential equations (PDEs) and large-scale matrix multiplication. Therefore, it is necessary to study CIM for numerical computations. This article reviews the recent developments of CIM for numerical computations. The different kinds of numerical methods solving partial differential equations and the transformation of matrixes are deduced in detail. This paper also discusses the iterative computation of a large-scale matrix, which tremendously affects the efficiency of numerical computations. The working procedure of the ReRAM-based partial differential equation solver is emphatically introduced. Moreover, other PDEs solvers, and other research about CIM for numerical computations, are also summarized. Finally, prospects and the future of CIM for numerical computations with high accuracy are discussed.

Mathematical modeling of unsteady flow with uniform/non-uniform temperature and magnetic intensity in a half-moon shaped domain

The mathematical modeling of two-dimensional unsteady free convective flow and thermal transport inside a half-moon shaped domain charged in the presence of uniform/non-uniform temperature and magnetic effects with Brownian motion of the nanoparticles has been conducted. Thirty-two types of nanofluids in a combination of various nanoparticles and base fluids having different sizes, shapes, and solid concentrations of nanoparticles are chosen to examine the better performance of heat transfer. The circular boundary is cooled while the diameter boundary is heated with uniform/non-uniform temperature. An external uniform/non-uniform/periodic magnetic field is imposed along diameter. The powerful partial differential equations solver, finite element method of Galerkin type, has been engaged in numerical simulation. The numerical solution's heat transfer mechanism reaches a steady state from the unsteady situation within a very short dimensionless time of about 0.65. The thermal transport rate enhances for increasing buoyancy force whereas decreases with higher magnetic intensity. The uniform thermal condition along the diameter of half-moon gives a higher thermal transport rate compared to non-uniform heating conditions. The non-uniform magnetic field provides greater values of the mean Nusselt number than the uniform field. In addition, the outcomes also predict that a better rate of temperature transport for kerosene-based nanofluid than water-based, ethylene glycol-based, and engine oil-based nanofluid. The heat transfer rate is observed at about 67.86 and 23.78% using Co-Kerosene and Co-water nanofluids, respectively, with an additional 1% nanoparticles volume fraction. The blade shape nanoparticles provide a better heat transfer rate than spherical, cylindrical, brick, and platelet shapes. Small size nanoparticles confirm a higher value of average Nusselt number than big size. Mean Nusselt number increases 22.1 and 5.4% using 1% concentrated Cu-water and Cu-engine oil nanofluid, respectively than base fluid.

Fast Method for Accelerating Convergence of Iterative Partial Differential Equation Solvers by Changing System Matrix to Laplacian Counterpart

In this work, we find that the matrix representing the curl-curl operator in a partial differential equation solver of Maxwell’s equations can be analytically decomposed into a gradient divergence operator and a Laplacian, both of which can be constructed from the mesh information without any need for computation. The curl-curl operator can hence be replaced by the Laplacian to find the divergence-free component of the field solution. The Laplacian is positive definite and well-conditioned. As a result, the convergence of an iterative solution of Maxwell’s equations can be guaranteed, and also significantly accelerated. Based on the finding, we represent the divergence-free component of the unknown field solution by deducting its curl-free component. The curl-free component resides in the nullspace of the curl-curl operator, which is also analytically known from the mesh information no matter it is a regular grid or an unstructured mesh. After the divergence-free component is rapidly solved from a Laplacian counterpart of the original system matrix, the curl-free component can also be solved from a Laplacian matrix, and hence having fast and guaranteed convergence. The total computational cost of the proposed method is simply a small number of sparse matrix-vector multiplications. The proposed method has been successfully applied to solve ill-conditioned onchip, packaging, and antenna radiation problems at both low and high frequencies, involving both inhomogeneous dielectrics and lossy conductors. Numerical experiments have demonstrated its fast and guaranteed convergence, as well as trivial computational cost independent of problem size.

An Energy Conserving Vlasov Solver That Tolerates Coarse Velocity Space Resolutions: Simulation of MMS Reconnection Events

AbstractVlasov solvers that operate on a phase‐space grid are highly accurate but also numerically demanding. Coarse velocity space resolutions, which are largely unproblematic in particle‐in‐cell (PIC) simulations, can lead to numerical heating or oscillations in continuum Vlasov methods. To address this issue, we present a new dual Vlasov solver which is based on an established positivity preserving advection scheme for the update of the distribution function and an energy conserving partial differential equation solver for the kinetic update of mean velocity and temperature. The solvers work together via moment fitting during which the maximum entropy part of the distribution function is replaced by the solution from the partial differential equation solver. This numerical scheme makes continuum Vlasov methods competitive with PIC methods concerning computational cost and enables us to model large scale reconnection in Earth's magnetosphere with a fully kinetic continuum method. The simulation results agree well with measurements by the MMS spacecraft.

A Hermite Method with a Discontinuity Sensor for Hamilton–Jacobi Equations

We present a Hermite interpolation based partial differential equation solver for Hamilton-Jacobi equations. Many Hamilton-Jacobi equations have a nonlinear dependency on the gradient, which gives rise to discontinuities in the derivatives of the solution, resulting in kinks. We built our solver with two goals in mind: 1) high order accuracy in smooth regions and 2) sharp resolution of kinks. To achieve this, we use Hermite interpolation with a smoothness sensor. The degrees-of freedom of Hermite methods are tensor-product Taylor polynomials of degree $m$ in each coordinate direction. The method uses $(m+1)^d$ degrees of freedom per node in $d$-dimensions and achieves an order of accuracy $(2m+1)$ when the solution is smooth. To obtain sharp resolution of kinks, we sense the smoothness of the solution on each cell at each timestep. If the solution is smooth, we march the interpolant forward in time with no modifications. When our method encounters a cell over which the solution is not smooth, it introduces artificial viscosity locally while proceeding normally in smooth regions. We show through numerical experiments that the solver sharply captures kinks once the solution losses continuity in the derivative while achieving $2m+1$ order accuracy in smooth regions.

Optimal Dual Schemes for Adaptive Grid Based Hexmeshing

Hexahedral meshes are a ubiquitous domain for the numerical resolution of partial differential equations. Computing a pure hexahedral mesh from an adaptively refined grid is a prominent approach to automatic hexmeshing, and requires the ability to restore the all hex property around the hanging nodes that arise at the interface between cells having different size. The most advanced tools to accomplish this task are based on mesh dualization. These approaches use topological schemes to regularize the valence of inner vertices and edges, such that dualizing the grid yields a pure hexahedral mesh. In this article, we study in detail the dual approach, and propose four main contributions to it: (i) We enumerate all the possible transitions that dual methods must be able to handle, showing that prior schemes do not natively cover all of them; (ii) We show that schemes are internally asymmetric, therefore not only their construction is ambiguous, but different implementative choices lead to hexahedral meshes with different singular structure; (iii) We explore the combinatorial space of dual schemes, selecting the minimum set that covers all the possible configurations and also yields the simplest singular structure in the output hexmesh; (iv) We enlarge the class of adaptive grids that can be transformed into pure hexahedral meshes, relaxing one of the tight topological requirements imposed by previous approaches. Our extensive experiments show that our transition schemes consistently outperform prior art in terms of ability to converge to a valid solution, amount and distribution of singular mesh edges, and element count. Last but not least, we publicly release our code and reveal a conspicuous amount of technical details that were overlooked in previous literature, lowering an entry barrier that was hard to overcome for practitioners in the field.

Integrating deep neural networks with full-waveform inversion: Reparameterization, regularization, and uncertainty quantification

Full-waveform inversion (FWI) is an accurate imaging approach for modeling the velocity structure by minimizing the misfit between recorded and predicted seismic waveforms. However, the strong nonlinearity of FWI resulting from fitting oscillatory waveforms can trap the optimization in local minima. We have adopted a neural-network-based full-waveform inversion (NNFWI) method that integrates deep neural networks with FWI by representing the velocity model with a generative neural network. Neural networks can naturally introduce spatial correlations as regularization to the generated velocity model, which suppresses noise in the gradients and mitigates local minima. The velocity model generated by neural networks is input to the same partial differential equation (PDE) solvers used in conventional FWI. The gradients of the neural networks and PDEs are calculated using automatic differentiation, which back propagates gradients through the acoustic PDEs and neural network layers to update the weights of the generative neural network. Experiments on 1D velocity models, the Marmousi model, and the 2004 BP model determine that NNFWI can mitigate local minima, especially for imaging high-contrast features such as salt bodies, and it significantly improves the inversion in the presence of noise. Adding dropout layers to the neural network model also allows analyzing the uncertainty of the inversion results through Monte Carlo dropout. NNFWI opens a new pathway to combine deep learning and FWI for exploiting the characteristics of deep neural networks and the high accuracy of PDE solvers. Because NNFWI does not require extra training data and optimization loops, it provides an attractive and straightforward alternative to conventional FWI.

Numerical Simulations of a Fluidized Granular Flow Entry Into Water: Insights Into Modeling Tsunami Generation by Pyroclastic Density Currents

AbstractThe tsunami generation potential of pyroclastic density currents (PDCs) entering the sea is poorly understood, due to limited data and observations. Thus far, tsunami generation by PDCs has been modeled in a similar manner to tsunami generation associated with landslides or debris flows, using two‐layer depth‐averaged approaches. Using the adaptive partial differential equation solver Basilisk and benchmarking with published laboratory experiments, this work explores some of the important parameters not yet accounted for in numerical models of PDC‐generated tsunamis. We use assumptions derived from experimental literature to approximate the granular, basal flow component of a PDC as a dense Newtonian fluid flowing down an inclined plane. This modeling provides insight into how the boundary condition of the slope and the viscosity of the dense granular‐fluid influence the characteristics of the waves generated. It is shown that the boundary condition of the slope has a first‐order impact on the interaction dynamics between the fluidized granular flow and water, as well as the energy transfer from the flow to the generated wave. The experimental physics is captured well in the numerical model, which confirms the underlying assumption of Newtonian fluid‐like behavior in the context of wave generation. The results from this study suggest the importance of considering vertical density and velocity stratification in wave generation models. Furthermore, we demonstrate that granular‐fluids denser than water are capable of shearing the water surface and generating significant amplitude waves, despite vigorous overturning.

Learning the solution operator of parametric partial differential equations with physics-informed DeepONets.

Partial differential equations (PDEs) play a central role in the mathematical analysis and modeling of complex dynamic processes across all corners of science and engineering. Their solution often requires laborious analytical or computational tools, associated with a cost that is markedly amplified when different scenarios need to be investigated, for example, corresponding to different initial or boundary conditions, different inputs, etc. In this work, we introduce physics-informed DeepONets, a deep learning framework for learning the solution operator of arbitrary PDEs, even in the absence of any paired input-output training data. We illustrate the effectiveness of the proposed framework in rapidly predicting the solution of various types of parametric PDEs up to three orders of magnitude faster compared to conventional PDE solvers, setting a previously unexplored paradigm for modeling and simulation of nonlinear and nonequilibrium processes in science and engineering.

Coral: a parallel spectral solver for fluid dynamics and partial differential equations

Coral: a parallel spectral solver for fluid dynamics and partial differential equations