Partial Differential Equation Solver Research Articles

Solving forward and inverse problems of contact mechanics using physics-informed neural networks

This paper explores the ability of physics-informed neural networks (PINNs) to solve forward and inverse problems of contact mechanics for small deformation elasticity. We deploy PINNs in a mixed-variable formulation enhanced by output transformation to enforce Dirichlet and Neumann boundary conditions as hard constraints. Inequality constraints of contact problems, namely Karush–Kuhn–Tucker (KKT) type conditions, are enforced as soft constraints by incorporating them into the loss function during network training. To formulate the loss function contribution of KKT constraints, existing approaches applied to elastoplasticity problems are investigated and we explore a nonlinear complementarity problem (NCP) function, namely Fischer–Burmeister, which possesses advantageous characteristics in terms of optimization. Based on the Hertzian contact problem, we show that PINNs can serve as pure partial differential equation (PDE) solver, as data-enhanced forward model, as inverse solver for parameter identification, and as fast-to-evaluate surrogate model. Furthermore, we demonstrate the importance of choosing proper hyperparameters, e.g. loss weights, and a combination of Adam and L-BFGS-B optimizers aiming for better results in terms of accuracy and training time.

Across Time and Space: Senju ’s Approach for Scaling Iterative Stencil Loop Accelerators on Single and Multiple FPGAs

Stencil-based applications play an essential role in high-performance systems as they occur in numerous computational areas, such as partial differential equation solving. In this context, Iterative Stencil Loops (ISLs) represent a prominent and well-known algorithmic class within the stencil domain. Specifically, ISL-based calculations iteratively apply the same stencil to a multi-dimensional point grid multiple times or until convergence. However, due to their iterative and intensive nature, ISLs are highly performance-hungry, demanding specialized solutions. Here, Field Programmable Gate Arrays (FPGAs) represent a valid architectural choice as they enable the design of custom, parallel, and scalable ISL accelerators. Besides, the regular structure of ISLs makes them an ideal candidate for automatic optimization and generation flows. For these reasons, this article introduces Senju , an automation framework for the design of highly parallel ISL accelerators targeting single-/multi-FPGA systems. Given an input description, Senju automates the entire design process and provides accurate performance estimations. The experimental evaluation shows remarkable and scalable results, outperforming single- and multi-FPGA literature approaches under different metrics. Finally, we present a new analysis of temporal and spatial parallelism trade-offs in a real-case scenario and discuss our performance through a single- and novel specialized multi-FPGA formulation of the Roofline Model.

LordNet: An efficient neural network for learning to solve parametric partial differential equations without simulated data

Neural operators, as a powerful approximation to the non-linear operators between infinite-dimensional function spaces, have proved to be promising in accelerating the solution of partial differential equations (PDE). However, it requires a large amount of simulated data, which can be costly to collect. This can be avoided by learning physics from the physics-constrained loss, which we refer to it as mean squared residual (MSR) loss constructed by the discretized PDE. We investigate the physical information in the MSR loss, which we called long-range entanglements, and identify the challenge that the neural network requires the capacity to model the long-range entanglements in the spatial domain of the PDE, whose patterns vary in different PDEs. To tackle the challenge, we propose LordNet, a tunable and efficient neural network for modeling various entanglements. Inspired by the traditional solvers, LordNet models the long-range entanglements with a series of matrix multiplications, which can be seen as the low-rank approximation to the general fully-connected layers and extracts the dominant pattern with reduced computational cost. The experiments on solving Poisson’s equation and (2D and 3D) Navier–Stokes equation demonstrate that the long-range entanglements from the MSR loss can be well modeled by the LordNet, yielding better accuracy and generalization ability than other neural networks. The results show that the Lordnet can be 40× faster than traditional PDE solvers. In addition, LordNet outperforms other modern neural network architectures in accuracy and efficiency with the smallest parameter size.

ANN-based model integrated in thermal-hydraulics codes: A case study of two-phase wall friction model

Accurate prediction of two-phase parameters is essential for the development, operation and safety of nuclear power plants. In this paper, the ANN-based model has been developed, implemented with PDE (Partial Differential Equation) solver in case study of two-phase frictional pressure drop prediction.

An Application-Driven Method for Assembling Numerical Schemes for the Solution of Complex Multiphysics Problems

Within recent years, considerable progress has been made regarding high-performance solvers for partial differential equations (PDEs), yielding potential gains in efficiency compared to industry standard tools. However, the latter largely remains the status quo for scientists and engineers focusing on applying simulation tools to specific problems in practice. We attribute this growing technical gap to the increasing complexity and knowledge required to pick and assemble state-of-the-art methods. Thus, with this work, we initiate an effort to build a common taxonomy for the most popular grid-based approximation schemes to draw comparisons regarding accuracy and computational efficiency. We then build upon this foundation and introduce a method to systematically guide an application expert through classifying a given PDE problem setting and identifying a suitable numerical scheme. Great care is taken to ensure that making a choice this way is unambiguous, i.e., the goal is to obtain a clear and reproducible recommendation. Our method not only helps to identify and assemble suitable schemes but enables the unique combination of multiple methods on a per-field basis. We demonstrate this process and its effectiveness using different model problems, each comparing the resulting numerical scheme from our method with the next best choice. For both the Allen–Cahn and advection equations, we show that substantial computational gains can be attained for the recommended numerical methods regarding accuracy and efficiency. Lastly, we outline how one can systematically analyze and classify a coupled multiphysics problem of considerable complexity with six different unknown quantities, yielding an efficient, mixed discretization that in configuration compares well to high-performance implementations from the literature.

Equivariant neural operators for gradient-consistent topology optimization

Abstract Most traditional methods for solving partial differential equations (PDEs) require the costly solving of large linear systems. Neural operators (NOs) offer remarkable speed-ups over classical numerical PDE solvers. Here, we conduct the first exploration and comparison of NOs for three-dimensional topology optimization. Specifically, we propose replacing the PDE solver within the popular Solid Isotropic Material with Penalization (SIMP) algorithm, which is its main computational bottleneck. For this, the NO not only needs to solve the PDE with sufficient accuracy but also has the additional challenge of providing accurate gradients which are necessary for SIMP’s density updates. To realize this, we do three things: (i) We introduce a novel loss term to promote gradient-consistency. (ii) We guarantee equivariance in our NOs to increase the physical correctness of predictions. (iii) We introduce a novel NO architecture called U-Net Fourier neural operator (U-Net FNO), which combines the multi-resolution properties of U-Nets with the Fourier neural operator (FNO)’s focus on local features in frequency space. In our experiments we demonstrate that the inclusion of the novel gradient loss term is necessary to obtain good results. Furthermore, enforcing group equivariance greatly improves the quality of predictions, especially on small training datasets. Finally, we show that in our experiments the U-Net FNO outperforms both a standard U-Net, as well as other FNO methods.

Neural PDE Solvers for Irregular Domains

Neural network-based approaches for solving partial differential equations (PDEs) have recently received special attention. However, most neural PDE solvers only apply to rectilinear domains and do not systematically address the imposition of boundary conditions over irregular domain boundaries. In this paper, we present a neural framework to solve partial differential equations over domains with irregularly shaped (non-rectilinear) geometric boundaries. Given the shape of the domain as an input (represented as a binary mask), our network is able to predict the solution field, and can generalize to novel (unseen) irregular domains; the key technical ingredient to realizing this model is a physics-informed loss function that directly incorporates the interior-exterior information of the geometry. We also perform a careful error analysis which reveals theoretical insights into several sources of error incurred in the model-building process. Finally, we showcase various applications in 2D and 3D, along with favorable comparisons with ground truth solutions.

Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems Using Transfer Learning

Physics-Informed Neural Network (PINN) is a data-driven solver for partial and ordinary differential equations (ODEs/PDEs). It provides a unified framework to address both forward and inverse problems. However, the complexity of the objective function often leads to training failures. This issue is particularly prominent when solving high-frequency and multi-scale problems. We proposed using transfer learning to boost the robustness and convergence of training PINN, starting training from low-frequency problems and gradually approaching high-frequency problems through fine-tuning. Through two case studies, we discovered that transfer learning can effectively train PINNs to approximate solutions from low-frequency problems to high-frequency problems without increasing network parameters. Furthermore, it requires fewer data points and less training time. We compare the PINN results using direct differences and L2 relative error showing the advantage of using transfer learning techniques. We describe our training strategy in detail, including optimizer selection, and suggest guidelines for using transfer learning to train neural networks to solve more complex problems.

Utilizing Optimal Physics-Informed Neural Networks for Dynamical Analysis of Nanocomposite One-Variable Edge Plates

This study presents an innovative Physics-Informed Neural Network (PINN) approach designed to predict the dynamic responses of the One-Variable Edge Plate (OVEP), a unique plate structure characterized by an edge defined by arbitrary mathematical functions. The OVEP is constructed from a nanocomposite material reinforced with graphene nanoplatelets. Utilizing an optimized PINN pipeline, this research successfully predicts the vibration characteristics of the OVEP, encompassing linear, nonlinear vibrations, and beating phenomena. The study also demonstrates that the optimal PINN outperforms conventional neural network in terms of stability, accuracy (achieving above 99% accuracy), and efficiency in predicting long-duration vibrations. Additionally, the computational time required for generating testing results is notably diminished compared to traditional partial differential equation (PDE) solvers (reduced by about 3 to 12 times). To demonstrate model robustness, synthetic noise is intentionally introduced into the training data. The results not only enhance our understanding of the complex dynamics of the OVEP but also highlight the effectiveness of the proposed PINN framework in capturing and forecasting the dynamic behaviors of advanced plate structures. This research presents a promising potential for addressing dynamic problems in the fields of aerospace, civil, and mechanical engineering.

Neural Physical Simulation with Multi-Resolution Hash Grid Encoding

We explore the generalization of the implicit representation in the physical simulation task. Traditional time-dependent partial differential equations (PDEs) solvers for physical simulation often adopt the grid or mesh for spatial discretization, which is memory-consuming for high resolution and lack of adaptivity. Many implicit representations like local extreme machine or Siren are proposed but they are still too compact to suffer from limited accuracy in handling local details and a long time of convergence. We contribute a neural simulation framework based on multi-resolution hash grid representation to introduce hierarchical consideration of global and local information, simultaneously. Furthermore, we propose two key strategies: 1) a numerical gradient method for computing high-order derivatives with boundary conditions; 2) a range analysis sample method for fast neural geometry boundary sampling with dynamic topologies. Our method shows much higher accuracy and strong flexibility for various simulation problems: e.g., large elastic deformations, complex fluid dynamics, and multi-scale phenomena which remain challenging for existing neural physical solvers.

Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution

Deep learning-based surrogate models have demonstrated remarkable advantages over classical solvers in terms of speed, often achieving speedups of 10 to 1000 times over traditional partial differential equation (PDE) solvers. However, a significant challenge hindering their widespread adoption in both scientific and industrial domains is the lack of understanding about their prediction uncertainties, particularly in scenarios that involve critical decision making. To address this limitation, we propose a method that integrates efficient and precise uncertainty quantification into a deep learning-based surrogate model. Our method, termed Latent Evolution of PDEs with Uncertainty Quantification (LE-PDE-UQ), endows deep learning-based surrogate models with robust and efficient uncertainty quantification capabilities for both forward and inverse problems. LE-PDE-UQ leverages latent vectors within a latent space to evolve both the system's state and its corresponding uncertainty estimation. The latent vectors are decoded to provide predictions for the system's state as well as estimates of its uncertainty. In extensive experiments, we demonstrate the accurate uncertainty quantification performance of our approach, surpassing that of strong baselines including deep ensembles, Bayesian neural network layers, and dropout. Our method excels at propagating uncertainty over extended auto-regressive rollouts, making it suitable for scenarios involving long-term predictions. Our code is available at: https://github.com/AI4Science-WestlakeU/le-pde-uq.

SNN-PDE: Learning Dynamic PDEs from Data with Simplicial Neural Networks

Dynamics of many complex systems, from weather and climate to spread of infectious diseases, can be described by partial differential equations (PDEs). Such PDEs involve unknown function(s), partial derivatives, and typically multiple independent variables. The traditional numerical methods for solving PDEs assume that the data are observed on a regular grid. However, in many applications, for example, weather and air pollution monitoring delivered by the arbitrary located weather stations of the National Weather Services, data records are irregularly spaced. Furthermore, in problems involving prediction analytics such as forecasting wildfire smoke plumes, the primary focus may be on a set of irregular locations associated with urban development. In recent years, deep learning (DL) methods and, in particular, graph neural networks (GNNs) have emerged as a new promising tool that can complement traditional PDE solvers in scenarios of the irregular spaced data, contributing to the newest research trend of physics informed machine learning (PIML). However, most existing PIML methods tend to be limited in their ability to describe higher dimensional structural properties exhibited by real world phenomena, especially, ones that live on manifolds. To address this fundamental challenge, we bring the elements of the Hodge theory and, in particular, simplicial convolution defined on the Hodge Laplacian to the emerging nexus of DL and PDEs. In contrast to conventional Laplacian and the associated convolution operation, the simplicial convolution allows us to rigorously describe diffusion across higher order structures and to better approximate the complex underlying topology and geometry of the data. The new approach, Simplicial Neural Networks for Partial Differential Equations (SNN PDE) offers a computationally efficient yet effective solution for time dependent PDEs. Our studies of a broad range of synthetic data and wildfire processes demonstrate that SNN PDE improves upon state of the art baselines in handling unstructured grids and irregular time intervals of complex physical systems and offers competitive forecasting capabilities for weather and air quality forecasting.

Efficient quantum amplitude encoding of polynomial functions

Loading functions into quantum computers represents an essential step in several quantum algorithms, such as quantum partial differential equation solvers. Therefore, the inefficiency of this process leads to a major bottleneck for the application of these algorithms. Here, we present and compare two efficient methods for the amplitude encoding of real polynomial functions on n qubits. This case holds special relevance, as any continuous function on a closed interval can be uniformly approximated with arbitrary precision by a polynomial function. The first approach relies on the matrix product state representation (MPS). We study and benchmark the approximations of the target state when the bond dimension is assumed to be small. The second algorithm combines two subroutines. Initially we encode the linear function into the quantum registers either via its MPS or with a shallow sequence of multi-controlled gates that loads the linear function's Hadamard-Walsh series, and we explore how truncating the Hadamard-Walsh series of the linear function affects the final fidelity. Applying the inverse discrete Hadamard-Walsh transform converts the state encoding the series coefficients into an amplitude encoding of the linear function. Thus, we use this construction as a building block to achieve an exact block encoding of the amplitudes corresponding to the linear function on k0 qubits and apply the quantum singular value transformation that implements a polynomial transformation to the block encoding of the amplitudes. This unitary together with the Amplitude Amplification algorithm will enable us to prepare the quantum state that encodes the polynomial function on k0 qubits. Finally we pad n−k0 qubits to generate an approximated encoding of the polynomial on n qubits, analyzing the error depending on k0. In this regard, our methodology proposes a method to improve the state-of-the-art complexity by introducing controllable errors.

DF-ParPINN: parallel PINN based on velocity potential field division and single time slice focus

Modified Benney-Luke equation (mBL equation) is a three-dimensional temporal-spatial equation with complex structures, that is a high-dimensional partial differential equation (PDE), it is also a new equation of the physical ocean field, and its solution is important for studying the internal wave-wave interaction of inclined seafloor. For conventional PDE solvers such as the pseudo-spectral method, it is difficult to solve mBL equation with both accuracy and speed. Physics-informed neural network (PINN) incorporates physical prior knowledge in deep neural networks, which can solve PDE with relative accuracy and speed. However, PINN is only suitable for solving low-dimensional PDE with simple structures, and not suitable for solving high-dimensional PDE with complex structures. This is mainly because high-dimensional PDEs usually have complex structures and high-order derivatives and are likely to be high-dimensional non-convex functions, and the high-dimensional non-convex optimization problem is an NP-hard problem, resulting in the PINN easily falling into inaccurate local optimal solutions when solving high-dimensional PDEs. Therefore, we improve the PINN for the characteristics of mBL equation and propose “DF-ParPINN: parallel PINN based on velocity potential field division and single time slice focus” to solve mBL equation with large amounts of data. DF-ParPINN consists of three modules: temporal-spatial division module of overall velocity potential field, data rational selection module of multiple time slices, and parallel computation module of high-velocity fields and low-velocity fields. The experimental results show that the solution time of DF-ParPINN is no more than 0.5s, and its accuracy is much higher than that of PINN, PIRNN, cPINN, and DeepONet. Moreover, the relative error of DF-ParPINN after deep training 1000000 epochs can be reduced to less than 0.1. The validity of DF-ParPINN proves that the improved PINN also can solve high dimensional PDE with complex structures and large amounts of data quickly and accurately, which is of great significance to the deep learning of the physical ocean field.

Response surface method and finite element-based investigation of uniform/nonuniform hydromagnetic convection of nanofluids inside a quarter-circular enclosure

Under the influence of magnetohydrodynamics (uniform, periodic, and inclined), the current research evaluates the two-dimensional unsteady dynamics of natural convection flow and heat transportation in the quarter-circular domain considering the Brownian motion of nanoparticles. The nonlinear partial differential equations (PDEs) solver finite element method (FEM) of the Galerkin type is used for the numerical simulation. In addition, the central composite design-based response surface methodology (RSM) is applied to analyze the optimization of the outcomes. To assess the models’ significance in learning the optimum thermal transport efficiency of nanofluids, an analysis of variance is performed. The impacts of several physical characteristics such as Rayleigh number, nanoparticles volume fraction, mean Nusselt number, Brownian effects, size, and shape of the nanoparticles, and magnetic effects (uniform, periodic, and inclined) are examined. The heat transfer rate is significantly influenced by the magnetic field’s period and inclined angle. As an illustration, the optimal heat transmission is noted at λ = 0.75 and γ = 0°. When Brownian effects are included, the heat transfer rate increases by 23.44% and it rises by 39.34% as nanoparticle size drops from 100 nm to 10 nm. Moreover, compared to spherical nano-sized particles, blade-shaped nano-sized particles can improve the transportation of heat by 9.56%. Furthermore, the optimization of the independent factors (Ra, Ha, ϕ, and d p) on the response function Nu (average) is demonstrated by the statistical RSM technique, and a novel correlation is proposed based on FEM numerical data. The optimal thermal transport (Nu av = 24.045) is found when Ra = 10, Ha = 0, ϕ = 0.05, and d p = 10 nm. The model’s average Nusselt number’s R 2-value of 0.9927 indicates that the results are effective.

Quasi-optimal domain decomposition method for neural network-based computation of the time-dependent Schrödinger equation

In this paper, we derive and analyze the performance of optimal/quasi-optimal Schwarz Waveform Relaxation (SWR) domain decomposition methods (DDM) for the time-dependent Schrödinger equation when implement with neural network-based Partial Differential Equations (PDE) solvers. Optimal SWR methods, which are based on Dirichlet-to-Neumann operators, are known to have a higher convergence rate than classical or optimized SWR methods. However, they are usually considered prohibitive due to their computational costs with standard PDE solvers. Thanks to Physics Informed Neural Network acceleration within the Schwarz waveform relaxation process and an efficient computation of Dirichlet-to-Neumann transmission operators, we demonstrate that optimal and quasi-optimal SWR methods can be performed almost as efficiently as classical or optimized SWR methods while maintaining a faster convergence rate. We present a few numerical examples to illustrate the performance and convergence of the proposed method.

Physics informed token transformer for solving partial differential equations

Solving partial differential equations (PDEs) is the core of many fields of science and engineering. While classical approaches are often prohibitively slow, machine learning models often fail to incorporate complete system information. Over the past few years, transformers have had a significant impact on the field of Artificial Intelligence and have seen increased usage in PDE applications. However, despite their success, transformers currently lack integration with physics and reasoning. This study aims to address this issue by introducing Physics Informed Token Transformer (PITT). The purpose of PITT is to incorporate the knowledge of physics by embedding PDEs into the learning process. PITT uses an equation tokenization method to learn an analytically-driven numerical update operator. By tokenizing PDEs and embedding partial derivatives, the transformer models become aware of the underlying knowledge behind physical processes. To demonstrate this, PITT is tested on challenging 1D and 2D PDE operator learning tasks. The results show that PITT outperforms popular neural operator models and has the ability to extract physically relevant information from governing equations.

Fast particle-mesh code for Milgromian dynamics

Context. Modified Newtonian dynamics (MOND) is a promising alternative to dark matter. To further test the theory, there is a need for fluid- and particle-dynamics simulations. The force in MOND is not a direct particle-particle interaction, but derives from a potential for which a nonlinear partial differential equation (PDE) needs to be solved. Normally, this makes the problem of simulating dynamical evolution computationally expensive. Aims. We intend to develop a fast particle-mesh (PM) code for MOND (the AQUAL formalism). Methods. We transformed the nonlinear equation for MOND into a system of linear PDEs plus one algebraic equation. An iterative scheme with the fast Fourier transform (FFT) produces successively better numerical approximations. Results. The algorithm was tested for dynamical systems in MOND where analytical solutions are known: the two-body problem, a body with a circular ring, and a spherical distribution of particles in thermal equilibrium in the self-consistent potential. Conclusions. The PM code can accurately calculate the forces at subpixel scale and reproduces the analytical solutions. Four iterations are required for the potential, but when the spatial steps are small compared to the kernel width, one iteration is suffices. The use of a smoothing kernel for the accelerations is inevitable in order to eliminate the self-gravity of the point particles. Our PDE solver is 15 to 42 times as slow as a standard Poisson solver. However, the smoothing and particle propagation takes up most of the time above one particle per 103 pixels. The FFTs, the smoothing, and the propagation part in the code can all be parallelized.

A Comparative Study of Hybrid Analytical and Laplace Transform Approaches for Solving Partial Differential Equations in Python

A Comparative Study of Hybrid Analytical and Laplace Transform Approaches for Solving Partial Differential Equations in Python

Deciphering and integrating invariants for neural operator learning with various physical mechanisms.

Neural operators have been explored as surrogate models for simulating physical systems to overcome the limitations of traditional partial differential equation(PDE) solvers. However, most existing operator learning methods assume that the data originate from a single physical mechanism, limiting their applicability and performance in more realistic scenarios. To this end, we propose the physical invariant attention neural operator (PIANO) to decipher and integrate the physical invariants for operator learning from the PDE series with various physical mechanisms. PIANO employs self-supervised learning to extract physical knowledge and attention mechanisms to integrate them into dynamic convolutional layers. Compared to existing techniques, PIANO can reduce the relative error by 13.6%-82.2% on PDE forecasting tasks across varying coefficients, forces or boundary conditions. Additionally, varied downstream tasks reveal that the PI embeddings deciphered by PIANO align well with the underlying invariants in the PDE systems, verifying the physical significance of PIANO.