Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

Abstract

We consider solving the forward and inverse partial differential equations (PDEs) which have sharp solutions with physics-informed neural networks (PINNs) in this work. In particular, to better capture the sharpness of the solution, we propose the adaptive sampling methods (ASMs) based on the residual and the gradient of the solution. We first present a residual only-based ASM denoted by ASM I. In this approach, we first train the neural network using a small number of residual points and divide the computational domain into a certain number of sub-domains, then we add new residual points in the sub-domain which has the largest mean absolute value of the residual, and those points which have the largest absolute values of the residual in this sub-domain as new residual points. We further develop a second type of ASM (denoted by ASM II) based on both the residual and the gradient of the solution due to the fact that only the residual may not be able to efficiently capture the sharpness of the solution. The procedure of ASM II is almost the same as that of ASM I, and we add new residual points which have not only large residuals but also large gradients. To demonstrate the effectiveness of the present methods, we use both ASM I and ASM II to solve a number of PDEs, including the Burger equation, the compressible Euler equation, the Poisson equation over an L-shape domain as well as the high-dimensional Poisson equation. It has been shown from the numerical results that the sharp solutions can be well approximated by using either ASM I or ASM II, and both methods deliver much more accurate solutions than the original PINNs with the same number of residual points. Moreover, the ASM II algorithm has better performance in terms of accuracy, efficiency, and stability compared with the ASM I algorithm. This means that the gradient of the solution improves the stability and efficiency of the adaptive sampling procedure as well as the accuracy of the solution. Furthermore, we also employ the similar adaptive sampling technique for the data points of boundary conditions (BCs) if the sharpness of the solution is near the boundary. The result of the L-shape Poisson problem indicates that the present method can significantly improve the efficiency, stability, and accuracy.