A hierarchically normalized physics-informed neural network for solving differential equations: Application for solid mechanics problems

Abstract

Physics-informed neural networks (PINNs) usually confront significant difficulties to accurately solve partial differential equations (PDEs) due to many pathologies caused by gradient failures during training process. In this paper, a gradient disease regarding the ill-conditioned loss function of the PINN methodology is intensively investigated in both theoretical and experimental aspects. Particularly, the regular PINN approaches can obtain good predictions for trivial problems but may fail to learn solutions when their PDE coefficients and/or physical domain sizes vary. Besides, the influences of neural network structure to the PINN training quality are inferred from the theoretical analyses and also confirmed by relating experiments, in which using large-width multi-layer perceptrons (MLPs) is proved to be beneficial in stabilizing PINN training process. To overcome the abovementioned restrictions, a novel PINN methodology using hierarchically normalized technique (hnPINN) motivated from the developed theory is devised. The key idea of the proposed hnPINN is, on the one hand, to transform the original PDE system into one of two proposed dimensionless forms to alleviate the negative effects of PDE coefficients and domain size; on the other hand, to use secondary output scalers to flexibly calibrate the gradient flow for training effectively and improving the solution preciseness. The determination of the secondary output scaler is formulated by a heuristic framework inspired from theoretical analyses on the hnPINN gradient flow. The obtained results from some typical PDEs and common problems in solid mechanics strongly confirm the high effectiveness of the hnPINN in terms of solution accuracy, superior convergence and performance stability in comparison to those of the vanilla PINN and the non-dimensionalized PINN (ndPINN). As a preprocessing procedure, the hnPINN method is independent on network architectures and also greatly potential to combine with other state-of-the-art PINN models for effectively solving challenged issues in practice.