Adaptive deep neural networks methods for high-dimensional partial differential equations

Abstract

We present three adaptive techniques to improve the computational performance of deep neural network (DNN) methods for high-dimensional partial differential equations (PDEs). They are adaptive choice of the loss function, adaptive activation function, and adaptive sampling, all of which will be applied to the training process of a DNN for PDEs. Several numerical experiments have shown that our adaptive techniques significantly improve the computational accuracy and accelerate the convergence speed with no need to increase the number of the layers or the number of neurons of a DNN. In particular, even for some 50-dimensional problems, the relative errors of our algorithm can achieve the order of O(10−4).