The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?

Stefano Markidis

doi:10.3389/fdata.2021.669097

Abstract

Physics-Informed Neural Networks (PINN) are neural networks encoding the problem governing equations, such as Partial Differential Equations (PDE), as a part of the neural network. PINNs have emerged as a new essential tool to solve various challenging problems, including computing linear systems arising from PDEs, a task for which several traditional methods exist. In this work, we focus first on evaluating the potential of PINNs as linear solvers in the case of the Poisson equation, an omnipresent equation in scientific computing. We characterize PINN linear solvers in terms of accuracy and performance under different network configurations (depth, activation functions, input data set distribution). We highlight the critical role of transfer learning. Our results show that low-frequency components of the solution converge quickly as an effect of the F-principle. In contrast, an accurate solution of the high frequencies requires an exceedingly long time. To address this limitation, we propose integrating PINNs into traditional linear solvers. We show that this integration leads to the development of new solvers whose performance is on par with other high-performance solvers, such as PETSc conjugate gradient linear solvers, in terms of performance and accuracy. Overall, while the accuracy and computational performance are still a limiting factor for the direct use of PINN linear solvers, hybrid strategies combining old traditional linear solver approaches with new emerging deep-learning techniques are among the most promising methods for developing a new class of linear solvers.

Highlights

Deep Learning (DL) has revolutionized the way of performing classification, pattern recognition, and regression tasks in various application areas, such as image and speech recognition, recommendation systems, natural language processing, drug discovery, medical imaging, bioinformatics, and fraud detection, among few examples (Goodfellow et al, 2016)
∇2u(x, y) f(x, y), (x, y) ∈ [0, 1] × [0, 1]. While this problem is linear in nature and Physics-Informed Neural Networks (PINN) can handle non-linear problems, we focus on the Poisson equation because it is one of the most solved Partial Differential Equations (PDE) in scientific applications
This paper presented a study to evaluate the potential of emerging new DL technologies to replace or accelerate old traditional approaches when solving the Poisson equation

Summary

INTRODUCTION

Deep Learning (DL) has revolutionized the way of performing classification, pattern recognition, and regression tasks in various application areas, such as image and speech recognition, recommendation systems, natural language processing, drug discovery, medical imaging, bioinformatics, and fraud detection, among few examples (Goodfellow et al, 2016). We focus on PINN application to solve a traditional scientific computing problem: the solution of a linear system arising. We solve the linear system arising from the Poisson equation, one of the most common PDEs whose solution still requires a non-negligible time with traditional approaches. PINNs are deep-learning networks that, after training (solving an optimization problem to minimize a residual function), output an approximated solution of differential equation/equations, given an input point in the integration domain (called collocation point). Poisson solvers are based on linear solvers, such as the Conjugate Gradient (CG) or Fast Fourier Transform (FFT) These approaches may require a large number of iterations before convergence and are computationally expensive as the fastest methods scale as O(Ng logNg), where Ng is the number of grid points in the simulation domain.

THE NEW

An Example

CHARACTERIZING PINNS AS LINEAR SOLVERS

THE IMPORTANCE OF TRANSFER LEARNING

THE OLD AND THE NEW

Findings

DISCUSSION AND CONCLUSION