Physics embedded neural network: Novel data-free approach towards scientific computing and applications in transfer learning

Abstract

Listen

Translate article

This paper introduces a novel approach for solving partial differential equations using neural networks for scientific computing applications. The proposed approach is referred to as physics embedded neural network (or PENN) due to its unique modified architecture designed to contain the partial differential equation information within the neural network, which is built upon a feed-forward neural network (NN) with fully connected layers. The key aspect of PENN is the numerical embedding of a differential equation associated with physical problem in the penultimate layer of the neural network. Here we propose that through such integration of PDE to the neural network’s penultimate layer, we can produce a fast numerical solution of the PDE, and with time performance comparable with Finite Element Method (FEM). Due to direct solution of the PDE, the NN solution occurs without training and testing phases which are normally applied when using NN in a data-based training. To demonstrate the effectiveness of the proposed architecture, we investigate variety of cases focused around second order in-homogeneous PDE in 2 dimensional (2D) problems. Additionally, we demonstrate the effectiveness of our approach by applying it to solve electromagnetic problem to illustrate its engineering applications. In this study, we also conducted a comparison of solution time and error achieved using our proposed network with a classical finite element method (FEM) solver. The convergence was found exponential with number of epochs proving an error significantly lower error compared to FEM and reported in other NN based algorithms such as Physics Informed Neural Networks. Furthermore, we explored the application of PENN in transfer learning. Our findings showed that by using transfer learning, up to 5 times additional reduction in simulation time is possible as compared to a direct solution through PENN. Overall, the study contributes to the development of efficient and accurate methods for solving PDEs through neural network optimization and provides important insights into the potential benefits of incorporating transfer learning techniques in numerical modeling of physical phenomenology.