Self-scalable Tanh (Stan): Multi-Scale Solutions for Physics-Informed Neural Networks.

Abstract

Differential equationsare fundamental in modeling numerous physical systems, including thermal, manufacturing, and meteorological systems. Traditionally, numerical methods often approximate the solutions of complex systems modeled by differential equations. With the advent of modern deep learning, Physics-informed Neural Networks (PINNs) are evolving as a new paradigm for solving differential equationswith a pseudo-closed form solution. Unlike numerical methods, the PINNs can solve the differential equationsmesh-free, integrate the experimental data, and resolve challenging inverse problems. However, one of the limitations of PINNs is the poor training caused by using the activation functions designed typically for purely data-driven problems. This work proposes a scalable tanh-based activation function for PINNs to improve learning the solutions of differential equations. The proposed Self-scalable tanh (Stan) function is smooth, non-saturating, and has a trainable parameter. It can allow an easy flow of gradients and enable systematic scaling of the input-output mapping during training. Various forward problems to solve differential equationsand inverse problems to find the parameters of differential equationsdemonstrate that the Stan activation function can achieve better training and more accurate predictions than the existing activation functions for PINN in the literature.