Loss-based optimizer switching to solve 1-D Helmholtz equation using neural networks

Abstract

In this paper, 1-D Helmholtz equation is solved using physics-informed neural networks. In general, either the L-BFGS or ADAM algorithm is used to perform the optimization procedure. Unlike ADAM, L-BFGS is structured to reduce the loss function at each iteration. However, it becomes stagnant and fails to reach the global minimum at higher frequencies due to lack of momentum in the direction of the global minimum. On the other hand, ADAM has the advantage of added momentum. However, it requires a manual tuning of hyperparameters at each frequency to converge to the global minimum. Hence, one optimizer alone is inefficient to predict acoustic field at higher frequencies. This work proposes an algorithm called loss-based optimizer switching (LOS). This approach intelligently switches between L-BFGS and ADAM based on the specific criteria on the loss value to leverage the strengths of both optimizers. The performance of the LOS is evaluated by comparing the relative error between the predicted solution and the exact solution up to 3500 Hz. At 500 Hz, the relative errors with all three algorithms lie below 1.5 × 10−3. However, at 3500 Hz, the relative errors are observed to be 6.6 × 10−3, 0.92, and 0.28 with LOS, ADAM and L-BFGS, respectively.