A deep learning feed-forward neural network framework for the solutions to singularly perturbed delay differential equations

Abstract

In this paper, we explore a deep learning feedforward artificial neural network (ANN) framework as a numerical tool for approximating the solutions to singularly perturbed delay differential equations (SPDDE). Our approach trains the network with fewer uniform data points along with linear interpolation as a dependent variable. More importantly, the exact solution is not used for training the network. A mean square or Euclidean norm type total loss function is utilized to assess the deep learning network’s performance. In the training and testing phase, our network adjusts itself to fit the tangent and the curvature by minimizing the total loss function and fine-tuning the network’s hyperparameters during the backpropagation stage. Our focus in this contribution is to investigate an answer to the question- “whether neural networks can learn to solve the differential equations with both delay and boundary layer behavior”? Traditional numerical methods are known to perform poorly in approximating the solutions to SPDDE due to boundary layer behavior, which is the sharp change in the gradient of the solution inside a region of very small width. Our proposed network architecture is used to investigate the above question for three varieties of SPDDE. The numerical results demonstrate that the fine-tuned adaptive deep learning architecture can effectively approximate the solution to SPDDE for varieties of delay and perturbation parameters. It is expected that the current method can be extendable to approximate 2D convection-diffusion partial differential equations with a boundary layer.