Abstract This paper proposes a diagonal recurrent neural network (DRNN) based identification model for approximating the unknown dynamics of the nonlinear plants. The proposed model offers deeper memory and a simpler structure. Thereafter, we have developed a dynamic back-propagation learning algorithm for tuning the parameters of DRNN. Further, to guarantee the faster convergence and stability of the overall system, dynamic (adaptive) learning rates are developed in the sense of Lyapunov stability method. The proposed scheme is also compared with multi-layer feed forward neural network (MLFFNN) and radial basis function network (RBFN) based identification models. Numerical experiments reveal that DRNN has performed much better in approximating the dynamics of the plant and have also shown more robustness toward system uncertainties.