In this paper, a comparative study is performed to test the approximation ability of different neural network structures. It involves three neural networks multilayer feedforward neural network (MLFFNN), diagonal recurrent neural network (DRNN), and nonlinear autoregressive with exogenous inputs (NARX) neural network. Their robustness is also tested and compared when the system is subjected to parameter variations and disturbance signals. Further, dynamic back-propagation algorithm is used to update the parameters associated with these neural networks. Four dynamical systems of different complexities including motor-driven robotic link are considered on which the comparative study is performed. The simulation results show the superior performance of DRNN identification model over NARX and MLFFNN identification models.