DNNs (Deep Neural Networks) are widely employed in advanced applications including image and audio processing. Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are two types of DNNs that have been popular for industrial applications in recent years. RNNs are well-suited to time variation problems due to their recursive structure. RNNs are ideally suited for temporal variation concerns due to their recursive structure, whereas CNNs are often employed in computer vision applications such as object recognition. Despite the fact that CNNs and RNNs are both DNNs, their implementation differs significantly. Recurrent Neural Networks (RNN) can be used to solve the sequence to sequence problem when both the input and output have sequential structures. There are always some unseen links between the structures. The traditional RNN technique, on the other hand, has trouble examining the links between the sequences appropriately. This survey introduces some attention-based RNN models, applications, types, and DNN algorithms that can focus on different features of the input for each output item in order to investigate and exploit the implicit links between the input and output items.