Improving Prediction of Protein Secondary Structures using Attention-enhanced Deep Neural Networks

Abstract

Protein secondary structure prediction is one of the hot research topics in computation biology. Accurate prediction of protein Secondary structures provide insights into drug discovery and design of enzyme. In addition, it plays an instrumental role in identifying structural-classes, protein-folds, and its three dimensional structure. However, the experimental determination of protein secondary structures is laborious and costly. It, therefore, hinges much on the use of computational techniques for prediction of secondary structures. In recent years, deep neural networks have been used extensively for protein secondary structure prediction. However, the deep learning models focusing on extracting local dependencies of a protein sequence face difficulties in effectively extracting non-local dependencies. Although LSTM recurrent neural network solved the problem of handling long range dependencies, these models suffer from vanishing gradients, exploding gradients and shallow layers. Moreover, these models fail to capture the dependencies that are very long. In this paper, we propose Attention augmented deep CNN-LSTM method to circumvent issues faced in LSTM RNNs. Our proposed model is able to efficiently capture both local and long range dependencies for enhancing the prediction of secondary structures. Experiments were conducted on CB6133, CB513, CASP10 and CASP11 benchmark datasets. The experimental results indicate that the performance of our method is better than the baseline methods.