Long-range Contact Prediction Research Articles

Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences

In the field of artificial intelligence, a combination of scale in data and model capacity enabled by unsupervised learning has led to major advances in representation learning and statistical generation. In the life sciences, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. Protein language modeling at the scale of evolution is a logical step toward predictive and generative artificial intelligence for biology. To this end, we use unsupervised learning to train a deep contextual language model on 86 billion amino acids across 250 million protein sequences spanning evolutionary diversity. The resulting model contains information about biological properties in its representations. The representations are learned from sequence data alone. The learned representation space has a multiscale organization reflecting structure from the level of biochemical properties of amino acids to remote homology of proteins. Information about secondary and tertiary structure is encoded in the representations and can be identified by linear projections. Representation learning produces features that generalize across a range of applications, enabling state-of-the-art supervised prediction of mutational effect and secondary structure and improving state-of-the-art features for long-range contact prediction.

ResPRE: high-accuracy protein contact prediction by coupling precision matrix with deep residual neural networks.

Contact-map of a protein sequence dictates the global topology of structural fold. Accurate prediction of the contact-map is thus essential to protein 3D structure prediction, which is particularly useful for the protein sequences that do not have close homology templates in the Protein Data Bank. We developed a new method, ResPRE, to predict residue-level protein contacts using inverse covariance matrix (or precision matrix) of multiple sequence alignments (MSAs) through deep residual convolutional neural network training. The approach was tested on a set of 158 non-homologous proteins collected from the CASP experiments and achieved an average accuracy of 50.6% in the top-L long-range contact prediction with L being the sequence length, which is 11.7% higher than the best of other state-of-the-art approaches ranging from coevolution coupling analysis to deep neural network training. Detailed data analyses show that the major advantage of ResPRE lies at the utilization of precision matrix that helps rule out transitional noises of contact-maps compared with the previously used covariance matrix. Meanwhile, the residual network with parallel shortcut layer connections increases the learning ability of deep neural network training. It was also found that appropriate collection of MSAs can further improve the accuracy of final contact-map predictions. The standalone package and online server of ResPRE are made freely available, which should bring important impact on protein structure and function modeling studies in particular for the distant- and non-homology protein targets. https://zhanglab.ccmb.med.umich.edu/ResPRE and https://github.com/leeyang/ResPRE. Supplementary data are available at Bioinformatics online.

Template-based and free modeling of I-TASSER and QUARK pipelines using predicted contact maps in CASP12.

We develop two complementary pipelines, "Zhang-Server" and "QUARK", based on I-TASSER and QUARK pipelines for template-based modeling (TBM) and free modeling (FM), and test them in the CASP12 experiment. The combination of I-TASSER and QUARK successfully folds three medium-size FM targets that have more than 150 residues, even though the interplay between the two pipelines still awaits further optimization. Newly developed sequence-based contact prediction by NeBcon plays a critical role to enhance the quality of models, particularly for FM targets, by the new pipelines. The inclusion of NeBcon predicted contacts as restraints in the QUARK simulations results in an average TM-score of 0.41 for the best in top five predicted models, which is 37% higher than that by the QUARK simulations without contacts. In particular, there are seven targets that are converted from non-foldable to foldable (TM-score >0.5) due to the use of contact restraints in the simulations. Another additional feature in the current pipelines is the local structure quality prediction by ResQ, which provides a robust residue-level modeling error estimation. Despite the success, significant challenges still remain in ab initio modeling of multi-domain proteins and folding of β-proteins with complicated topologies bound by long-range strand-strand interactions. Improvements on domain boundary and long-range contact prediction, as well as optimal use of the predicted contacts and multiple threading alignments, are critical to address these issues seen in the CASP12 experiment.

Folding Membrane Proteins by Contacts Inferred from Non-Membrane Proteins and Near-Atomic Level Refinement

Computational prediction of membrane protein structure is very challenging due to lack of sufficient solved structures as templates or training data. Recently direct evolutionary coupling (EC) analysis has shed some light on protein contact prediction and accordingly contact-assisted folding. Limited by information within a single protein family, EC analysis is effective only on some very large-sized families. We have developed a deep transfer learning method to predict contacts for membrane proteins not in a large family, by making use of information in thousands of non-membrane protein families. Deep learning can learn complex patterns from large datasets and has recently revolutionized object and speech recognition and the GO game. Our deep learning method learns contact patterns and the complex relationship between contacts and protein features (amino acid property, residue conservation level and residue co-evolution strength) from non-membrane proteins, but can predict membrane protein contacts very well. Tested on 398 non-redundant membrane proteins, the average top L/10 long-range contact prediction accuracy of our method is 0.78, much better than the representative EC method CCMpred (0.52), the CASP11 winner MetaPSICOV (0.61) and the methods trained by only membrane proteins (0.60). When some membrane proteins are added into the training data, we can further improve accuracy by ∼4%. Using only our predicted contacts, we can correctly fold 160 test membrane proteins with RMSD <8A while homology modeling, CCMpred and MetaPSICOV contacts can fold only 10, 60 and 87 proteins, respectively. We further refine the 3D models built from predicted contacts using Upside, a near-atom-level and ultra-fast molecular dynamics method. In addition to membrane-, residue-specific and depth-dependent energy potential, Upside improves model quality by also refining non-membrane portions and refolding loop regions. Initial results indicate that Upside can indeed improve our contact-assisted models.

Evaluation of residue–residue contact predictions in CASP9

This work presents the results of the assessment of the intramolecular residue-residue contact predictions submitted to CASP9. The methodology for the assessment does not differ from that used in previous CASPs, with two basic evaluation measures being the precision in recognizing contacts and the difference between the distribution of distances in the subset of predicted contact pairs versus all pairs of residues in the structure. The emphasis is placed on the prediction of long-range contacts (i.e., contacts between residues separated by at least 24 residues along sequence) in target proteins that cannot be easily modeled by homology. Although there is considerable activity in the field, the current analysis reports no discernable progress since CASP8.

Prediction of protein long-range contacts using an ensemble of genetic algorithm classifiers with sequence profile centers

BackgroundPrediction of long-range inter-residue contacts is an important topic in bioinformatics research. It is helpful for determining protein structures, understanding protein foldings, and therefore advancing the annotation of protein functions.ResultsIn this paper, we propose a novel ensemble of genetic algorithm classifiers (GaCs) to address the long-range contact prediction problem. Our method is based on the key idea called sequence profile centers (SPCs). Each SPC is the average sequence profiles of residue pairs belonging to the same contact class or non-contact class. GaCs train on multiple but different pairs of long-range contact data (positive data) and long-range non-contact data (negative data). The negative data sets, having roughly the same sizes as the positive ones, are constructed by random sampling over the original imbalanced negative data. As a result, about 21.5% long-range contacts are correctly predicted. We also found that the ensemble of GaCs indeed makes an accuracy improvement by around 5.6% over the single GaC.ConclusionsClassifiers with the use of sequence profile centers may advance the long-range contact prediction. In line with this approach, key structural features in proteins would be determined with high efficiency and accuracy.

A comprehensive assessment of sequence-based and template-based methods for protein contact prediction

Pair-wise residue-residue contacts in proteins can be predicted from both threading templates and sequence-based machine learning. However, most structure modeling approaches only use the template-based contact predictions in guiding the simulations; this is partly because the sequence-based contact predictions are usually considered to be less accurate than that by threading. With the rapid progress in sequence databases and machine-learning techniques, it is necessary to have a detailed and comprehensive assessment of the contact-prediction methods in different template conditions. We develop two methods for protein-contact predictions: SVM-SEQ is a sequence-based machine learning approach which trains a variety of sequence-derived features on contact maps; SVM-LOMETS collects consensus contact predictions from multiple threading templates. We test both methods on the same set of 554 proteins which are categorized into 'Easy', 'Medium', 'Hard' and 'Very Hard' targets based on the evolutionary and structural distance between templates and targets. For the Easy and Medium targets, SVM-LOMETS obviously outperforms SVM-SEQ; but for the Hard and Very Hard targets, the accuracy of the SVM-SEQ predictions is higher than that of SVM-LOMETS by 12-25%. If we combine the SVM-SEQ and SVM-LOMETS predictions together, the total number of correctly predicted contacts in the Hard proteins will increase by more than 60% (or 70% for the long-range contact with a sequence separation > or =24), compared with SVM-LOMETS alone. The advantage of SVM-SEQ is also shown in the CASP7 free modeling targets where the SVM-SEQ is around four times more accurate than SVM-LOMETS in the long-range contact prediction. These data demonstrate that the state-of-the-art sequence-based contact prediction has reached a level which may be helpful in assisting tertiary structure modeling for the targets which do not have close structure templates. The maximum yield should be obtained by the combination of both sequence- and template-based predictions.

Protein structure prediction: concepts and applications

Foreword. Preface. Acknowledgments. Introduction. 1 Sequence, Function, and Structure Relationships. 1.1 Introduction. 1.2 Protein Structure. 1.3 The Properties of Amino Acids. 1.4 Experimental Determination of Protein Structures. 1.5 The PDB Protein Structure Data Archive. 1.6 Classification of Protein Structures. 1.7 The Protein-folding Problem. 1.8 Inference of Function from Structure. 1.9 The Evolution of Protein Function. 1.10 The Evolution of Protein Structure. 1.11 Relationship Between Evolution of Sequence and Evolution of Structure. 2 Reliability of Methods for Prediction of Protein Structure. 2.1 Introduction. 2.2 Prediction of Secondary Structure. 2.3 Prediction of Tertiary Structure. 2.4 Benchmarking a Prediction Method. 2.5 Blind Automatic Assessments. 2.6 The CASP Experiments. 3 Ab-initio Methods for Prediction of Protein Structures. 3.1 The Energy of a Protein Configuration. 3.2 Interactions and Energies. 3.3 Covalent Interactions. 3.4 Electrostatic Interactions. 3.5 Potential-energy Functions. 3.6 Statistical-mechanics Potentials. 3.7 Energy Minimization. 3.8 Molecular Dynamics. 3.9 Other Search Methods: Monte Carlo and Genetic Algorithms. 3.10 Effectiveness of Ab-initio Methods for Folding a Protein. 4 Evolutionary-based Methods for Predicting Protein Structure: Comparative Modeling. 4.1 Introduction. 4.2 Theoretical Basis of Comparative Modeling. 4.3 Detection of Evolutionary Relationships from Sequences. 4.4 The Needleman and Wunsch Algorithm. 4.5 Substitution Matrices. 4.6 Template(s) Identification Part I. 4.7 The Problem of Domains. 4.8 Alignment. 4.9 Template(s) Identification Part II. 4.10 Building the Main Chain of the Core. 4.11 Building Structurally Divergent Regions. 4.12 A Special Case: Immunoglobulins. 4.13 Side-chains. 4.14 Model Optimization. 4.15 Other Approaches. 4.16 Effectiveness of Comparative Modeling Methods. 5 Sequence-Structure Fitness Identification: Fold-recognition Methods. 5.1 The Theoretical Basis of Fold-recognition. 5.2 Profile-based Methods for Fold-recognition. 5.3 Threading Methods. 5.4 Profile-Profile Methods. 5.5 Construction and Optimization of the Model. 6 Methods Used to Predict New Folds: Fragment-based Methods. 6.1 Introduction. 6.2 Fragment-based Methods. 6.3 Splitting the Sequence into Fragments and Selecting Fragments from the Database. 6.4 Generation of Structures. 7 Low-dimensionality Prediction: Secondary Structure and Contact Prediction. 7.1 Introduction. 7.2 A Short History of Secondary structure Prediction Methods. 7.3 Automatic learning Methods. 7.3.1 Artificial Neural Networks. 7.3.2 Support Vector Machines. 7.4 Secondary structure Prediction Methods Based on Automatic Learning Techniques. 7.5 Prediction of Long-range Contacts. 8 Membrane Proteins. 8.1 Introduction. 8.2 Prediction of the Secondary Structure of Membrane Proteins. 8.3 The Hydrophobic Moment. 8.4 Prediction of the Topology of Membrane Proteins. 9 Applications and Examples. 9.1 Introduction. 9.2 Early Attempts. 9.3 The HIV Protease. 9.4 Leptin and Obesity. 9.5 The Envelope Glycoprotein of the Hepatitis C Virus. 9.6 HCV Protease. 9.7 Cyclic Nucleotide Gated Channels. 9.8 The Effectiveness of Models of Proteins in Drug Discovery. 9.9 The Effectiveness of Models of Proteins in X-ray Structure Solution. Conclusions. Glossary. Index.

PROFcon: novel prediction of long-range contacts

Despite the continuing advance in the experimental determination of protein structures, the gap between the number of known protein sequences and structures continues to increase. Prediction methods can bridge this sequence-structure gap only partially. Better predictions of non-local contacts between residues could improve comparative modeling, fold recognition and could assist in the experimental structure determination. Here, we introduced PROFcon, a novel contact prediction method that combines information from alignments, from predictions of secondary structure and solvent accessibility, from the region between two residues and from the average properties of the entire protein. In contrast to some other methods, PROFcon predicted short and long proteins at similar levels of accuracy. As expected, PROFcon was clearly less accurate when tested on sparse evolutionary profiles, that is, on families with few homologs. Prediction accuracy was highest for proteins belonging to the SCOP alpha/beta class. PROFcon compared favorably with state-of-the-art prediction methods at the CASP6 meeting. While the performance may still be perceived as low, our method clearly pushed the mark higher. Furthermore, predictions are already accurate enough to seed predictions of global features of protein structure.