Affinity Prediction Research Articles

AffinityVAE: A multi-objective model for protein-ligand affinity prediction and drug design

In the prediction of protein-ligand affinity, the traditional methods require a large amount of computing resources, and have certain limitations in predicting and simulating the structural changes. Although employing data-driven approaches can yield favorable outcomes in deep learning, it entails a lack of interpretability. Some methods may require additional structural information or domain knowledge to support the interpretation, which may limit their applicability. This paper proposes an affinity variational autoencoder (AffinityVAE) using interaction feature mapping and a variational autoencoder, which consists of a multi-objective model capable of end-to-end affinity prediction and drug discovery. In this study, the limitations of affinity prediction in terms of interpretability are tackled by proposing the concept of a protein-ligand interaction feature map. This increases the diversity and quantity of protein-ligand binding data by designing an adaptive autoencoder of target chemical properties to generate new ligands similar to known ligands and adding them to the original training set. AffinityVAE is then retrained using this extended training set to further validate the protein-ligand binding affinity prediction. Comparisons were conducted between the AffinityVAE and recent methods to demonstrate the high efficiency of the proposed model. The experimental results show that AffinityVAE has very high prediction performance, and it has the potential to enhance the diversity and the amount of protein-ligand binding data, which promotes the drug development.

ResBiGAAT: Residual Bi-GRU with attention for protein-ligand binding affinity prediction

Protein-ligand interaction plays a crucial role in drug discovery, facilitating efficient drug development and enabling drug repurposing. Several computational algorithms, such as Graph Neural Networks and Convolutional Neural Networks, have been proposed to predict the binding affinity using the three-dimensional structure of ligands and proteins. However, there are limitations due to the need for experimental characterization of the three-dimensional structure of protein sequences, which is still lacking for some proteins. Moreover, these models often suffer from unnecessary complexity, resulting in extraneous computations. This study presents ResBiGAAT, a novel deep learning model that combines a deep Residual Bidirectional Gated Recurrent Unit with two-sided self-attention mechanisms. ResBiGAAT leverages protein and ligand sequence-level features and their physicochemical properties to efficiently predict protein-ligand binding affinity. Through rigorous evaluation using 5-fold cross-validation, we demonstrate the performance of our proposed approach. The model exhibits competitive performance on an external dataset, highlighting its generalizability. Our publicly available web interface, located at resbigaat.streamlit.app, allows users to conveniently input protein and ligand sequences to estimate binding affinity.

A Multi-perspective Model for Protein-Ligand-Binding Affinity Prediction.

Gathering information from multi-perspective graphs is an essential issue for many applications especially for protein-ligand-binding affinity prediction. Most of traditional approaches obtained such information individually with low interpretability. In this paper, we harness the rich information from multi-perspective graphs with a general model, which abstractly represents protein-ligand complexes with better interpretability while achieving excellent predictive performance. In addition, we specially analyze the protein-ligand-binding affinity problem, taking into account the heterogeneity of proteins and ligands. Experimental evaluations demonstrate the effectiveness of our data representation strategy on public datasets by fusing information from different perspectives. All codes are available in the https://github.com/Jthy-af/HaPPy .

Optimization of drug-target affinity prediction methods through feature processing schemes.

Numerous high-accuracy drug-target affinity (DTA) prediction models, whose performance is heavily reliant on the drug and target feature information, are developed at the expense of complexity and interpretability. Feature extraction and optimization constitute a critical step that significantly influences the enhancement of model performance, robustness, and interpretability. Many existing studies aim to comprehensively characterize drugs and targets by extracting features from multiple perspectives; however, this approach has drawbacks: (i) an abundance of redundant or noisy features; and (ii) the feature sets often suffer from high dimensionality. In this study, to obtain a model with high accuracy and strong interpretability, we utilize various traditional and cutting-edge feature selection and dimensionality reduction techniques to process self-associated features and adjacent associated features. These optimized features are then fed into learning to rank to achieve efficient DTA prediction. Extensive experimental results on two commonly used datasets indicate that, among various feature optimization methods, the regression tree-based feature selection method is most beneficial for constructing models with good performance and strong robustness. Then, by utilizing Shapley Additive Explanations values and the incremental feature selection approach, we obtain that the high-quality feature subset consists of the top 150D features and the top 20D features have a breakthrough impact on the DTA prediction. In conclusion, our study thoroughly validates the importance of feature optimization in DTA prediction and serves as inspiration for constructing high-performance and high-interpretable models. https://github.com/RUXIAOQING964914140/FS_DTA.

Experimental Uncertainty in Training Data for Protein-Ligand Binding Affinity Prediction Models

The accuracy of machine learning models for protein-ligand binding affinity prediction depends on the quality of the experimental data they are trained on. Most of these models are trained and tested on different subsets of the PDBbind database, which is the main source of protein-ligand complexes with annotated binding affinity in the public domain. However, estimating its experimental uncertainty is not straightforward because just a few protein-ligand complexes have more than one measurement associated. In this work, we analyze bioactivity data from ChEMBL to estimate the experimental uncertainty associated with the three binding affinity measures included in the PDBbind (Ki, Kd, and IC50), as well as the effect of combining them. The experimental uncertainty of combining these three affinity measures was characterized by a mean absolute error of 0.78 logarithmic units, a root mean square error of 1.04 and a Pearson correlation coefficient of 0.76. These estimations were contrasted with the performances obtained by state-of-the-art machine learning models for binding affinity prediction, showing that these models tend to be overoptimistic when evaluated on the core set from PDBbind.

Computing the relative binding affinity of ligands based on a pairwise binding comparison network

Structure-based lead optimization is an open challenge in drug discovery, which is still largely driven by hypotheses and depends on the experience of medicinal chemists. Here we propose a pairwise binding comparison network (PBCNet) based on a physics-informed graph attention mechanism, specifically tailored for ranking the relative binding affinity among congeneric ligands. Benchmarking on two held-out sets (provided by Schrödinger and Merck) containing over 460 ligands and 16 targets, PBCNet demonstrated substantial advantages in terms of both prediction accuracy and computational efficiency. Equipped with a fine-tuning operation, the performance of PBCNet reaches that of Schrödinger’s FEP+, which is much more computationally intensive and requires substantial expert intervention. A further simulation-based experiment showed that active learning-optimized PBCNet may accelerate lead optimization campaigns by 473%. Finally, for the convenience of users, a web service for PBCNet is established to facilitate complex relative binding affinity prediction through an easy-to-operate graphical interface.

TeM-DTBA: time-efficient drug target binding affinity prediction using multiple modalities with Lasso feature selection.

Drug discovery, especially virtual screening and drug repositioning, can be accelerated through deeper understanding and prediction of Drug Target Interactions (DTIs). The advancement of deep learning as well as the time and financial costs associated with conventional wet-lab experiments have made computational methods for DTI prediction more popular. However, the majority of these computational methods handle the DTI problem as a binary classification task, ignoring the quantitative binding affinity that determines the drug efficacy to their target proteins. Moreover, computational space as well as execution time of the model is often ignored over accuracy. To address these challenges, we introduce a novel method, called Time-efficient Multimodal Drug Target Binding Affinity (TeM-DTBA), which predicts the binding affinity between drugs and targets by fusing different modalities based on compound structures and target sequences. We employ the Lasso feature selection method, which lowers the dimensionality of feature vectors and speeds up the proposed model training time by more than 50%. The results from two benchmark datasets demonstrate that our method outperforms state-of-the-art methods in terms of performance. The mean squared errors of 18.8% and 23.19%, achieved on the KIBA and Davis datasets, respectively, suggest that our method is more accurate in predicting drug-target binding affinity.

Improving the accuracy of the FMO binding affinity prediction of ligand-receptor complexes containing metals.

Polarization and charge transfer strongly characterize the ligand-receptor interaction when metal atoms are present, as for the Au(I)-biscarbene/DNA G-quadruplex complexes. In a previous work (J Comput Aided Mol Des2022, 36, 851-866) we used the ab initio FMO2 method at the RI-MP2/6-31G* level of theory with the PCM [1] solvation approach to calculate the binding energy (ΔEFMO) of two Au(I)-biscarbene derivatives, [Au(9-methylcaffein-8-ylidene)2]+ and [Au(1,3-dimethylbenzimidazole-2-ylidene)2]+, able to interact with DNA G-quadruplex motif. We found that ΔEFMO and ligand-receptor pair interaction energies (EINT) show very large negative values making the direct comparison with experimental data difficult and related this issue to the overestimation of the embedded charge transfer energy between fragments containing metal atoms. In this work, to improve the accuracy of the FMO method for predicting the binding affinity of metal-based ligands interacting with DNA G-quadruplex (Gq), we assess the effect of the following computational features: (i) the electron correlation, considering the Hartree-Fock (HF) and a post-HF method, namely RI-MP2; (ii) the two (FMO2) and three-body (FMO3) approaches; (iii) the basis set size (polarization functions and double-ζ vs. triple-ζ) and (iv) the embedding electrostatic potential (ESP). Moreover, the partial screening method was systematically adopted to simulate the solvent screening effect for each calculation. We found that the use of the ESP computed using the screened point charges for all atoms (ESP-SPTC) has a critical impact on the accuracy of both ΔEFMO and EINT, eliminating the overestimation of charge transfer energy and leading to energy values with magnitude comparable with typical experimental binding energies. With this computational approach, EINT values describe the binding efficiency of metal-based binders to DNA Gq more accurately than ΔEFMO. Therefore, to study the binding process of metal containing systems with the FMO method, the adoption of partial screening solvent method combined with ESP-SPCT should be considered. This computational protocol is suggested for FMO calculations on biological systems containing metals, especially when the adoption of the default ESP treatment leads to questionable results.

Breaking the barriers of data scarcity in drug-target affinity prediction.

Accurate prediction of drug-target affinity (DTA) is of vital importance in early-stage drug discovery, facilitating the identification of drugs that can effectively interact with specific targets and regulate their activities. While wet experiments remain the most reliable method, they are time-consuming and resource-intensive, resulting in limited data availability that poses challenges for deep learning approaches. Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue. To overcome this challenge, we present the Semi-Supervised Multi-task training (SSM) framework for DTA prediction, which incorporates three simple yet highly effective strategies: (1) A multi-task training approach that combines DTA prediction with masked language modeling using paired drug-target data. (2) A semi-supervised training method that leverages large-scale unpaired molecules and proteins to enhance drug and target representations. This approach differs from previous methods that only employed molecules or proteins in pre-training. (3) The integration of a lightweight cross-attention module to improve the interaction between drugs and targets, further enhancing prediction accuracy. Through extensive experiments on benchmark datasets such as BindingDB, DAVIS and KIBA, we demonstrate the superior performance of our framework. Additionally, we conduct case studies on specific drug-target binding activities, virtual screening experiments, drug feature visualizations and real-world applications, all of which showcase the significant potential of our work. In conclusion, our proposed SSM-DTA framework addresses the data limitation challenge in DTA prediction and yields promising results, paving the way for more efficient and accurate drug discovery processes.

Enhancing reinforcement learning for de novo molecular design applying self-attention mechanisms.

The drug discovery process can be significantly improved by applying deep reinforcement learning (RL) methods that learn to generate compounds with desired pharmacological properties. Nevertheless, RL-based methods typically condense the evaluation of sampled compounds into a single scalar value, making it difficult for the generative agent to learn the optimal policy. This work combines self-attention mechanisms and RL to generate promising molecules. The idea is to evaluate the relative significance of each atom and functional group in their interaction with the target, and to utilize this information for optimizing the Generator. Therefore, the framework for de novo drug design is composed of a Generator that samples new compounds combined with a Transformer-encoder and a biological affinity Predictor that evaluate the generated structures. Moreover, it takes the advantage of the knowledge encapsulated in the Transformer's attention weights to evaluate each token individually. We compared the performance of two output prediction strategies for the Transformer: standard and masked language model (MLM). The results show that the MLM Transformer is more effective in optimizing the Generator compared with the state-of-the-art works. Additionally, the evaluation models identified the most important regions of each molecule for the biological interaction with the target. As a case study, we generated synthesizable hit compounds that can be putative inhibitors of the enzyme ubiquitin-specific protein 7 (USP7).

GPCNDTA: Prediction of drug-target binding affinity through cross-attention networks augmented with graph features and pharmacophores

Drug-target affinity prediction is a challenging task in drug discovery. The latest computational models have limitations in mining edge information in molecule graphs, accessing to knowledge in pharmacophores, integrating multimodal data of the same biomolecule and realizing effective interactions between two different biomolecules. To solve these problems, we proposed a method called Graph features and Pharmacophores augmented Cross-attention Networks based Drug-Target binding Affinity prediction (GPCNDTA). First, we utilized the GNN module, the linear projection unit and self-attention layer to correspondingly extract features of drugs and proteins. Second, we devised intramolecular and intermolecular cross-attention to respectively fuse and interact features of drugs and proteins. Finally, the linear projection unit was applied to gain final features of drugs and proteins, and the Multi-Layer Perceptron was employed to predict drug-target binding affinity. Three major innovations of GPCNDTA are as follows: (i) developing the residual CensNet and the residual EW-GCN to correspondingly extract features of drug and protein graphs, (ii) regarding pharmacophores as a new type of priors to heighten drug-target affinity prediction performance, and (iii) devising intramolecular and intermolecular cross-attention, in which the intramolecular cross-attention realizes the effective fusion of different modal data related to the same biomolecule, and the intermolecular cross-attention fulfills the information interaction between two different biomolecules in attention space. The test results on five benchmark datasets imply that GPCNDTA achieves the best performance compared with state-of-the-art computational models. Besides, relying on ablation experiments, we proved effectiveness of GNN modules, pharmacophores and two cross-attention strategies in improving the prediction accuracy, stability and reliability of GPCNDA. In case studies, we applied GPCNDTA to predict binding affinities between 3C-like proteinase and 185 drugs, and observed that most binding affinities predicted by GPCNDTA are close to corresponding experimental measurements.

Drug-target binding affinity prediction using message passing neural network and self supervised learning

BackgroundDrug-target binding affinity (DTA) prediction is important for the rapid development of drug discovery. Compared to traditional methods, deep learning methods provide a new way for DTA prediction to achieve good performance without much knowledge of the biochemical background. However, there are still room for improvement in DTA prediction: (1) only focusing on the information of the atom leads to an incomplete representation of the molecular graph; (2) the self-supervised learning method could be introduced for protein representation.ResultsIn this paper, a DTA prediction model using the deep learning method is proposed, which uses an undirected-CMPNN for molecular embedding and combines CPCProt and MLM models for protein embedding. An attention mechanism is introduced to discover the important part of the protein sequence. The proposed method is evaluated on the datasets Ki and Davis, and the model outperformed other deep learning methods.ConclusionsThe proposed model improves the performance of the DTA prediction, which provides a novel strategy for deep learning-based virtual screening methods.

SubMDTA: drug target affinity prediction based on substructure extraction and multi-scale features

BackgroundDrug–target affinity (DTA) prediction is a critical step in the field of drug discovery. In recent years, deep learning-based methods have emerged for DTA prediction. In order to solve the problem of fusion of substructure information of drug molecular graphs and utilize multi-scale information of protein, a self-supervised pre-training model based on substructure extraction and multi-scale features is proposed in this paper.ResultsFor drug molecules, the model obtains substructure information through the method of probability matrix, and the contrastive learning method is implemented on the graph-level representation and subgraph-level representation to pre-train the graph encoder for downstream tasks. For targets, a BiLSTM method that integrates multi-scale features is used to capture long-distance relationships in the amino acid sequence. The experimental results showed that our model achieved better performance for DTA prediction.ConclusionsThe proposed model improves the performance of the DTA prediction, which provides a novel strategy based on substructure extraction and multi-scale features.

Chimeric RNAs reveal putative neoantigen peptides for developing tumor vaccines for breast cancer.

We present here a strategy to identify immunogenic neoantigen candidates from unique amino acid sequences at the junctions of fusion proteins which can serve as targets in the development of tumor vaccines for the treatment of breastcancer. We mined the sequence reads of breast tumor tissue that are usually discarded as discordant paired-end reads and discovered cancer specific fusion transcripts using tissue from cancer free controls as reference. Binding affinity predictions of novel peptide sequences crossing the fusion junction were analyzed by the MHC Class I binding predictor, MHCnuggets. CD8+ T cell responses against the 15 peptides were assessed through in vitro Enzyme Linked Immunospot (ELISpot). We uncovered 20 novel fusion transcripts from 75 breast tumors of 3 subtypes: TNBC, HER2+, and HR+. Of these, the NSFP1-LRRC37A2 fusion transcript was selected for further study. The 3833 bp chimeric RNA predicted by the consensus fusion junction sequence is consistent with a read-through transcription of the 5'-gene NSFP1-Pseudo gene NSFP1 (NSFtruncation at exon 12/13) followed by trans-splicing to connect withLRRC37A2 located immediately 3' through exon 1/2. A total of 15 different 8-mer neoantigen peptides discovered from the NSFP1 and LRRC37A2 truncations were predicted to bind to a total of 35 unique MHC class I alleles with a binding affinity of IC50<500nM.); 1 of which elicited a robust immune response. Our data provides a framework to identify immunogenic neoantigen candidates from fusion transcripts and suggests a potential vaccine strategy to target the immunogenic neopeptides in patients with tumors carrying the NSFP1-LRRC37A2 fusion.

DataDTA: a multi-feature and dual-interaction aggregation framework for drug-target binding affinity prediction.

Accurate prediction of drug-target binding affinity (DTA) is crucial for drug discovery. The increase in the publication of large-scale DTA datasets enables the development of various computational methods for DTA prediction. Numerous deep learning-based methods have been proposed to predict affinities, some of which only utilize original sequence information or complex structures, but the effective combination of various information and protein-binding pockets have not been fully mined. Therefore, a new method that integrates available key information is urgently needed to predict DTA and accelerate the drug discovery process. In this study, we propose a novel deep learning-based predictor termed DataDTA to estimate the affinities of drug-target pairs. DataDTA utilizes descriptors of predicted pockets and sequences of proteins, as well as low-dimensional molecular features and SMILES strings of compounds as inputs. Specifically, the pockets were predicted from the three-dimensional structure of proteins and their descriptors were extracted as the partial input features for DTA prediction. The molecular representation of compounds based on algebraic graph features was collected to supplement the input information of targets. Furthermore, to ensure effective learning of multiscale interaction features, a dual-interaction aggregation neural network strategy was developed. DataDTA was compared with state-of-the-art methods on different datasets, and the results showed that DataDTA is a reliable prediction tool for affinities estimation. Specifically, the CI of DataDTA is 0.806 and the R value is 0.814 on the test dataset, which is higher than other methods. The codes of DataDTA are available at https://github.com/YanZhu06/DataDTA. Supplementary data are available at Bioinformatics online.

DeepMHCI: an anchor position-aware deep interaction model for accurate MHC-I peptide binding affinity prediction.

Computationally predicting major histocompatibility complex class I (MHC-I) peptide binding affinity is an important problem in immunological bioinformatics, which is also crucial for the identification of neoantigens for personalized therapeutic cancer vaccines. Recent cutting-edge deep learning-based methods for this problem cannot achieve satisfactory performance, especially for non-9-mer peptides. This is because such methods generate the input by simply concatenating the two given sequences: a peptide and (the pseudo sequence of) an MHC class I molecule, which cannot precisely capture the anchor positions of the MHC binding motif for the peptides with variable lengths. We thus developed an anchor position-aware and high-performance deep model, DeepMHCI, with a position-wise gated layer and a residual binding interaction convolution layer. This allows the model to control the information flow in peptides to be aware of anchor positions and model the interactions between peptides and the MHC pseudo (binding) sequence directly with multiple convolutional kernels. The performance of DeepMHCI has been thoroughly validated by extensive experiments on four benchmark datasets under various settings, such as 5-fold cross-validation, validation with the independent testing set, external HPV vaccine identification, and external CD8+ epitope identification. Experimental results with visualization of binding motifs demonstrate that DeepMHCI outperformed all competing methods, especially on non-9-mer peptides binding prediction. DeepMHCI is publicly available at https://github.com/ZhuLab-Fudan/DeepMHCI.

An approach of computer-aided drug design (CADD) tools for in silico assessment of various inhibitors of lanosterol-14α demethylase

Structure-based virtual screening is a modern CADD tool in drug discovery that identifies active inhibitors based upon the prediction of binding affinity as well as molecular interactions of ligand molecules (or inhibitors) with their corresponding target proteins or enzymes. This advanced technique is used to identify potential candidates from natural origin possessing antifungal potentials through inhibition of cell wall synthesis in deleterious species Candida albicans, Saccharomyces cerevisiae, etc. 3D confirmations for the chemical structures of identified phytoconstituents were prepared through ChemDraw 16.0 program for prediction of activity (way2drug), mechanism (PyRx), physicochemical & pharmacokinetics (SWISS-ADME) and toxicological (PROTOX-2) probabilities through in-silico approaches. BIOVIA Discovery studio visualizer was used to generate a 3D and 2D image of the most active 18 molecules among 110 identified ligands. The research concludes with some most probable ligands of pteridophytic origin that interact with cell wall synthesis in normal and resistant fungal species. The results suggested that out of 18 active molecules, Mol-9, Mol-13, Mol-4 and Mol-16 showed highest docking score on lanosterol-14-alpha demethylase obtained from different sources. Hence, these molecules may possess good bioactivity against fungal growth and can be further developed as potent antifungal drug. However further research is required to explore individual molecule for their potency and efficacy and for exploring further molecular pathways against fungal infections.

SGNet: Sequence-Based Convolution and Ligand Graph Network for Protein Binding Affinity Prediction.

Protein-ligand binding can play an important role in many fields. It is of great importance to accurately predict the binding affinity between molecules by computational methods. Most computational binding affinity methods require molecular structures. However, there are still a large number of protein molecules with known amino acid sequences whose structures have not yet been solved. To address this issue, this paper proposes a sequence-based convolution and ligand graph network, called SGNet, to fuse the molecular graph information and the amino acid sequence information. This method integrates Conjoint Triad (CT) encoding of amino acid sequence and one-dimensional convolutional neural network module to extract protein molecules, develops graph attention network to extract molecular features of ligand, and then fuses the two feature sets to predict the binding affinity between molecules from the fully connected layer. As a result, SGNet achieves good prediction performance on both KIKD and IC50 data sets, with prediction error RMSEs of 1.287 and 1.58, and correlation Pearson Rs of 0.687 and 0.592, respectively. Comparative experimental results under the same conditions showed that SGNet outperformed Kdeep and GraphDTA in predicting binding affinities between protein-ligand molecules.

GINCM-DTA: A graph isomorphic network with protein contact map representation for potential use against COVID-19 and Omicron subvariants BQ.1, BQ.1.1, XBB.1.5, XBB.1.16

The COVID-19 pandemic’s profound impact on global health and the economy underscores the need for new computational strategies to rapidly identify potential antiviral drugs against emerging and re-emerging infectious diseases. AI-based drug repurposing methods have the potential to significantly shorten the drug discovery process, rendering them a crucial and efficacious approach for screening anti-SARS-CoV-2 compounds. In this paper, we proposed GINCM-DTA, an AI-based graph isomorphism network with protein contact map representation for drug–target affinity (DTA) prediction, and presented the COVID-DTA dataset for SARS-CoV-2-specific drug repurposing. Pre-training GINCM-DTA on large-scale Davis and KIBA datasets achieved state-of-the-art performance, while fine-tuning on the COVID-DTA dataset using transfer learning resulted in a mean square error of 0.026 and concordance index of 0.967. Utilizing the fine-tuned GINCM-DTA model, we predicted DTAs for drugs against SARS-CoV-2 main targets, focusing on key conserved proteins and unmutated spike proteins for virtual screening. We integrated GINCM-DTA predictions with molecular docking simulations to identify five candidates. In vitro assays revealed Hydralazine (5.737 nM) as a potential inhibitor of ACE2 and TMPRSS2, effectively blocking SARS-CoV-2 infection. Moreover, the results highlight the potential of Hydralazine as a broad-spectrum drug candidate, demonstrating its efficacy in obstructing the binding between ACE2 and spike proteins of multiple Omicron variants, including the currently dominant strains BQ.1, BQ.1.1, XBB.1.5, and XBB.1.16. Our method offers a novel computational strategy for COVID-19 antiviral screening and prepares for potential future epidemics incited by emerging viruses.

Integration of pre-trained protein language models into geometric deep learning networks

Geometric deep learning has recently achieved great success in non-Euclidean domains, and learning on 3D structures of large biomolecules is emerging as a distinct research area. However, its efficacy is largely constrained due to the limited quantity of structural data. Meanwhile, protein language models trained on substantial 1D sequences have shown burgeoning capabilities with scale in a broad range of applications. Several preceding studies consider combining these different protein modalities to promote the representation power of geometric neural networks but fail to present a comprehensive understanding of their benefits. In this work, we integrate the knowledge learned by well-trained protein language models into several state-of-the-art geometric networks and evaluate a variety of protein representation learning benchmarks, including protein-protein interface prediction, model quality assessment, protein-protein rigid-body docking, and binding affinity prediction. Our findings show an overall improvement of 20% over baselines. Strong evidence indicates that the incorporation of protein language models’ knowledge enhances geometric networks’ capacity by a significant margin and can be generalized to complex tasks.