PDBbind Research Articles

Evaluation of the performance of four molecular docking programs on a diverse set of protein‐ligand complexes

Many molecular docking programs are available nowadays, and thus it is of great practical value to evaluate and compare their performance. We have conducted an extensive evaluation of four popular commercial molecular docking programs, including Glide, GOLD, LigandFit, and Surflex. Our test set consists of 195 protein-ligand complexes with high-resolution crystal structures (resolution <or=2.5 A) and reliable binding data [dissociation constant (K(d)) or inhibition constant (K(i))], which are selected from the PDBbind database with an emphasis on diversity. The top-ranked solutions produced by these programs are compared to the native ligand binding poses observed in crystal structures. Glide and GOLD demonstrate better accuracy than the other two on the entire test set. Their results are also less sensitive to the starting structures for docking. Comparison of the results produced by these programs at three different computation levels reveal that their accuracy are not always proportional to CPU cost as one may expect. The binding scores of the top-ranked solutions produced by these programs are in low to moderate correlations with experimentally measured binding data. Further analyses on the outcomes of these programs on three suites of subsets of protein-ligand complexes indicate that these programs are less capable to handle really flexible ligands and relatively flat binding sites, and they have different preferences to hydrophilic/hydrophobic binding sites. Our evaluation can help other researchers to make reasonable choices among available molecular docking programs. It is also valuable for program developers to improve their methods further.

Inclusion of Solvation and Entropy in the Knowledge-Based Scoring Function for Protein−Ligand Interactions

The effects of solvation and entropy play a critical role in determining the binding free energy in protein-ligand interactions. Despite the good balance between speed and accuracy, no current knowledge-based scoring functions account for the effects of solvation and configurational entropy explicitly due to the difficulty in deriving the corresponding pair potentials and the resulting double counting problem. In the present work, we have included the solvation effect and configurational entropy in the knowledge-based scoring function by an iterative method. The newly developed scoring function has yielded a success rate of 91% in identifying near-native binding modes with Wang et al.'s benchmark of 100 diverse protein-ligand complexes. The results have been compared with the results of 15 other scoring functions for validation purpose. In binding affinity prediction, our scoring function has yielded a correlation of R(2) = 0.76 between the predicted binding scores and the experimentally measured binding affinities on the PMF validation sets of 77 diverse complexes. The results have been compared with R(2) of four other well-known knowledge-based scoring functions. Finally, our scoring function was also validated on the large PDBbind database of 1299 protein-ligand complexes and yielded a correlation coefficient of 0.474. The present computational model can be applied to other scoring functions to account for solvation and entropic effects.

A Statistical Survey on the Binding Constants of Covalently Bound Protein-Ligand Complexes.

We have conducted a statistical survey to compare the binding constants of covalently and noncovalently bound protein-ligand complexes as two groups. In our study, a total of 1602 complexes formed between various types of proteins and small-molecule ligands were selected from the PDBbind database (version 2008), all of which had high-resolution three-dimensional structures and reliable experimentally measured binding constants. These complexes were further classified as 79 covalent complexes, 131 Zn-containing complexes, and 1392 noncovalent complexes. Covalent complexes formed through reversible mechanisms are found to be associated with higher binding constants than noncovalent complexes. Two-sample T-test indicates that the difference is statistically significant. The advantage, however, is only modest (<20 folds). The same trend is also observed on a set of covalent and noncovalent complexes formed by thrombin. Our results indicate that reversible covalent bonding formed between protein and ligand will not automatically lead to a much tighter binding in general. Thus, our survey does not provide any supporting evidence for Houk's hypothesis which states that covalent bonding formed between enzyme and transition state accounts for the extraordinary proficiency of enzymes.

Comparative Assessment of Scoring Functions on a Diverse Test Set

Scoring functions are widely applied to the evaluation of protein-ligand binding in structure-based drug design. We have conducted a comparative assessment of 16 popular scoring functions implemented in main-stream commercial software or released by academic research groups. A set of 195 diverse protein-ligand complexes with high-resolution crystal structures and reliable binding constants were selected through a systematic nonredundant sampling of the PDBbind database and used as the primary test set in our study. All scoring functions were evaluated in three aspects, that is, "docking power", "ranking power", and "scoring power", and all evaluations were independent from the context of molecular docking or virtual screening. As for "docking power", six scoring functions, including GOLD::ASP, DS::PLP1, DrugScore(PDB), GlideScore-SP, DS::LigScore, and GOLD::ChemScore, achieved success rates over 70% when the acceptance cutoff was root-mean-square deviation < 2.0 A. Combining these scoring functions into consensus scoring schemes improved the success rates to 80% or even higher. As for "ranking power" and "scoring power", the top four scoring functions on the primary test set were X-Score, DrugScore(CSD), DS::PLP, and SYBYL::ChemScore. They were able to correctly rank the protein-ligand complexes containing the same type of protein with success rates around 50%. Correlation coefficients between the experimental binding constants and the binding scores computed by these scoring functions ranged from 0.545 to 0.644. Besides the primary test set, each scoring function was also tested on four additional test sets, each consisting of a certain number of protein-ligand complexes containing one particular type of protein. Our study serves as an updated benchmark for evaluating the general performance of today's scoring functions. Our results indicate that no single scoring function consistently outperforms others in all three aspects. Thus, it is important in practice to choose the appropriate scoring functions for different purposes.

A novel method for protein‐ligand binding affinity prediction and the related descriptors exploration

In this study, a novel method was developed to predict the binding affinity of protein-ligand based on a comprehensive set of structurally diverse protein-ligand complexes (PLCs). The 1300 PLCs with binding affinity (493 complexes with K(d) and 807 complexes with K(i)) from the refined dataset of PDBbind Database (release 2007) were studied in the predictive model development. In this method, each complex was described using calculated descriptors from three blocks: protein sequence, ligand structure, and binding pocket. Thereafter, the PLCs data were rationally split into representative training and test sets by full consideration of the validation of the models. The molecular descriptors relevant to the binding affinity were selected using the ReliefF method combined with least squares support vector machines (LS-SVMs) modeling method based on the training data set. Two final optimized LS-SVMs models were developed using the selected descriptors to predict the binding affinities of K(d) and K(i). The correlation coefficients (R) of training set and test set for K(d) model were 0.890 and 0.833. The corresponding correlation coefficients for the K(i) model were 0.922 and 0.742, respectively. The prediction method proposed in this work can give better generalization ability than other recently published methods and can be used as an alternative fast filter in the virtual screening of large chemical database.

SFCscore: Scoring functions for affinity prediction of protein–ligand complexes

Empirical scoring functions to calculate binding affinities of protein-ligand complexes have been calibrated based on experimental structure and affinity data collected from public and industrial sources. Public data were taken from the AffinDB database, whereas access to industrial data was gained through the Scoring Function Consortium (SFC), a collaborative effort with various pharmaceutical companies and the Cambridge Crystallographic Data Center. More than 850 complexes were obtained by the data collection procedure and subsequently used to setup different training sets for the parameterization of new scoring functions. Over 60 different descriptors were evaluated for all complexes, including terms accounting for interactions with and among aromatic ring systems as well as many surface-dependent terms. After exploratory correlation and regression analyses, stepwise variable selection procedures and systematic searches, the most suitable descriptors were chosen as variables to calibrate regression functions by means of multiple linear regression or partial least squares analysis. Eight different functions are presented herein. Cross-validated r(2) (Q(2)) values of up to 0.72 and standard errors (s(PRESS)) generally below 1.15 pK(i) units suggest highly predictive functions. Extensive unbiased validation was carried out by testing the functions on large data sets from the PDBbind database as used by Wang et al. (J Chem Inf Comput Sci 2004;44:2114-2125) in a comparative analysis of other scoring functions. Superior performance of the SFCscore functions is observed in many cases, but the results also illustrate the need for further improvements.

A Multivariate Approach to Investigate Docking Parameters' Effects on Docking Performance

Increasingly powerful docking programs for analyzing and estimating the strength of protein-ligand interactions have been developed in recent decades, and they are now valuable tools in drug discovery. Software used to perform dockings relies on a number of parameters that affect various steps in the docking procedure. However, identifying the best choices of the settings for these parameters is often challenging. Therefore, the settings of the parameters are quite often left at their default values, even though scientists with long experience with a specific docking tool know that modifying certain parameters can improve the results. In the study presented here, we have used statistical experimental design and subsequent regression based on root-mean-square deviation values using partial least-square projections to latent structures (PLS) to scrutinize the effects of different parameters on the docking performance of two software packages: FRED and GOLD. Protein-ligand complexes with a high level of ligand diversity were selected from the PDBbind database for the study, using principal component analysis based on 1D and 2D descriptors, and space-filling design. The PLS models showed quantitative relationships between the docking parameters and the ability of the programs to reproduce the ligand crystallographic conformation. The PLS models also revealed which of the parameters and what parameter settings were important for the docking performance of the two programs. Furthermore, the variation in docking results obtained with specific parameter settings for different protein-ligand complexes in the diverse set examined indicates that there is great potential for optimizing the parameter settings for selected sets of proteins.

Hierarchical PLS Modeling for Predicting the Binding of a Comprehensive Set of Structurally Diverse Protein−Ligand Complexes

A new approach is presented for predicting ligand binding to proteins using hierarchical partial-least-squares regression to latent structures (Hi-PLS). Models were based on information from the 2002 release of the PDBbind database containing (after in-house refinement) high-resolution X-ray crystallography and binding affinity (Kd or Ki) data for 612 protein-ligand complexes. The complexes were characterized by four different descriptor blocks: three-dimensional (3D) structural descriptors of the proteins, protein-ligand interactions according to the Validate scoring function, binding site surface areas, and ligand 2D and 3D descriptors. These descriptor blocks were used in Hi-PLS models, generated using both linear and nonlinear terms, to relate the characterizations to pKd/i. The results show that each of the four descriptor blocks contributed to the model, and the predictions of pKd/i of the internal test set gave a root-mean-square error of prediction (RMSEP) of 1.65. The data were further divided according to the structural classification of the proteins, and Hi-PLS models were constructed for the resulting subclasses. The models for the four subclasses differed considerably in terms of both their ability to predict pKd/i (with RMSEPs ranging from 0.8 to 1.56) and the descriptor block that had the strongest influence. The models were validated with an external test set of 174 complexes from the 2003 release of the PDBbind database. The overall results show that the presented Hi-PLS methodology could facilitate the difficult task of predicting binding affinity.

Chemometric analysis of ligand receptor complementarity: identifying Complementary Ligands Based on Receptor Information (CoLiBRI).

We have developed a novel structure-based chemoinformatics approach to search for Complimentary Ligands Based on Receptor Information (CoLiBRI). CoLiBRI is based on the representation of both receptor binding sites and their respective ligands in a space of universal chemical descriptors. The binding site atoms involved in the interaction with ligands are identified by the means of a computational geometry technique known as Delaunay tessellation as applied to X-ray characterized ligand-receptor complexes. TAE/RECON multiple chemical descriptors are calculated independently for each ligand as well as for its active site atoms. The representation of both ligands and active sites using chemical descriptors allows the application of well-known chemometric techniques in order to correlate chemical similarities between active sites and their respective ligands. We have established a protocol to map patterns of nearest neighbor active site vectors in a multidimensional TAE/RECON space onto those of their complementary ligands and vice versa. This protocol affords the prediction of a virtual complementary ligand vector in the ligand chemical space from the position of a known active site vector. This prediction is followed by chemical similarity calculations between this virtual ligand vector and those calculated for molecules in a chemical database to identify real compounds most similar to the virtual ligand. Consequently, the knowledge of the receptor active site structure affords straightforward and efficient identification of its complementary ligands in large databases of chemical compounds using rapid chemical similarity searches. Conversely, starting from the ligand chemical structure, one may identify possible complementary receptor cavities as well. We have applied the CoLiBRI approach to a data set of 800 X-ray characterized ligand-receptor complexes in the PDBbind database. Using a k nearest neighbor (kNN) pattern recognition approach and variable selection, we have shown that knowledge of the active site structure affords identification of its complimentary ligand among the top 1% of a large chemical database in over 90% of all test active sites when a binding site of the same protein family was present in the training set. In the case where test receptors are highly dissimilar and not present among the receptor families in the training set, the prediction accuracy is decreased; however, CoLiBRI was still able to quickly eliminate 75% of the chemical database as improbable ligands. CoLiBRI affords rapid prefiltering of a large chemical database to eliminate compounds that have little chance of binding to a receptor active site.

PMF Scoring Revisited

Knowledge-based scoring functions have become accepted choices for fast scoring putative protein-ligand complexes according to their binding affinities. Since their introduction 5 years ago, the knowledge base of protein-ligand complexes has grown to the point were rederiving potentials of mean force becomes meaningful for statistical reasons. Revisiting potential of mean force (PMF) scoring (J. Med. Chem. 1999, 42, 791), we present an updated PMF04 scoring function that is based on 7152 protein-ligand complexes from the PDB. This constitutes an increase of about 10-fold compared to the knowledge base of the original PMF99 score (697 complexes). Because of the increased statistical basis of the PMF04 score, potentials for metal ions have been derived for the first time. In addition, potentials for halogens have reached statistical significance and are included also. Comparison of scoring accuracies between PMF99 and PMF04 shows an increased performance of the new score for many well-established test sets. Extending the testing of PMF scoring to the recently introduced PDBbind database containing the large number of 800 protein-ligand complexes illustrates the current limits of the approach.

The PDBbind Database: Methodologies and Updates

We have developed the PDBbind database to provide a comprehensive collection of binding affinities for the protein-ligand complexes in the Protein Data Bank (PDB). This paper gives a full description of the latest version, i.e., version 2003, which is an update to our recently reported work. Out of 23 790 entries in the PDB release No.107 (January 2004), 5897 entries were identified as protein-ligand complexes that meet our definition. Experimentally determined binding affinities (K(d), K(i), and IC(50)) for 1622 of these were retrieved from the references associated with these complexes. A total of 900 complexes were selected to form a "refined set", which is of particular value as a standard data set for docking and scoring studies. All of the final data, including binding affinity data, reference citations, and processed structural files, have been incorporated into the PDBbind database accessible on-line at http:// www.pdbbind.org/.

An Extensive Test of 14 Scoring Functions Using the PDBbind Refined Set of 800 Protein−Ligand Complexes

Fourteen popular scoring functions, i.e., X-Score, DrugScore, five scoring functions in the Sybyl software (D-Score, PMF-Score, G-Score, ChemScore, and F-Score), four scoring functions in the Cerius2 software (LigScore, PLP, PMF, and LUDI), two scoring functions in the GOLD program (GoldScore and ChemScore), and HINT, were tested on the refined set of the PDBbind database, a set of 800 diverse protein-ligand complexes with high-resolution crystal structures and experimentally determined Ki or Kd values. The focus of our study was to assess the ability of these scoring functions to predict binding affinities based on the experimentally determined high-resolution crystal structures of proteins in complex with their ligands. The quantitative correlation between the binding scores produced by each scoring function and the known binding constants of the 800 complexes was computed. X-Score, DrugScore, Sybyl::ChemScore, and Cerius2::PLP provided better correlations than the other scoring functions with standard deviations of 1.8-2.0 log units. These four scoring functions were also found to be robust enough to carry out computation directly on unaltered crystal structures. To examine how well scoring functions predict the binding affinities for ligands bound to the same target protein, the performance of these 14 scoring functions were evaluated on three subsets of protein-ligand complexes from the test set: HIV-1 protease complexes (82 entries), trypsin complexes (45 entries), and carbonic anhydrase II complexes (40 entries). Although the results for the HIV-1 protease subset are less than desirable, several scoring functions are able to satisfactorily predict the binding affinities for the trypsin and the carbonic anhydrase II subsets with standard deviation as low as 1.0 log unit (corresponding to 1.3-1.4 kcal/mol at room temperature). Our results demonstrate the strengths as well as the weaknesses of current scoring functions for binding affinity prediction.

The PDBbind database: collection of binding affinities for protein-ligand complexes with known three-dimensional structures.

We have screened the entire Protein Data Bank (Release No. 103, January 2003) and identified 5671 protein-ligand complexes out of 19 621 experimental structures. A systematic examination of the primary references of these entries has led to a collection of binding affinity data (K(d), K(i), and IC(50)) for a total of 1359 complexes. The outcomes of this project have been organized into a Web-accessible database named the PDBbind database.