Performance of machine-learning scoring functions in structure-based virtual screening

Maciej Wójcikowski,Pawel Siedlecki,Pedro J Ballester

doi:10.1038/srep46710

Abstract

Classical scoring functions have reached a plateau in their performance in virtual screening and binding affinity prediction. Recently, machine-learning scoring functions trained on protein-ligand complexes have shown great promise in small tailored studies. They have also raised controversy, specifically concerning model overfitting and applicability to novel targets. Here we provide a new ready-to-use scoring function (RF-Score-VS) trained on 15 426 active and 893 897 inactive molecules docked to a set of 102 targets. We use the full DUD-E data sets along with three docking tools, five classical and three machine-learning scoring functions for model building and performance assessment. Our results show RF-Score-VS can substantially improve virtual screening performance: RF-Score-VS top 1% provides 55.6% hit rate, whereas that of Vina only 16.2% (for smaller percent the difference is even more encouraging: RF-Score-VS top 0.1% achieves 88.6% hit rate for 27.5% using Vina). In addition, RF-Score-VS provides much better prediction of measured binding affinity than Vina (Pearson correlation of 0.56 and −0.18, respectively). Lastly, we test RF-Score-VS on an independent test set from the DEKOIS benchmark and observed comparable results. We provide full data sets to facilitate further research in this area (http://github.com/oddt/rfscorevs) as well as ready-to-use RF-Score-VS (http://github.com/oddt/rfscorevs_binary).

Highlights

scoring functions (SFs) have outperformed classical SFs at binding affinity prediction has been highlighted by several reviews[13,18,19,20]
Directory of Useful Decoys – Enhanced (DUD-E) dataset[5] was built primarily to test performance of docking and scoring software, but it fits into the constrains of a screening dataset described above
The target types are quite diverse and consist of receptors (GPCR, chemokine and nuclear), globular enzymes, kinases and virus proteases among others. It is heterogeneous in case of ligand abundance; Catechol O-methyltransferase (COMT) has only 41 active compounds, compared to MAP kinase p38 alpha (MK14) which has 578 unique, dissimilar compounds

Summary

Introduction

SFs have outperformed classical SFs at binding affinity prediction has been highlighted by several reviews[13,18,19,20]. Any adjustable parameter of the machine-learning SF is selected from data not used to estimate the performance of the model[13] (e.g. k-fold cross-validation[29] is done for either model selection or estimating generalization performance, but not both) Neither of these safeguards against model overfitting is enforced when measuring the performance of classical SFs30. Another prime example is that of MIEC-SVM retrospectively outperforming Glide and X-Score[32] on 40 DUD2 targets, in a study that showed that VS performance increases with training set size as expected This machine-learning SF has been found superior to classical SFs in prospective VS studies on kinases[33]. We investigate what is the influence of including negative data instances (inactive molecules docked to targets) on machine learning SF Such chimeric complexes are currently discarded from training procedures. We assess the VS performance of the SFs in both established-target and novel-target settings, either tailored for broad application or for a specific target

Objectives

Methods

Results

Conclusion