Abstract
Recently much effort has been invested in using convolutional neural network (CNN) models trained on 3D structural images of protein-ligand complexes to distinguish binding from non-binding ligands for virtual screening. However, the dearth of reliable protein-ligand x-ray structures and binding affinity data has required the use of constructed datasets for the training and evaluation of CNN molecular recognition models. Here, we outline various sources of bias in one such widely-used dataset, the Directory of Useful Decoys: Enhanced (DUD-E). We have constructed and performed tests to investigate whether CNN models developed using DUD-E are properly learning the underlying physics of molecular recognition, as intended, or are instead learning biases inherent in the dataset itself. We find that superior enrichment efficiency in CNN models can be attributed to the analogue and decoy bias hidden in the DUD-E dataset rather than successful generalization of the pattern of protein-ligand interactions. Comparing additional deep learning models trained on PDBbind datasets, we found that their enrichment performances using DUD-E are not superior to the performance of the docking program AutoDock Vina. Together, these results suggest that biases that could be present in constructed datasets should be thoroughly evaluated before applying them to machine learning based methodology development.
Highlights
Virtual screening plays an essential role in lead identification in the early stages of drug discovery [1,2]
We showed that the performance of protein-ligand convolutional neural network (CNN) models is affected by hidden biases in the Database of Useful Decoys-Enhanced (DUD-E) dataset
We showed that analogue biases are common both within the sets of actives associated with each target and across sets of actives associated with different targets
Summary
Virtual screening plays an essential role in lead identification in the early stages of drug discovery [1,2]. Accurate lead identification can dramatically reduce the time and costs associated with experimental assays. Developing computational tools that can identify lead compounds with pharmacological activity against a selected protein target has been a longstanding goal for computational chemists. Hidden bias in the DUD-E dataset leads to misleading performance of deep learning in virtual screening
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have