CavBench: A benchmark for protein cavity detection methods.

Sérgio Dias,Alfredo Ferreira,Abel J P Gomes,Joaquim Jorge,Ana Mafalda Martins,Tiago Simões,Francisco Fernandes,Manuela Helmer-Citterich

doi:10.1371/journal.pone.0223596

Abstract

Extensive research has been applied to discover new techniques and methods to model protein-ligand interactions. In particular, considerable efforts focused on identifying candidate binding sites, which quite often are active sites that correspond to protein pockets or cavities. Thus, these cavities play an important role in molecular docking. However, there is no established benchmark to assess the accuracy of new cavity detection methods. In practice, each new technique is evaluated using a small set of proteins with known binding sites as ground-truth. However, studies supported by large datasets of known cavities and/or binding sites and statistical classification (i.e., false positives, false negatives, true positives, and true negatives) would yield much stronger and reliable assessments. To this end, we propose CavBench, a generic and extensible benchmark to compare different cavity detection methods relative to diverse ground truth datasets (e.g., PDBsum) using statistical classification methods.

Highlights

Modeling protein-ligand interactions is crucial to drug discovery and design, as well as to understand bio-molecular structures
CavDataset comprises 2293 proteins, among which we find 660 apo proteins and 1633 holo proteins
We identify false positives by zeroed rows of the overlapping matrix or, equivalently, cavities detected by the method, but that do not exist in the ground-truth

Summary

Introduction

Modeling protein-ligand interactions is crucial to drug discovery and design, as well as to understand bio-molecular structures. While extensive efforts have been applied for many years into discovering new methods to model protein-ligand interactions, a comprehensive mechanism to compare and assess such methods (and algorithms) is still lacking. This makes it difficult to properly ascertain the contributions of each method in the context of the myriad of approaches developed over the past decades. We are interested in benchmarking protein cavity (or pocket) detection methods against one or more databases of cavities (e.g., PDBsum [1, 2]) or even databases of already-known binding sites (e.g., scPDB [3]) Often, these already-known protein binding sites correspond to protein cavities, so they may work as ground-truth cavities. Pocket detection plays an important role in protein-ligand docking and structure-based drug design

Methods

Results

Conclusion