Large-scale virtual screening on public cloud resources with Apache Spark

Marco Capuccini,Wesley Schaal,Laeeq Ahmed,Erwin Laure,Ola Spjuth

doi:10.1186/s13321-017-0204-4

Marco Capuccini, Wesley Schaal + Show 3 more

Open Access

https://doi.org/10.1186/s13321-017-0204-4

Copy DOI

Abstract

BackgroundStructure-based virtual screening is an in-silico method to screen a target receptor against a virtual molecular library. Applying docking-based screening to large molecular libraries can be computationally expensive, however it constitutes a trivially parallelizable task. Most of the available parallel implementations are based on message passing interface, relying on low failure rate hardware and fast network connection. Google’s MapReduce revolutionized large-scale analysis, enabling the processing of massive datasets on commodity hardware and cloud resources, providing transparent scalability and fault tolerance at the software level. Open source implementations of MapReduce include Apache Hadoop and the more recent Apache Spark.ResultsWe developed a method to run existing docking-based screening software on distributed cloud resources, utilizing the MapReduce approach. We benchmarked our method, which is implemented in Apache Spark, docking a publicly available target receptor against sim 2.2 M compounds. The performance experiments show a good parallel efficiency (87%) when running in a public cloud environment.ConclusionOur method enables parallel Structure-based virtual screening on public cloud resources or commodity computer clusters. The degree of scalability that we achieve allows for trying out our method on relatively small libraries first and then to scale to larger libraries. Our implementation is named Spark-VS and it is freely available as open source from GitHub (https://github.com/mcapuccini/spark-vs).Graphical abstract.

Highlights

Structure-based virtual screening is an in-silico method to screen a target receptor against a virtual molecular library
We developed a method for parallel Structure-based virtual screening (SBVS) following the MR approach, which enables the screening of large molecular libraries on public cloud resources or on commodity hardware clusters
The method is implemented in Apache Spark and it is distributed as an open source library, named Spark-VS, along with some example ready-to-run SBVS pipelines

Summary

Introduction

Structure-based virtual screening is an in-silico method to screen a target receptor against a virtual molecular library. Applying docking-based screening to large molecular libraries can be computationally expensive, it constitutes a trivially parallelizable task. Google’s MapReduce revolutionized large-scale analysis, enabling the processing of massive datasets on commodity hardware and cloud resources, providing transparent scalability and fault tolerance at the software level. A widely established approach for this is High-Throughput Screening (HTS), where large molecular libraries are screened against a bioassay in fully automated environments [1]. Structure-based virtual screening (SBVS) is a complementary in silico method [3] that has been successfully used to generate new drug leads [4, 5]. Methods in high-throughput structural biology allowed the production of massive virtual molecular libraries. ZINC [6] represents an excellent

Methods

Results

Discussion

Conclusion