Abstract
Scientific communities have been in the forefront of adopting new technologies and methodologies in the computing. Scientific computing has influenced how science is done today, achieving breakthroughs that were impossible to achieve several decades ago. For the past decade several such communities in the Open Science Grid (OSG) and the European Grid Infrastructure (EGI) have been using GlideinWMS to run complex application workflows to effectively share computational resources over the grid. GlideinWMS is a pilot-based workload management system (WMS) that creates on demand, a dynamically sized overlay HTCondor batch system on grid resources. At present, the computational resources shared over the grid are just adequate to sustain the computing needs. We envision that the complexity of the science driven by "Big Data" will further push the need for computational resources. To fulfill their increasing demands and/or to run specialized workflows, some of the big communities like CMS are investigating the use of cloud computing as Infrastructure-As-A-Service (IAAS) with GlideinWMS as a potential alternative to fill the void. Similarly, communities with no previous access to computing resources can use GlideinWMS to setup up a batch system on the cloud infrastructure. To enable this, the architecture of GlideinWMS has been extended to enable support for interfacing GlideinWMS with different Scientific and commercial cloud providers like HLT, FutureGrid, FermiCloud and Amazon EC2. In this paper, we describe a solution for cloud bursting with GlideinWMS. The paper describes the approach, architectural changes and lessons learned while enabling support for cloud infrastructures in GlideinWMS.
Highlights
Distributed and High Throughput computing in form of computing grids has been widely adopted by widespread scientific communities with high computing demands, such as high-energy physics (HEP)
GlideinWMS is a pilot-based workload management system (WMS) that creates on demand a dynamically sized overlay HTCondor batch system [7][8] on grid and cloud resources to address the complex needs of Virtual Organizations (VO) in running application workflows
Interfacing GlideinWMS with Computing Clouds Today, GlideinWMS is the leading pilot based WMS used by the scientific communities in the Open Science Grid (OSG) and the European Grid Infrastructure (EGI)
Summary
Distributed and High Throughput computing in form of computing grids has been widely adopted by widespread scientific communities with high computing demands, such as high-energy physics (HEP). GlideinWMS is a pilot-based WMS that creates on demand a dynamically sized overlay HTCondor batch system [7][8] on grid and cloud resources to address the complex needs of VOs in running application workflows. The GlideinWMS architecture supports multiple glidein factories and frontends working together through a common HTCondor collector This separation of tasks allows for better scalability and separation of operational responsibilities between grid debugging and policy management [12]. 4. Interfacing GlideinWMS with Computing Clouds Today, GlideinWMS is the leading pilot based WMS used by the scientific communities in the OSG and the EGI. Grid Site Glidein is an HTCondor job that runs inside a worker node This does not require any GlideinWMS software to be installed on the worker node. The end user running jobs on grid and/or cloud resources is completely shielded and would not notice any functional differences between the resources provisioned in the cloud versus those in the grids
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
