A Lightweight Door into Non-Grid Sites

J M Dost ,Huijun Zhu,Derek Weitzel,Edgar Fajardo,Lincoln Bryant,Marco Mascheroni,Brian Bockelman,Mátyás Selmeci ,I Sfiligoi ,J Letts ,F Würthwein ,Brian Lin ,J Stephen ,Timothy J Cartwright ,R W Gardner

doi:10.1051/epjconf/202024507005

Abstract

The Open Science Grid (OSG) provides a common service for resource providers and scientific institutions, and supports sciences such as High Energy Physics, Structural Biology, and other community sciences. As scientific frontiers expand, so does the need for resources to analyze new data. For example, High Energy Physics experiments such as the LHC experiments foresee an exponential growth in the amount of data collected, which comes with corresponding growth in the need for computing resources. Allowing resource providers an easy way to share their resources is paramount to ensure the grow of resources available to scientists. In this context, the OSG Hosted CE initiative provides site administrator a way to reduce the effort needed to install and maintain a Compute Element (CE), and represents a solution for sites who do not have the effort and expertise to run their own Grid middleware. An HTCondor Compute Element is installed on a remote VM at UChicago for each site that joins the Hosted CE initiative. The hardware/software stack is maintained by OSG Operations staff in a homogeneus and automated way, providing a reduction in the overall operational effort needed to maintain the CEs: one single organization does it in an uniform way, instead of each single resource provider doing it in their own way. Currently, more than 20 institutions joined the Hosted CE initiative. This contribution discusses the technical details behind a Hosted CE installation, highlighting key strengths and common pitfalls, and outlining future plans to further reduce operational experience.

Highlights

Grid computing[1] has been the technology of choice to address the distributed computational needs of many scientific fields like High Energy Physics, Structural Biology, and other science communities
The installation is unique in that some software needs to be installed on the dedicated host owned by Open Science Grid (OSG), and other scripts and configuration files need to be staged into the non-root user home directory on the head node of the remote cluster owned by the site
With the Hosted Compute Element (CE) initiative, the hardware/software stack needed to operate a CE is maintained by OSG Operations staff in a homogeneous and automated way

Summary

Introduction

Grid computing[1] has been the technology of choice to address the distributed computational needs of many scientific fields like High Energy Physics, Structural Biology, and other science communities. The Open Science Grid[2] (OSG) is a national, distributed computing partnership for data-intensive research that facilitates access to distributed high throughput computing for. Allowing resource providers (sites) an easy way to share their resources is paramount to ensure that the growth of resources is available to scientists. The Hosted CE initiative by the OSG provides the site administrator a way to reduce the effort needed to install and maintain a Compute Element (CE), the Grid portal to a compute cluster, facilitating the process of sharing resources between the scientific communities served by OSG

Traditional CEs vs Hosted CEs

Routine Operations

Future Plans

Conclusions