Abstract
The File Transfer Service (FTS3) is a data movement service developed at CERN which is used to distribute the majority of the Large Hadron Collider's data across the Worldwide LHC Computing Grid (WLCG) infrastructure. At Fermilab, we have deployed FTS3 instances for Intensity Frontier experiments (e.g. DUNE) to transfer data in America and Europe, using a container-based strategy. In this article we summarize our experience building docker images based on work from the SLATE project (slateci.io) and deployed in OKD, the community distribution of Red Hat OpenShift. Additionally, we discuss our method of certificate management and maintenance utilizing Kubernetes CronJobs. Finally, we also report on the two different configurations currently running at Fermilab, comparing and contrasting a Docker-based OKD deployment against a traditional RPM-based deployment.
Highlights
The File Transfer Service (FTS3) [1] distributes the majority of the Large Hadron Collider (LHC) [2] data across the Worldwide LHC Computing Grid (WLCG) [3] infrastructure. It is integrated with experiment frameworks such as Rucio [4] and DIRAC [5] and it is used by more than 35 experiments at CERN and in other data-intensive sciences outside of the LHC and even outside the High Energy Physics (HEP) domain
The fetch-crls-first-time initContainer downloads the certificate authorities and initial certificate revocation lists - these are needed by the FTS3 server
These tokens can be provided by two different Entity Providers: tokens for Fermilab-based VOs are provided by a CILogon [22] issuer, while tokens for the CMS experiment will be issued by WLCG IAM [23]
Summary
The File Transfer Service (FTS3) [1] distributes the majority of the Large Hadron Collider (LHC) [2] data across the Worldwide LHC Computing Grid (WLCG) [3] infrastructure. One of the biggest advantages of FTS3 is its ability to be run without link and channel configuration per source and destination pair It enables parallel transfer scheduling and optimisation to get the most from the network without saturating the storage systems, with support for intelligent priorities, activity shares and VO shares for classification of transfers. SLATE has taken a page from industry to provide a common, Kubernetes-based platform to deploy and operate services at sites It adds a federation layer, essentially a privilege model to provide secure access to trusted members of central production teams which have expertise in operating globally distributed systems. The containerized version of the FTS3 software described in this paper is one such successful use of this feature
Sign-in/Register to view highlights & summary 
Published Version
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call