Abstract
The computing power of most modern commodity computers is far from being fully exploited by standard usage patterns. In this work we describe the development and setup of a virtual computing cluster based on Docker containers used as worker nodes. The facility is based on Plancton: a lightweight fire-and-forget background service. Plancton spawns and controls a local pool of Docker containers on a host with free resources, by constantly monitoring its CPU utilisation. It is designed to release the resources allocated opportunistically, whenever another demanding task is run by the host user, according to configurable policies. This is attained by killing a number of running containers. One of the advantages of a thin virtualization layer such as Linux containers is that they can be started almost instantly upon request. We will show how fast the start-up and disposal of containers eventually enables us to implement an opportunistic cluster based on Plancton daemons without a central control node, where the spawned Docker containers behave as job pilots. Finally, we will show how Plancton was configured to run up to 10 000 concurrent opportunistic jobs on the ALICE High-Level Trigger facility, by giving a considerable advantage in terms of management compared to virtual machines.
Highlights
Opportunistic computing in High Energy Physics We refer to opportunistic computing as the exploitation of computing resources designed for a certain purpose during their times of inactivity, i.e. when the main task is not running
In the most trivial scenario two parties are involved: the resource owner, which might either be an enthusiast member [1] of a volunteer computing community or the administrator of an entire computing facility, decides to donate CPU cycles to a certain computing project; the second party is the end user, exploiting the ensemble of the donated resources to her own advantage. Both the volunteer and the opportunistic use cases are interesting to High-Energy Physics, where a baseline of available resources can be conveniently complemented in order to absorb computing peak times
The experience with Plancton at the High Level Trigger (HLT), which started as an experimental setup, is a huge success, making ALICE the first experiment using in production pilot containers on an online facility for opportunistic exploitation
Summary
- Docker experience at INFN-Pisa Grid Data Center E. - STAR Data Reconstruction at NERSC/Cori, an adaptable Docker container approach for HPC Mustafa Mustafa, Jan Balewski, Jérôme Lauret et al. - Integrating Containers in the CERN Private Cloud Bertrand Noel, Davide Michelino, Mathieu Velten et al. CHEP IOP Conf.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
