Abstract

DESY is one of the largest accelerator laboratories in Europe. It develops and operates state of the art accelerators for fundamental science in the areas of high energy physics, photon science and accelerator development. While for decades high energy physics (HEP) has been the most prominent user of the DESY compute, storage and network infrastructure, various scientific areas as science with photons and accelerator development have caught up and are now dominating the demands on the DESY infrastructure resources, with significant consequences for the IT resource provisioning. In this contribution, we will present an overview of the computational, storage and network resources covering the various physics communities on site. Ranging from high-throughput computing (HTC) batch-like offline processing in the Grid and the interactive user analyses resources in the National Analysis Factory (NAF) for the HEP community, to the computing needs of accelerator development or of photon sciences such as PETRA III or the European XFEL. Since DESY is involved in these experiments and their data taking, their requirements include fast low-latency online processing for data taking and calibration as well as offline processing, thus high-performance computing (HPC) workloads, that are run on the dedicated Maxwell HPC cluster. As all communities face significant challenges due to changing environments and increasing data rates in the following years, we will discuss how this will reflect in necessary changes to the computing and storage infrastructures. We will present DESY compute cloud and container orchestration plans as a basis for infrastructure and platform services. We will show examples of Jupyter notebooks for small scale interactive analysis, as well as its integration into large scale resources such as batch systems or Spark clusters. To overcome the fragmentation of the various resources for all scientific communities at DESY, we explore how to integrate them into a seamless user experience in an Interdisciplinary Data Analysis Facility.

Highlights

  • DESY as an interdisciplinary laboratoryDESY is a large research laboratory in Germany, carrying out fundamental research

  • DESY carries out astroparticle physics and other branches of science more loosely coupled to accelerator research and usage

  • DESY IT has worked together with Petra-III, FLASH and European XFEL free-electron laser (EuXFEL) experts to set up the data taking, data storage and data analysis facilities

Read more

Summary

Introduction

DESY is a large research laboratory in Germany, carrying out fundamental research. Particle accelerators were the founding basis of DESY and until today are a key. DESY develops and operates accelerators and carries out science related to accelerators. This is mainly research with photons and high energy particle physics. DESY carries out astroparticle physics and other branches of science more loosely coupled to accelerator research and usage. We will briefly describe the different activities

Developing and operating accelerators
Research with Photons
High energy physics
Other activities
Grid Services and Compute
Maxwell HPC Services and Compute
Storage and Data Taking
New computing models and needs
Implementation
Further combining clusters
Summary and outlook
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call