Computing In High Energy Physics Research Articles

Potential of the Julia Programming Language for High Energy Physics Computing

Research in high energy physics (HEP) requires huge amounts of computing and storage, putting strong constraints on the code speed and resource usage. To meet these requirements, a compiled high-performance language is typically used; while for physicists, who focus on the application when developing the code, better research productivity pleads for a high-level programming language. A popular approach consists of combining Python, used for the high-level interface, and C++, used for the computing intensive part of the code. A more convenient and efficient approach would be to use a language that provides both high-level programming and high-performance. The Julia programming language, developed at MIT especially to allow the use of a single language in research activities, has followed this path. In this paper the applicability of using the Julia language for HEP research is explored, covering the different aspects that are important for HEP code development: runtime performance, handling of large projects, interface with legacy code, distributed computing, training, and ease of programming. The study shows that the HEP community would benefit from a large scale adoption of this programming language. The HEP-specific foundation libraries that would need to be consolidated are identified.

Data and computing for high energy physics experiments

<p indent=0mm>High energy physics (HEP), which is also known as particle physics, is the study of the fundamental building blocks of nature, the interactions of elementary particles, and the origin of the Universe. Large-scale HEP experiments can produce hundreds of petabytes of real and simulated data yearly. This capability has made HEP a major explorer of scientific computing in the last few decades. In this work, three computing tasks, i.e., event reconstruction, physics simulation, and data analysis, are introduced. Subsequently, data and computing challenges are demonstrated with different examples of HEP experiments. To address the challenges, the HEP community has been focusing on developing computing models in the last thirty years. The HEP-computing model has evolved from local centers with mainframes and workstations to distributed clusters and worldwide computing grids. Some technologies generated through HEP computing, such as WWW, have made a considerable impact on the world. In the concluding part of this work, the perspective of computing technologies in HEP is discussed.

PandAna: A Python Analysis Framework for Scalable High Performance Computing in High Energy Physics

Modern experiments in high energy physics analyze millions of events recorded in particle detectors to select the events of interest and make measurements of physics parameters. These data can often be stored as tabular data in files with detector information and reconstructed quantities. Most current techniques for event selection in these files lack the scalability needed for high performance computing environments. We describe our work to develop a high energy physics analysis framework suitable for high performance computing. This new framework utilizes modern tools for reading files and implicit data parallelism. Framework users analyze tabular data using standard, easy-to-use data analysis techniques in Python while the framework handles the file manipulations and parallelism without the user needing advanced experience in parallel programming. In future versions, we hope to provide a framework that can be utilized on a personal computer or a high performance computing cluster with little change to the user code.

CLUE: A Fast Parallel Clustering Algorithm for High Granularity Calorimeters in High-Energy Physics

One of the challenges of high granularity calorimeters, such as that to be built to cover the endcap region in the CMS Phase-2 Upgrade for HL-LHC, is that the large number of channels causes a surge in the computing load when clustering numerous digitized energy deposits (hits) in the reconstruction stage. In this article, we propose a fast and fully parallelizable density-based clustering algorithm, optimized for high-occupancy scenarios, where the number of clusters is much larger than the average number of hits in a cluster. The algorithm uses a grid spatial index for fast querying of neighbors and its timing scales linearly with the number of hits within the range considered. We also show a comparison of the performance on CPU and GPU implementations, demonstrating the power of algorithmic parallelization in the coming era of heterogeneous computing in high-energy physics.

Computing for the DUNE Long-Baseline Neutrino Oscillation Experiment

This paper is based on a talk given at Computing in High Energy Physics in Adelaide, South Australia, Australia in November 2019. It is partially intended to explain the context of DUNE Computing for computing specialists. The Deep Underground Neutrino Experiment (DUNE) collaboration consists of over 180 institutions from 33 countries. The experiment is in preparation now, with commissioning of the first 10kT fiducial volume Liquid Argon TPC expected over the period 2025-2028 and a long data taking run with 4 modules expected from 2029 and beyond. An active prototyping program is already in place with a short test-beam run with a 700T, 15,360 channel prototype of single-phase readout at the Neutrino Platform at CERN in late 2018 and tests of a similar sized dual-phase detector scheduled for mid-2019. The 2018 test-beam run was a valuable live test of our computing model. The detector produced raw data at rates of up to 2GB/s. These data were stored at full rate on tape at CERN and Fermilab and replicated at sites in the UK and Czech Republic. In total 1.2 PB of raw data from beam and cosmic triggers were produced and reconstructed during the six week testbeam run. Baseline predictions for the full DUNE detector data, starting in the late 2020’s are 30-60 PB of raw data per year. In contrast to traditional HEP computational problems, DUNE’s Liquid Argon TPC data consist of simple but very large (many GB) 2D data objects which share many characteristics with astrophysical images. This presents opportunities to use advances in machine learning and pattern recognition as a frontier user of High Performance Computing facilities capable of massively parallel processing.

Operational security, threat intelligence &amp; distributed computing: the WLCG Security Operations Center Working Group

The modern security landscape for distributed computing in High Energy Physics (HEP) includes a wide range of threats employing different attack vectors. The nature of these threats is such that the most effective method for dealing with them is to work collaboratively, both within the HEP community and with partners further afield - these can, and should, include institutional and campus security teams. In parallel with this work, an appropriate technology stack is essential, incorporating current work on Big Data analytics. The work of the Worldwide LHC Computing Grid (WLCG) Security Operations Center (SOC) Working Group (WG) [1] is to pursue these goals to form a reference design (or guidelines) for WLCG sites of different types. The strategy of the group is to identify necessary components - starting with threat intelligence (MISP [2]) and network intrusion detection (Bro [3]), building a working model over time. We present on the progress of the working group thus far, in particular on the programme of workshops now underway. These workshops give an opportunity to engage with sites to allow the development of advice and procedures for deployment, as well as facilitating wider discussions on how to best work with trust groups at different levels. These trust groups vary in scope but can include institutes, National Grid Infrastructures and the WLCG as a whole.

Modeling and Simulation of Load Balancing Strategies for Computing in High Energy Physics

The amount of data to be processed by experiments in high energy physics (HEP) will increase tremendously in the coming years. To cope with this increasing load, most efficient usage of the resources is mandatory. Furthermore, the computing resources for user jobs in HEP will be increasingly distributed and heterogeneous, resulting in more difficult scheduling due to the increasing complexity of the system. We aim to create a simulation for the WLCG helping the HEP community to solve both challenges: a more efficient utilization of the grid and coping with the rising complexity of the system. There is currently no simulation in existence which helps the operators of the grid to make the correct decisions while optimizing the load balancing strategy. This paper presents a proof of concept in which the computing jobs at the Tier 1 center GridKa are modeled and simulated. To model the computing jobs we extended the Palladio simulator with a mechanism to simulate load balancing strategies. Furthermore, we implemented an automated model parameter analysis and model creation. Finally, the simulation results are validated using real-word performance data. Our results suggest that simulating larger parts of the grid is feasible and can help to optimize the utilization of the grid.

Dynamic Integration and Management of Opportunistic Resources for HEP

Demand for computing resources in high energy physics (HEP) shows a highly dynamic behavior, while the provided resources by the Worldwide LHC Computing Grid (WLCG) remains static. It has become evident that opportunistic resources such as High Performance Computing (HPC) centers and commercial clouds are well suited to cover peak loads. However, the utilization of these resources gives rise to new levels of complexity, e.g. resources need to be managed highly dynamically and HEP applications require a very specific software environment usually not provided at opportunistic resources. Furthermore, aspects to consider are limitations in network bandwidth causing I/O-intensive workflows to run inefficiently. The key component to dynamically run HEP applications on opportunistic resources is the utilization of modern container and virtualization technologies. Based on these technologies, the Karlsruhe Institute of Technology (KIT) has developed ROCED, a resource manager to dynamically integrate and manage a variety of opportunistic resources. In combination with ROCED, HTCondor batch system acts as a powerful single entry point to all available computing resources, leading to a seamless and transparent integration of opportunistic resources into HEP computing. KIT is currently improving the resource management and job scheduling by focusing on I/O requirements of individual workflows, available network bandwidth as well as scalability. For these reasons, we are currently developing a new resource manager, called TARDIS. In this paper, we give an overview of the utilized technologies, the dynamic management, and integration of resources as well as the status of the I/O-based resource and job scheduling.

WLCG and IPv6 – the HEPiX IPv6 working group

The HEPiX (http://www.hepix.org) IPv6 Working Group has been investigating the many issues which feed into the decision on the timetable for the use of IPv6 (http://www.ietf.org/rfc/rfc2460.txt) networking protocols in High Energy Physics (HEP) Computing, in particular in the Worldwide Large Hadron Collider (LHC) Computing Grid (WLCG). RIPE NCC, the European Regional Internet Registry (RIR), ran out ofIPv4 addresses in September 2012. The North and South America RIRs are expected to run out soon. In recent months it has become more clear that some WLCG sites, including CERN, are running short of IPv4 address space, now without the possibility of applying for more. This has increased the urgency for the switch-on of dual-stack IPv4/IPv6 on all outward facing WLCG services to allow for the eventual support of IPv6-only clients. The activities of the group include the analysis and testing of the readiness for IPv6 and the performance of many required components, including the applications, middleware, management and monitoring tools essential for HEP computing. Many WLCG Tier 1/2 sites are participants in the group's distributed IPv6 testbed and the major LHC experiment collaborations are engaged in the testing. We are constructing a group web/wiki which will contain useful information on the IPv6 readiness of the various software components and a knowledge base (http://hepix-ipv6.web.cern.ch/knowledge-base). This paper describes the work done by the working group and its future plans.

Scholarly literature and the press: scientific impact and social perception of physics computing

The broad coverage of the search for the Higgs boson in the mainstream media is a relative novelty for high energy physics (HEP) research, whose achievements have traditionally been limited to scholarly literature. This paper illustrates the results of a scientometric analysis of HEP computing in scientific literature, institutional media and the press, and a comparative overview of similar metrics concerning representative particle physics measurements. The picture emerging from these scientometric data documents the relationship between the scientific impact and the social perception of HEP physics research versus that of HEP computing. The results of this analysis suggest that improved communication of the scientific and social role of HEP computing via press releases from the major HEP laboratories would be beneficial to the high energy physics community.

On enhancing GridFTP and GPFS performances

One of the most demanding tasks which Computing in High Energy Physics has to deal with is reliable and high throughput transfer of large data volumes. Maximization and optimization of the data throughput are therefore key issues which have to be addressed by detailed investigations of the involved infrastructures and services.In this note, we present some transfer performance tests carried out at the INFN-CNAF Tier-1 center, using SLC4 64-bit Grid File Transfer Protocol (GridFTP) servers and a disk storage system based on the General Parallel File System (GPFS) from IBM. We describe the testbed setup and report the measurements of throughput performances in function of some fundamental variables, such as number of parallel file and number of streams per transfer, concurrent read and write activity and size of data blocks transferred.During this activity, we have verified that a significant improvement in performances of the GridFTP server can be obtained using 64bit version of Operating System and GPFS from IBM.

From Factory to Farm: Dissemination of Computing in High-Energy Physics

Computing in high-energy physics relied on large computers and dedicated central facilities in the 1960s and 1970s. With the advent of colliding-beam accelerators, minicomputers, and microcomputers in the 1980s, physicists elaborated a new decentralized model of computing in high-energy physics. They restored control to and autonomy of their analysis of physics data, while simultaneously replacing large computers with farms, or clusters of microprocessors custom-built for their tasks. This new division of labor has become dominant in scientific collaborations in other fields and is presented here in the context of the culture of the Fermi National Accelerator Laboratory, which was a crucible for the creative efforts required to disseminate computing power to users. Fermilab, the last National Accelerator Laboratory, came on line as federal expenditures for high-energy physics ebbed in the 1970s and 1980s.

Quantifying the digital divide: a scientific overview of network connectivity and grid infrastructure in South Asian countries

The future of Computing in High Energy Physics (HEP) applications depends on both the Network and Grid infrastructure. South Asian countries such as India and Pakistan are making significant progress by building clusters as well as improving their network infrastructure However to facilitate the use of these resources, they need to manage the issues of network connectivity to be among the leading participants in Computing for HEP experiments. In this paper we classify the connectivity for academic and research institutions of South Asia. The quantitative measurements are carried out using the PingER methodology; an approach that induces minimal ICMP traffic to gather active end-to-end network statistics. The PingER project has been measuring the Internet performance for the last decade. Currently the measurement infrastructure comprises of over 700 hosts in more than 130 countries which collectively represents approximately 99% of the world's Internet-connected population. Thus, we are well positioned to characterize the world's connectivity. Here we present the current state of the National Research and Educational Networks (NRENs) and Grid Infrastructure in the South Asian countries and identify the areas of concern. We also present comparisons between South Asia and other developing as well as developed regions. We show that there is a strong correlation between the Network performance and several Human Development indices.

Large systems and specific solutions Summary of CHEP97 Session F

Current topics in large systems and specific solutions for high energy physics experiments are discussed with emphasis on those topics presented in papers submitted to the International Conference on Computing in High Energy Physics, Berlin, April 7–11, 1997.

Trends and challenges in data acquisition and control systems Summary of CHEP97 Session B

Current topics in computing for data acquisition, control, and trigger systems for high energy physics experiments are discussed with emphasis on those topics presented in papers submitted to the International Conference on Computing in High Energy Physics, Berlin, April 7–11, 1997.

Bringing together high energy physicist and computer scientist: A summary of the Oxford conference on computing in high energy physics

The Oxford Conference on Computing in High Energy Physics approached the physics and computing issues with the question, “Can computer science help?” always in mind. This summary is a personal recollection of what I considered to be the highlights of the conference: the parts which contributed to my own learning experience. It can be used as a general introduction to the following papers, or as a brief overview of the current states of computer science within high energy physics.

The computing environment: A summary of the Asilomar Conference on Computing in High Energy Physics

The Asilomar Conference on Computing in High Energy Physics took as its theme the Computing Environment. This summary paper attempts to give a broad perspective of the material presented at the conference, using a framework with an emphasis on the points of particular relevance for off-line computing by experimental high energy physicists. It can be used as an introduction and guide to the other papers or to give a relatively briefoverview of our computing environment.

Asilomar conference on managing complexity in high energy physics: A summary and renaming of the conference

Even if it was not either concious or deliberate, physicists at the Asilomar Conference on Computing in High Energy Physics, probably for the first time, focussed a conference on the multitude of problems associated with the new scale of complexity in high energy physics experiments. This summary of the conference emphasizes issues of managing this complexity with new computer techniques. It also points to the emerging relevance of Artificial Intelligence subjects to particle physics.