High Performance Computing Centers Research Articles

Pareto Approximation Empirical Results of Energy-Aware Optimization for Precedence-Constrained Task Scheduling Considering Switching Off Completely Idle Machines

Recent advances in cloud computing, large language models, and deep learning have started a race to create massive High-Performance Computing (HPC) centers worldwide. These centers increase in energy consumption proportionally to their computing capabilities; for example, according to the top 500 organization, the HPC centers Frontier, Aurora, and Super Computer Fugaku report energy consumptions of 22,786 kW, 38,698 kW, and 29,899 kW, respectively. Currently, energy-aware scheduling is a topic of interest to many researchers. However, as far as we know, this work is the first approach considering the idle energy consumption by the HPC units and the possibility of turning off unused units entirely, driven by a quantitative objective function. We found that even when turning off unused machines, the objectives of makespan and energy consumption still conflict and, therefore, their multi-objective optimization nature. This work presents empirical results for AGEMOEA, AGEMOEA2, GWASFGA, MOCell, MOMBI, MOMBI2, NSGA2, and SMS-EMOA. The best-performing algorithm is MOCell for the 400 real scheduling problem tests. In contrast, the best-performing algorithm is GWASFGA for a small-instance synthetic testbed.

Enabling high reliability via matroidal connectivity and conditional matroidal connectivity on arrangement graph networks

Connectivity indicators are commonly used to evaluate system fault tolerance and reliability. However, with the high demand for multi-processor systems in high-performance computing and data center networks, the number of processors is getting larger, and the network is getting more complex. Thus, traditional connectivity and other indicators are hardly competent in assessing the reliability of complex networks. The matroidal connectivity and conditional matroidal connectivity are novel connectivity metrics that measure the actual fault-tolerant capability based on the constraints of each network dimension. In this paper, we study matroidal connectivity and conditional matroidal connectivity of the (n,k)-arrangement graph network An,k from the natural perspective of the partition of edge dimension and obtain their theoretically accurate values. Moreover, we conduct numerical analysis to compare matroidal connectivity with other conditional edge connectivities in An,k. Additionally, we explore the distribution pattern of edge failure through simulation experiments in An,k and attain the relation of conditional matroidal connectivity related to network scales. Our investigations include two famous network classes: alternating group graphs and star graphs.

Chem4Energy: a consortium of the Royal Society Africa Capacity-Building Initiative.

The Africa Capacity-Building Initiative is a Royal Society programme funded by the former UK Department for International Development to develop collaborative research between scientists in sub-Saharan Africa and the UK. Initially, four institutions were involved in the Chem4Energy consortium: Cardiff University in the UK and three African partners, the Kwame Nkrumah University of Science and Technology, Ghana, the University of Namibia and the University of Botswana, soon also including the Botswana International University of Science and Technology. The Chem4Energy research programme focused on 'New materials for a sustainable energy future: linking computation with experiment', aiming to deploy the synergy between state-of-the-art computational and experimental techniques to design and optimize new catalysts and semiconductor materials for renewable energy applications, based on materials that are abundant and readily available in African countries. The Chem4Energy consortium has achieved ambitious research goals, graduated seven PhD students and delivered a high-quality cross-disciplinary training programme in materials science and simulation techniques relevant to renewable energy applications. Since 2021, the extended consortium, including North-West University and the Centre for High-Performance Computing in South Africa, has remained active through an annual Chem4Energy conference series, with the sixth meeting taking place in Namibia in April 2025.

Analysis model of energy consumption variables for data processing in high-performance computing systems

One of the main challenges in the efficient operation of a high-performance computing (HPC) center is the energy consumption generated by the operation of the data center where the HPC equipment is housed, mainly because this consumption is reflected in very high accounts payable, and this may affect the level of service offered to users. The study of the different factors and elements that can make energy consumption more efficient in these data centers provides an opportunity to focus these resources on elements that favor the use of HPC. The design variables provided by manufacturers to manage HPC systems and monitoring systems provide an accurate view of the behavior of these variables according to how they are used. HPC architectures are configured in a very particular way for each HPC data center, creating particular scenarios of operation and performance in each implementation. Various proposals and technologies have been developed for the analysis of the energy consumption of a data center, and the processing elements include a series of indicators and technologies that manufacturers have developed to determine the energy efficiency. This article seeks to identify this series of processing and performance variables, which affect the energy consumption of HPC equipment, for the implemented computing architectures based on the analysis of performance models to obtain a general over-view of their effect on energy consumption in a case study to identify the behaviors of particular job assignment factors and provide an analysis of the energy consumption under particular conditions.

The GEA pipeline for characterizing Escherichia coli and Salmonella genomes

Salmonella enterica and Escherichia coli are major food-borne human pathogens, and their genomes are routinely sequenced for clinical surveillance. Computational pipelines designed for analyzing pathogen genomes should both utilize the most current information from annotation databases and increase the coverage of these databases over time. We report the development of the GEA pipeline to analyze large batches of E. coli and S. enterica genomes. The GEA pipeline takes as input paired Illumina raw reads files which are then assembled followed by annotation. Alternatively, assemblies can be provided as input and directly annotated. The pipeline provides predictive genome annotations for E. coli and S. enterica with a focus on the Center for Genomic Epidemiology tools. Annotation results are provided as a tab delimited text file. The GEA pipeline is designed for large-scale E. coli and S. enterica genome assembly and characterization using the Center for Genomic Epidemiology command-line tools and high-performance computing. Large scale annotation is demonstrated by an analysis of more than 14,000 Salmonella genome assemblies. Testing the GEA pipeline on E. coli raw reads demonstrates reproducibility across multiple compute environments and computational usage is optimized on high performance computers.

The Advanced Computing Hub at BSC: improving fusion codes following modern software engineering standards

Several dedicated High-Performance Computing (HPC) centers provide essential expertise and support in developing a suitable portfolio of EUROfusion standard codes. Barcelona Supercomputing Center (BSC) is one of these HPC hubs involved in this complex task. Several fusion codes were selected, installed and analyzed to meet the developers’ requirements, ranging from portability to GPU, improving the performance, getting better data management, extending the capacity of coupling with other codes, etc. In this paper, we will describe the work developed by BSC and some of the tasks carried out in this project. We will explain briefly how the project is faced and the work required to create good quality codes, i.e. robust and trustable software capable of running efficiently in modern HPC systems.

Vitamin B1 Supplementation Ameliorates Obesogenic Effects of a High Fat Diet in Male Mice

Dietary deficiencies in vitamins such as thiamine (Vitamin B1) are common even in developed countries and can have a range of deleterious health effects. This trend in Vitamin B1 deficiency coincides with increased fat consumption in the American diet, particularly of soybean oil, the most widely consumed cooking oil in the U.S. which is also present in many processed foods. Here, we hypothesized that the concurrent malnutrition caused by a soybean oil-based high fat diet (SO-HFD) and a physiological deficiency in thiamine can exacerbate conditions such as obesity, diabetes, and fatty liver that are associated with consumption of HFDs. To test this, C57BL/6N male mice (N=8-10 per group) were fed either a low-fat chow or an SO-HFD, with or without supplementation with 640 mg/kg of the thiamine analog thiamine tetrahydrofurfuryl disulfide (TTFD), for 18 weeks. Our results showed that the SO-HFD supplemented with TTFD leads to significantly decreased weight gain and delayed the onset of glucose intolerance compared to SO-HFD without added TTFD (p<0.05). In contrast, TTFD supplementation did not have any effect on body weight or glucose intolerance in the context of a low-fat diet. Metabolomic analysis revealed that the SO-HFD decreased absolute thiamine levels both in the blood and liver tissue compared to the low fat control diet (p<0.05); this decrease was reversed by TTFD supplementation suggesting that some of the negative effects of the SO-HFD may be due to its impact on the Vitamin B1 pathway. Untargeted metabolomic analysis identified several other metabolites — such as amino acids, organic acids, methylamine osmolytes and those involved in nucleotide metabolism — that were also affected by TTFD supplementation, but in a beneficial fashion. Transcriptomic analysis of the liver and intestines revealed alterations in thiamine absorption and metabolic pathways by the SO-HFD; effects of TTFD on hepatic gene expression and fatty liver will be presented. TTFD supplementation given after the onset of SO-HFD-induced obesity was not able to reverse the phenotype, suggesting that the deleterious effects of the SO-HFD-induced Vitamin B1 deficiency may be permanent. In conclusion, our findings indicate that consumption of an SO-enriched diet, similar in fat content to what is consumed by most Americans, can result in a Vitamin B1 deficiency. Although TTFD supplementation may mitigate some of the adverse metabolic effects of this diet, this intervention must be done at an early stage. Funding: The Thiamine Advocacy Foundation Research Grant (FMS, PHD, PD); NIH R01DK127082 (FMS)Acknowledgements: UCR Genomics Core Facility, UCR Metabolomics Core Facility, UCR High Performance Computing Center (HPCC). This is the full abstract presented at the American Physiology Summit 2024 meeting and is only available in HTML format. There are no additional versions or additional content available for this abstract. Physiology was not involved in the peer review process.

HPC resources for CMS offline computing: An integration and scalability challenge for the Submission Infrastructure

HPC resources for CMS offline computing: An integration and scalability challenge for the Submission Infrastructure

Optical switching will innovate intra data center networks [Invited Tutorial

Reflecting the recent slow-down in Moore’s law and the proliferation of artificial intelligence/machine learning workloads, the performance and energy consumption of networks are becoming barriers in high-performance computing (HPC) and data centers. Optical switches are expected to break these barriers, and indeed their introduction has recently commenced in data centers. This paper discusses how optical switching technologies can innovate future intra data center networks. Hyperscale data centers are much bigger in scale, and network requirements are slightly different from those of HPC. This paper focus on data center networks, since the impact of optical technologies will be more significant in data centers than in HPC. In addition to the scale issue, important metrics to be considered for network design are traffic characteristics and latency, both of which are highlighted in this paper. For hybrid (electrical packet and optical circuit) switching networks, the target latency for the optical circuit switch network (connection setup/teardown time) is shown to be around 10 µs, and the needed technologies are clarified and verified by experiments. The optical switch can simplify the present multi-tier switching network above tier-1 switches into a single tier configuration, which is possible with the development of efficient large port count optical switches. Among the different switching architectures, combining the different dimensions of space and wavelength is shown to be one of the best solutions. Fast switching needs fast device response time. Si photonics devices using Mach–Zehnder interferometers or ring-resonator-based switches and tunable filters are the most promising candidates; they offer cost-effective mass-production and fast operation and so are excellent candidates for the optical switches envisaged. Another critical technology to maximize the benefits of optical switches is a simple and low-latency control mechanism. Different approaches have been suggested as summarized in this work. Among them, harnessing optical switch parallelism is a unique technique that matches recent advances in electrical switch chips. A fast control network is realized by using a fully decentralized and asynchronous control mechanism. A hyperscale data center offers a wide variety of services, and no one system fits all needs. Optimization of parameters is an important task for maximizing the impact of optical switching in different kinds of data centers.

The Roles of the National Center for High-performance Computing in Projects Involving Health-Care Big Data

<p>生物檢體數位化後將有機會永久保存生物檢體中的資訊與成就資料再利用之最大效益，然而數位化後的生醫巨量資料將仰賴大型的資訊基礎設施使資料服務為之不墜。近年來隨著生醫科技的進展，無論是基因定序或人工智慧技術的應用下，資料的規模，或者是計算規模，與背後的資訊設備需求，早已不是個別研究人員的實驗室資源所可以承受。再者隨著對個人資料保護的要求、注重資訊安全，及尊重資料當事人對自身資料權利表達機會，乃至於資料使用者對於資料分析的方便性與功能完整性的要求，讓生醫資料的運用與處理更仰賴於各項先進資訊技術集結，透過資訊專業建置出符合各界期待的生醫平台，讓生醫資料發揮其最大的效益。</p> <p> </p><p>The digitalization of biosamples offers an opportunity to permanently retain the information contained in biosamples and maximize data reutilization. However, the digitalized biomedical big data require large-scale information infrastructure to sustain relevant services. With the advancement of biomedical technologies, applications involving gene sequencing and artificial intelligence have exceeded a level that individual researchers or laboratories can afford in terms of data volume, computational capacity, and hardware requirements. The demands for personal information protection, information security, right of information self-determination, and the convenience and functional completeness of data analysis have also made biomedical data application and processing more reliant on advanced information technologies. A biomedical platform that meets the expectations of all stakeholders should thus be established through advanced information technologies to extract the maximum potential from biomedical data.</p> <p> </p>

Genome-wide association study meta-analysis supports association between MUC1 and ectopic pregnancy.

Can we identify genetic variants associated with ectopic pregnancy by undertaking the first genome-wide association study (GWAS) leveraging two large-scale biobank initiatives? We identified two novel genome-wide significant associations with ectopic pregnancy, highlighting MUC1 (mucin 1) as the most plausible affected gene. Ectopic pregnancy is an important cause of maternal morbidity and mortality worldwide. Despite being a common early pregnancy complication, the genetic predisposition to this condition remains understudied and no large scale genetic studies have been performed so far. A GWAS meta-analysis including 7070 women with ectopic pregnancy and 248 810 controls from Estonian Biobank and the FinnGen study. We identified ectopic pregnancy cases from national registers by ICD (International Classification of Disease) codes (ICD-10 O00), and all remaining women were considered controls. We carried out standard GWAS meta-analysis and additionally annotated GWAS signals, analysed co-localization with quantitative trait loci, estimated genetic correlations and identified associated phenotypes to characterize the genetic signals, as well as to analyse the genetic and phenotypic relationships with the condition. We identified two genome-wide significant loci on chromosomes 1 (rs4971091, P = 5.32×10-9) and 10 (rs11598956, P = 2.41×10-8) potentially associated with ectopic pregnancy. Follow-up analyses propose MUC1, which codes for an epithelial glycoprotein with an important role in barrier function, as the most likely candidate gene for the association on chromosome 1. We also characterize the phenotypic and genetic correlations with other phenotypes, identifying a genetic correlation with smoking and diseases of the (genito)urinary and gastrointestinal system, and phenotypic correlations with various reproductive health diagnoses, reflecting the previously known epidemiological associations. The GWAS meta-analysis summary statistics are available from the GWAS Catalogue (GCST90272883). The main limitation is that the findings are based on European-based ancestry populations, with limited data on other populations, and we only captured maternal genomes. Additionally, further larger meta-analysis or independent studies are needed to validate these findings. This study encourages the use of large-scale genetic datasets to unravel genetic factors linked to ectopic pregnancy, which is difficult to study in experimental settings. Increased sample size might bring additional genetic factors associating with ectopic pregnancy and inform its heritability. Altogether, our results provide more insight into the biology of ectopic pregnancy and, accordingly, the biological processes governing embryo implantation. N.P.G. was supported by MATER Marie Sklodowska-Curie which received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No. 813707. This study was funded by European Union through the European Regional Development Fund Project No. 2014-2020.4.01.15-0012 GENTRANSMED. Computations were performed in the High-Performance Computing Center of University of Tartu. The authors declare no competing interests.

Advanced numerical techniques for predicting frequency response functions

Numerical methods have been widely used to predict Frequency Response Functions (FRF) of structures. The FRF can be used to characterize structural dynamics and support sound and vibration control. Generally, frequency sweep is conducted in the FRF calculations to identify damping and resonant frequencies of structures under excitations. However, as numerical models are getting complex and larger, the demands for computational resources (e.g., CPU time, memory, disk space) are greatly increased. Lately, high performance computing (HPC) at the DoD HPC center has been used with advanced numerical techniques to reduce the overall computational time. Those advanced numerical techniques include Krylov subspace and Galerkin Projection (KGP) and Pade Approximation. Significantly CPU time reduction has been demonstrated for relatively smaller FE models. This paper will further discuss adaptive KGP by automating the frequency sweep process via calculated relative residues, then combine with Finite Element Tearing and Interconnecting (FETI) to solve for FRF of a large FEM model in the order of 60M DOF. A flat plate with viscoelastic layer will serve as an example to demonstrate accuracy and efficiency of AKGP with FETI. The benefits of using HPC with advanced numerical techniques to solve for large FEM models are clearly displayed.

Variable Neighborhood Search for precedence-constrained tasks optimization on heterogeneous systems

Supercomputing power is one of the fundamental pillars of the digital society, which depends on the accurate scheduling of parallel applications in High-Performance Computing (HPC) centers to minimize computing times. However, precedence-constraint task scheduling is a well-known NP-Hard optimization problem, and no optimal polynomial-time algorithm exists to solve it. Therefore, as accurate as possible to the optimal values, heuristic algorithms are of relevant interest. Our new scheduling proposal name is EFT-GVNS, which stands for Earliest Finish Time - General Variable Neighborhood Search. EFT-GVNS uses a Composite Local Search (CLS), making our proposal more efficient than traditional GVNS. EFT-GVNS accuracy against four high-performance algorithms from the state-of-the-art (EDA, EFT-ILS, GRASP-CPA, MPQGA) and one reference algorithm in the literature (HEFT) is studied. Experimental results over four real-world applications (Fpppp, LIGO, Robot, Sparse) and 14 synthetic instances from the literature show that EFT-GVNS outperforms in terms of the median achieved results, with a global improvement of 37.6%, 27.4%, 17.8%, 6.1%, 2.2% to HEFT, EDA, EFT-ILS, GRASP-CPA, and MPQGA, respectively. EFT-GVNS achieves all the 14 optimal values of the synthetic benchmark.

Namibia's first high performance computer

High performance computing (HPC) refers to the practice of aggregating computing power of several computing nodes in a way that delivers much higher performance than one could achieve by a typical desktop computer in order to solve large problems in business, science, or engineering. The University of Namibia has so far received two HPC racks from the Centre for High Performance Computing in South Africa, of which one is operational. The primary use of the rack was foreseen to be human capacity development and awareness in HPC and to form part of Namibia’s readiness in participating in the Square Kilometre Array (SKA) and the African Very Long Baseline Interferometry Network (AVN) projects, but is now also being used for research in multi-wavelength astronomy and beyond. This is one of the first HPC services set up and operated by an entirely African team. We perform tests to benchmark the computational power and data transfer capabilities of the system and find that each node, on average, has a peak performance power of 82.4±1.1 GFLOPS. We also summarise all the projects that have enlisted the HPC facility.

Efficient and Reliable Data Management for Biomedical Applications.

This chapter discusses the challenges and requirements of modern Research Data Management (RDM), particularly for biomedical applications in the context of high-performance computing (HPC). The FAIR data principles (Findable, Accessible, Interoperable, Reusable) are of special importance. Data formats, publication platforms, annotation schemata, automated data management and staging, the data infrastructure in HPC centers, file transfer and staging methods in HPC, and the EUDAT components are discussed. Tools and approaches for automated data movement and replication in cross-center workflows are explained, as well as the development of ontologies for structuring and quality-checking of metadata in computational biomedicine. The CompBioMed project is used as a real-world example of implementing these principles and tools in practice. The LEXIS project has built a workflow-execution and data management platform that follows the paradigm of HPC-Cloud convergence for demanding Big Data applications. It is used for orchestrating workflows with YORC, utilizing the data documentation initiative (DDI) and distributed computing resources (DCI). The platform is accessed by a user-friendly LEXIS portal for workflow and data management, making HPC and Cloud Computing significantly more accessible. Checkpointing, duplicate runs, and spare images of the data are used to create resilient workflows. The CompBioMed project is completing the implementation of such a workflow, using data replication and brokering, which will enable urgent computing on exascale platforms.

Cloud benchmarking and performance analysis of an HPC application in Amazon EC2

Cloud computing platforms have been continuously evolving. Features such as the Elastic Fabric Adapter (EFA) in the Amazon Web Services (AWS) platform have brought yet another revolution in the High Performance Computing (HPC) world, further accelerating the convergence of HPC and cloud computing. Other public clouds also support similar features further fueling this change. In this paper, we show how and why the performance of a large-scale computational fluid dynamics (CFD) HPC application on AWS competes very closely with the one on Beskow—a Cray XC40 supercomputer at the PDC Center for High-Performance Computing - in terms of cost-efficiency with strong scaling up to 2304 processes. We perform an extensive set of micro and macro benchmarks in both environments and conduct a comparative analysis. Until as recently as 2020 these benchmarks have notoriously yielded unsatisfactory results for the cloud platforms compared with on-premise infrastructures. Our aim is to access the HPC capabilities of the cloud, and in general to demonstrate how researchers can scale and evaluate the performance of their application in the cloud.

The N-shaped partition method: A novel parallel implementation of the Crank Nicolson algorithm

We develop an algorithm to solve tridiagonal systems of linear equations, which appear in implicit finite-difference schemes of partial differential equations (PDEs), being the time-dependent Schrödinger equation (TDSE) an ideal candidate to benefit from it. Our N-shaped partition method optimizes the implementation of the numerical calculation on parallel architectures, without memory size constraints. Specifically, we discuss the realization of our method on graphics processing units (GPUs) and the Message Passing Interface (MPI). In GPU implementations, our scheme is particularly advantageous for systems whose size exceeds the global memory of a single processor. Moreover, because of its lack of memory constraints and the generality of the algorithm, it is well-suited for mixed architectures, typically available in large high performance computing (HPC) centers. We also provide an analytical estimation of the optimal parameters to implement our algorithm, and test numerically the suitability of our formula in a GPU implementation. Our method will be helpful to tackle problems which require large spatial grids for which ab-initio studies might be otherwise prohibitive both because of large shared-memory requirements and computation times.

Overcoming Challenges to Continuous Integration in HPC

Continuous integration (CI) has become a ubiquitous practice in modern software development, with major code hosting services offering free automation on popular platforms. CI offers major benefits, as it enables detecting bugs in code prior to committing changes. While high-performance computing (HPC) research relies heavily on software, HPC machines are not considered "common" platforms. This presents several challenges that hinder the adoption of CI in HPC environments, making it difficult to maintain bug-free HPC projects, and resulting in adverse effects on the research community. In this article, we explore the challenges that impede HPC CI, such as hardware diversity, security, isolation, administrative policies, and non-standard authentication, environments, and job submission mechanisms. We propose several solutions that could enhance the quality of HPC software and the experience of developers. Implementing these solutions would require significant changes at HPC centers, but if these changes are made, it would ultimately enable faster and better science.

Segment Linking: A Highly Parallelizable Track Reconstruction Algorithm for HL-LHC

The High Luminosity upgrade of the Large Hadron Collider (HL-LHC) will produce particle collisions with up to 200 simultaneous proton-proton interactions. These unprecedented conditions will create a combinatorial complexity for charged-particle track reconstruction that demands a computational cost that is expected to surpass the projected computing budget using conventional CPUs. Motivated by this and taking into account the prevalence of heterogeneous computing in cutting-edge High Performance Computing centers, we propose an efficient, fast and highly parallelizable bottom-up approach to track reconstruction for the HL-LHC, along with an associated implementation on GPUs, in the context of the Phase 2 CMS outer tracker. Our algorithm, called Segment Linking (or Line Segment Tracking), takes advantage of localized track stub creation, combining individual stubs to progressively form higher level objects that are subject to kinematical and geometrical requirements compatible with genuine physics tracks. The local nature of the algorithm makes it ideal for parallelization under the Single Instruction, Multiple Data paradigm, as hundreds of objects can be built simultaneously. The computing and physics performance of the algorithm has been tested on an NVIDIA Tesla V100 GPU, already yielding efficiency and timing measurements that are on par with the latest, multi-CPU versions of existing CMS tracking algorithms.

European HPC cloud infrastructure for managing SARS-CoV-2 data in compliance with GDPR

Abstract The Connecting European SARS-CoV-2 Cohorts to Increase Common and Effective Response to SARS-CoV-2 Pandemic (ORCHESTRA) consortium, led by University of Verona (Italy), brings together key European academic experts and research institutions in infectious diseases, data management and High Performance Computing (HPC) from 26 organizations (extending to 37 partners) from 15 countries. The project aims to create a new pan-European cohort built on existing and new large-scale population cohorts in European and non-European countries to significantly impact on the responsiveness to SARS-CoV-2. The integration and analysis of the very heterogeneous characteristics of SARS-CoV-2 data coming from many different sources such as EHR, retrospective and prospective patient registries, and related ‘omics’ data (incl. genomics, proteomics and transcriptomics) can benefit of data analytics enabled by HPC, where both high compute performance and fast storage capabilities are immensely important. During the first year of the project, a dedicated HPC cloud infrastructure have been designed and partially deployed to fulfill the functional requirements for data management ensuring healthcare data confidentiality/privacy, integrity and security in compliance with the European GDPR regulations. The result is an infrastructure for Data Management composed by three main layers: National Data Providers; National Hubs (one for each HPC center involved: CINECA - Italy, CINES - France and HLRS - Germany), to centralize data at national level and to support data storage, sharing and analysis on data ingested from the National Data Providers; ORCHESTRA Data Portal: the pan-European portal for sharing aggregated data and results. Currently data collection is on going; at the end of the project, clinical centers are expected to have enrolled more than 10.000 patients with about 50.000 samples for the prospective studies. Key messages • The SARS-CoV-2 crisis made evident the need to manage and analyse very heterogeneous health data coming from many different resources across different countries. • The HPC cloud infrastructure released for the Orchestra project can act as a model to manage future public health threats.