Extreme-scale Computing Research Articles

Magnetohydrodynamics with physics informed neural operators

The modeling of multi-scale and multi-physics complex systems typically involves the use of scientific software that can optimally leverage extreme scale computing. Despite major developments in recent years, these simulations continue to be computationally intensive and time consuming. Here we explore the use of AI to accelerate the modeling of complex systems at a fraction of the computational cost of classical methods, and present the first application of physics informed neural operators (NOs) (PINOs) to model 2D incompressible magnetohydrodynamics (MHD) simulations. Our AI models incorporate tensor Fourier NOs as their backbone, which we implemented with the TensorLY package. Our results indicate that PINOs can accurately capture the physics of MHD simulations that describe laminar flows with Reynolds numbers . We also explore the applicability of our AI surrogates for turbulent flows, and discuss a variety of methodologies that may be incorporated in future work to create AI models that provide a computationally efficient and high fidelity description of MHD simulations for a broad range of Reynolds numbers. The scientific software developed in this project is released with this manuscript.

Block low‐rank single precision coarse grid solvers for extreme scale multigrid methods

AbstractExtreme scale simulation requires fast and scalable algorithms, such as multigrid methods. To achieve asymptotically optimal complexity, it is essential to employ a hierarchy of grids. The cost to solve the coarsest grid system can often be neglected in sequential computings, but cannot be ignored in massively parallel executions. In this case, the coarsest grid can be large and its efficient solution becomes a challenging task. We propose solving the coarse grid system using modern, approximate sparse direct methods and investigate the expected gains compared with traditional iterative methods. Since the coarse grid system only requires an approximate solution, we show that we can leverage block low‐rank techniques, combined with the use of single precision arithmetic, to significantly reduce the computational requirements of the direct solver. In the case of extreme scale computing, the coarse grid system is too large for a sequential solution, but too small to permit massively parallel efficiency. We show that the agglomeration of the coarse grid system to a subset of processors is necessary for the sparse direct solver to achieve performance. We demonstrate the efficiency of the proposed method on a Stokes‐type saddle point system solved with a monolithic Uzawa multigrid method. In particular, we show that the use of an approximate sparse direct solver for the coarse grid system can outperform that of a preconditioned minimal residual iterative method. This is demonstrated for the multigrid solution of systems of order up to degrees of freedom on a petascale supercomputer using 43,200 processes.

Online data analysis and reduction: An important Co-design motif for extreme-scale computers

A growing disparity between supercomputer computation speeds and I/O rates means that it is rapidly becoming infeasible to analyze supercomputer application output only after that output has been written to a file system. Instead, data-generating applications must run concurrently with data reduction and/or analysis operations, with which they exchange information via high-speed methods such as interprocess communications. The resulting parallel computing motif, online data analysis and reduction (ODAR), has important implications for both application and HPC systems design. Here we introduce the ODAR motif and its co-design concerns, describe a co-design process for identifying and addressing those concerns, present tools that assist in the co-design process, and present case studies to illustrate the use of the process and tools in practical settings.

FALL3D-8.0: a computational model for atmospheric transport and deposition of particles, aerosols and radionuclides – Part 2: Model validation

Abstract. This paper presents model validation results for the latest version release of the FALL3D atmospheric transport model. The code has been redesigned from scratch to incorporate different categories of species and to overcome legacy issues that precluded its preparation towards extreme-scale computing. The model validation is based on the new FALL3D-8.0 test suite, which comprises a set of four real case studies that encapsulate the major features of the model; namely, the simulation of long-range fine volcanic ash dispersal, volcanic SO2 dispersal, tephra fallout deposits and the dispersal and deposition of radionuclides. The first two test suite cases (i.e. the June 2011 Puyehue-Cordón Caulle ash cloud and the June 2019 Raikoke SO2 cloud) are validated against geostationary satellite retrievals and demonstrate the new FALL3D data insertion scheme. The metrics used to validate the volcanic ash and SO2 simulations are the structure, amplitude and location (SAL) metric and the figure of merit in space (FMS). The other two test suite cases (i.e. the February 2013 Mt. Etna ash cloud and associated tephra fallout deposit, and the dispersal of radionuclides resulting from the 1986 Chernobyl nuclear accident) are validated with scattered ground-based observations of deposit load and local particle grain size distributions and with measurements from the Radioactivity Environmental Monitoring database. For validation of tephra deposit loads and radionuclides, we use two variants of the normalised root-mean-square error metric. We find that FALL3D-8.0 simulations initialised with data insertion consistently improve agreement with satellite retrievals at all lead times up to 48 h for both volcanic ash and SO2 simulations. In general, SAL scores lower than 1.5 and FMS scores greater than 0.40 indicate acceptable agreement with satellite retrievals of volcanic ash and SO2. In addition, we show very good agreement, across several orders of magnitude, between the model and observations for the 2013 Mt. Etna and 1986 Chernobyl case studies. Our results, along with the validation datasets provided in the publicly available test suite, form the basis for future improvements to FALL3D (version 8 or later) and also allow for model intercomparison studies.

PETSc DMNetwork

We present DMNetwork, a high-level package included in the PETSc library for the simulation of multiphysics phenomena over large-scale networked systems. The library aims at applications that have networked structures such as those in electrical, gas, and water distribution systems. DMNetwork provides data and topology management, parallelization for multiphysics systems over a network, and hierarchical and composable solvers to exploit the problem structure. DMNetwork eases the simulation development cycle by providing the necessary infrastructure through simple abstractions to define and query the network components. This article presents the design of DMNetwork, illustrates its user interface, and demonstrates its ability to solve multiphysics systems, such as an electric circuit, a network of power grid and water subnetworks, and transient hydraulic systems over large networks with more than 2 billion variables on extreme-scale computers using up to 30,000 processors.

FALL3D-8.0: a computational model for atmospheric transport and deposition of particles, aerosols and radionuclides – Part 1: Model physics and numerics

Abstract. This paper presents FALL3D-8.0, the last version release of an open-source code with 15+ years of track record and a growing number of users in the volcanological and atmospheric communities. The code has been redesigned and rewritten from scratch in the framework of the EU Centre of Excellence for Exascale in Solid Earth (ChEESE) in order to overcome legacy issues and allow for successive optimisations in the preparation of the code towards extreme-scale computing. However, this baseline version already contains substantial improvements in terms of model physics, solving algorithms, and code accuracy and performance. The code, originally conceived for atmospheric dispersal and deposition of tephra particles, has been extended to model other types of particles, aerosols and radionuclides. The solving strategy has also been changed, replacing the former central-difference scheme for a high-resolution central-upwind scheme derived from finite volumes, which minimises numerical diffusion even in the presence of sharp concentration gradients and discontinuities. The parallelisation strategy, input/output (I/O), model pre-process workflows and memory management have also been reconsidered, leading to substantial improvements on code scalability, efficiency and overall capability to handle much larger problems. All these new features and improvements have implications on operational model performance and allow, among others, adding data assimilation and ensemble forecast in future releases. This paper details the FALL3D-8.0 model physics and the numerical implementation of the code.

Exploiting potential of deep neural networks by layer-wise fine-grained parallelism

Deepneuralnetworks (DNNs) have become more and more important for big data analysis. They usually use data parallelism or model parallelism for extreme scale computing. However, the two approaches realize the performance improvement mainly by using coarse-grained parallelization schemes. Neither can fully exploit the potentials of the parallelism of many-core systems (such as GPUs) for neural network models. Here, a new fine−grainedparallelismstrategy (named FiLayer) is presented based on layer-wise parallelization. It has two components: inter-layer parallelism and intra-layer parallelism. The inter-layer parallelism makes several neighboring layers be processed by using a pipeline manner in a network model. For intra-layer parallelism, the operations in one layer are separated into several parts and processed concurrently. To implement above fine-grained parallelism methods, CUDA streams are used. A mathematical analysis is presented for the influence of fragment number on performance of the inter-layer parallelism, and also an analysis for the influence of CUDA stream number on the performance of the intra-layer parallelism is given. The proposed approach is realized based on Caffe. Some representative datasets including CIFAR100 and ImageNet, are applied for experiments. The evaluation results show that it can help Caffe realize remarkable speedups, which makes much sense to big data analysis.

Large-Scale and Extreme-Scale Computing with Stranded Green Power: Opportunities and Costs

Power consumption and associated carbon emissions are increasingly critical challenges for large-scale computing. Recent research proposes exploiting stranded power - uneconomic renewable power - for green supercomputing in a system called Zero-Carbon Cloud (ZCCloud) [1], [2], [3]. These efforts studied production supercomputing workloads on stranded-power based computing resources, demonstrating their achievable productivity. We explore economic viability of stranded-power based supercomputing, using three datacenter total-cost-of-ownership (TCO) models to study cost-effectiveness. These studies show that ZCCloud's approach can be cost-effective in the USA today, and is even more attractive in regions with higher power prices (e.g., Japan, Germany), achieving cost advantages as large as 50 percent. Environmental and power-grid benefits are a further advantage. We also explore the sensitivity of these results to changes in hardware TCO; cheaper hardware or longer lifetimes magnify the attractiveness of stranded-power based approaches, yielding advantages as large as 91 percent. These results are robust across different TCO models. Finally, we study extreme-scale supercomputers (>100 MW), finding stranded-power can increase peak capability per cost by as much as 80 percent.

Research and Education in Computational Science and Engineering

This report presents challenges, opportunities and directions for computational science and engineering (CSE) research and education for the next decade. Over the past two decades the field of CSE has penetrated both basic and applied research in academia, industry, and laboratories to advance discovery, optimize systems, support decision-makers, and educate the scientific and engineering workforce. Informed by centuries of theory and experiment, CSE performs computational experiments to answer questions that neither theory nor experiment alone is equipped to answer. CSE provides scientists and engineers with algorithmic inventions and software systems that transcend disciplines and scales. CSE brings the power of parallelism to bear on troves of data. Mathematics-based advanced computing has become a prevalent means of discovery and innovation in essentially all areas of science, engineering, technology, and society; and the CSE community is at the core of this transformation. However, a combination of disruptive developments-including the architectural complexity of extreme-scale computing, the data revolution and increased attention to data-driven discovery, and the specialization required to follow the applications to new frontiers-is redefining the scope and reach of the CSE endeavor. With these many current and expanding opportunities for the CSE field, there is a growing demand for CSE graduates and a need to expand CSE educational offerings. This need includes CSE programs at both the undergraduate and graduate levels, as well as continuing education and professional development programs, exploiting the synergy between computational science and data science. Yet, as institutions consider new and evolving educational programs, it is essential to consider the broader research challenges and opportunities that provide the context for CSE education and workforce development.

Education

This issue of SIAM Review presents “Research and Education in Computational Science and Engineering” in the Education section. This is a collaborative report, coauthored by U. Rüde, K. Willcox, L. Curfman McInnes, and H. De Sterck, surveying current challenges and future directions for computational science and engineering research and education. Twenty-nine other scientists and educators have contributed to this survey representing the views of the SIAM Activity Group on Computational Science and Engineering. The article starts with a definition of computational science and engineering (CSE) as a multidisciplinary field integrating mathematics, statistics, computer science, and core disciplines of science and engineering. Its subject is the development and use of computational methods for scientific discovery. The first part of the article makes us aware of the large impact of CSE on science, technology, and our society in general. The emergence of this new field and its advance cannot be taken for granted; it requires thoughtful design of educational programs integrating knowledge and research in interacting areas, keeping a forward view on strategically important directions. The impact of CSE is amplified by the growing presence of massive data sets and increasing computing power which enable data-driven approaches to scientific discovery. Throughout the article, the authors highlight several examples of recent success stories such as computational medicine, computer animation, CSE in the petroleum industry, climate research, and others, which demonstrate the key role of CSE theory, methods, and software in modern science and industry. The second part of the article outlines directions where new challenges appear and substantial advances would bring a qualitative leap in CSE. Mathematical models with increased robustness and stability, algorithms of better efficiency and smaller complexity, advances in nonsmooth optimization and optimal control, and uncertainty quantification for high-fidelity prediction are among the key areas discussed. Next, the report addresses the needs in education and research, the scope and specificity of educational programs which will bring forth a capable and agile workforce. The authors note that almost all sectors of industry, the national laboratories, and the government are using more sophisticated predictive models, high-end computing, and mathematical model-aided decision-making more frequently. Graduates with interdisciplinary expertise are in very high demand due to the need to develop and utilize advanced mathematical models and computational methods in their respective scientific, engineering, or business fields in order to “give the business the ability to compete and survive.” Here the authors cite statements of the National Council on Competitiveness and the U.S. Department of Energy Advanced Scientific Computing Advisory Committee. CSE educational programs have to address the growing demand for CSE professionals at both undergraduate and graduate levels, as well as provide possibilities for continuing education and professional development. Specific recommendations for the content of undergraduate programs are included in section 3.2 of the article. Substantial attention is paid to the set of learning outcomes that a CSE graduate program should target. The authors predict a growing demand for terminal master's degrees for the needs of industry and national laboratories. The education of CSE professionals should include three main components defined as follows. The graduate should possess a solid foundation in applied mathematics, including probability and statistics; should have an understanding of modern computing, computer science, programming languages, principles of software engineering, and high-performance computing; and should have good knowledge of modern science and engineering. Of course, the breadth and depth of topics covered would depend on the focus of the specific degree. The authors distinguish between the following categories in CSE preparation: CSE domain scientists and engineers preparing themselves as method users or as method developers; core researchers and developers focusing on broadly applicable methods; and scientists preparing to deal with uncertainty quantification and data science. These roles determine essential aspects of the educational programs. The report contains also a discussion on the role and interaction of two areas within CSE education: parallel extreme-scale computing and computing with massive data. The authors also elaborate on software sustainability and data management, which become more involved as software is frequently offered by geographically distributed developers. Growing concern about reproducibility of scientific results based on code and data generates additional demand for skilled professionals. Therefore, on one hand CSE education should account for these needs; on the other hand, those areas offer new research opportunities. In the final part of the third chapter, some thoughts on the changing educational infrastructure are shared: online digital materials and open educational resources including open online courses provide new ways and opportunities to enhance the learning process. The report concludes with findings and recommendations regarding the emergence of CSE, its role, and education.

Algorithm-based fault recovery of adaptively refined parallel multilevel grids

On future extreme scale computers, it is expected that faults will become an increasingly serious problem as the number of individual components grows and failures become more frequent. This is driving the interest in designing algorithms with built-in fault tolerance that can continue to operate and that can replace data even if part of the computation is lost in a failure. For fault-free computations, the use of adaptive refinement techniques in combination with finite element methods is well established. Furthermore, iterative solution techniques that incorporate information about the grid structure, such as the parallel geometric multigrid method, have been shown to be an efficient approach to solving various types of partial different equations. In this article, we present an advanced parallel adaptive multigrid method that uses dynamic data structures to store a nested sequence of meshes and the iteratively evolving solution. After a fail-stop fault, the data residing on the faulty processor will be lost. However, with suitably designed data structures, the neighbouring processors contain enough information so that a consistent mesh can be reconstructed in the faulty domain with the goal of resuming the computation without having to restart from scratch. This recovery is based on a set of carefully designed distributed algorithms that build on the existing parallel adaptive refinement routines, but which must be carefully augmented and extended.

A marshalled data format for pointers in relocatable data blocks

As future computing hardware progresses towards extreme-scale technology, new challenges arise for addressing heterogeneous compute and memory resources, for providing application resilience in the presence of more frequent failures, and for working within strict energy constraints. While C++ has gained popularity in recent years within the HPC community, some concepts of object-oriented program design may be at odds with the techniques we use to address the challenges of extreme-scale computing. In this work, we focus on the challenges related to using aggregate data structures that include pointer values within a programming model where the runtime may frequently relocate data, and traditional serialization techniques are not practical. We propose and evaluate a marshalled encoding for relocatable data blocks, and present a C++ library and other tools to simplify the work of the application programmer developing new applications or porting existing applications to such emerging programming models.

Soft real-time analysis of ITER magnetics streaming data using SPECTRE

The SAGE [1] European Horizon 2020 project (grant agreement 671500), led by Seagate with 10 partners, is investigating solutions of future exascale storage systems for data intensive applications. CCFE is one of the partners and SPECTRE (SPECtral Research Engine) is one of the tools being developed to take advantage of the improved data I/O and throughput capability of the SAGE platform.SPECTRE is designed to emulate and process the levels of data acquisition required to ingest magnetics data during ITER operations. ITER will have about 200 magnetics sources each generating data at a rate ∼40MB/s giving an input of ∼8GB/s for the whole pulse [5,6]. A typical pulse will last hundreds of seconds and could create up to 1 TB of magnetics data.Typically magnetics data are stored and processed off-line between pulses to generate diagnostic data vital for the preparation of the next experiment. SPECTRE, using streaming technologies such as Apache Flink or Apache Storm in concert with the SAGE platform, will greatly reduce the time needed to process this diagnostic data.In addition to the significant inter-pulse analysis speed up using SAGE and ‘Big Data & Extreme-Scale Computing’ (BDEC) technologies, it is possible to provide soft real-time (by which we mean as close to real-time as possible) diagnostics from live streaming data – an essential aid for experiment ROs in running and guiding the pulse.Synthetic data generated at levels matching those of ITER have been used to stress test SPECTRE and demonstrate the potential benefits of the SAGE platform and related technologies. Additionally, real data generated using prototype ITER data acquisition hardware being developed by F4E, CCFE and others [2], will be analysed.

A characterization of workflow management systems for extreme-scale applications

Automation of the execution of computational tasks is at the heart of improving scientific productivity. Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications. By allowing scientists to model and express entire data processing steps and their dependencies, workflow management systems relieve scientists from the details of an application and manage its execution on a computational infrastructure. As the resource requirements of today’s computational and data science applications that process vast amounts of data keep increasing, there is a compelling case for a new generation of advances in high-performance computing, commonly termed as extreme-scale computing, which will bring forth multiple challenges for the design of workflow applications and management systems. This paper presents a novel characterization of workflow management systems using features commonly associated with extreme-scale computing applications. We classify 15 popular workflow management systems in terms of workflow execution models, heterogeneous computing environments, and data access methods. The paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems.

SIERRA—Simulation environment for memory redundancy algorithms

Extreme-scale computer systems take advantage of large arrays of general-purpose multicore processors coupled with specialized manycore accelerators. In order to support complex applications and correctly feed such processing elements, increasingly larger memory cores are integrated at different levels of the hierarchy. However, the adoption of increasingly aggressive manufacturing processes makes the memory sub-system particularly sensitive to faults. Error correcting codes (ECCs) allow the memory to recover from faults at run-time without interfering with the application execution. However, due to the loss of performance introduced every time an error must be corrected, the persistence of faults requires a more radical repair approach in which faulty cells are physically replaced by spare ones. Memory redundancy analysis (MRA) algorithms are used to drive the allocation process of spare resources. Many one-dimensional and two-dimensional MRAs have been proposed, but tools for evaluating their recovering capability are still not well established. This paper presents SIERRA, a simulation environment for precisely evaluating the repair efficiency of an MRA considering different fault signatures and faulty memory configurations. Our simulation engine provides a precise estimation of the MRA quality by analyzing the behavior of the MRA on several faulty memory configurations. To this end, different parameters such as the area of the memory blocks and the defect density are taken into account. The evaluation of the quality of an MRA takes into account its repairing capability, the power consumption derived from its execution, and the area overhead. Thanks to the use of a database for storing information, our tool is able to speed-up the simulation process by distributing it among several nodes. All these features make SIERRA essential in supporting the design of next-generation high-performance computers.

Efficient clustering for ultra-scale application tracing

Extreme-scale computing poses a number of challenges to application performance. Developers need to study application behavior by collecting detailed information with the help of tracing toolsets to determine shortcomings. But not only applications are “scalability challenged”, current tracing toolsets also fall short of exascale requirements for low background overhead since trace collection for each execution entity is becoming infeasible. One effective solution is to cluster processes with the same behavior into groups. Instead of collecting performance information from each individual node, this information can be collected from just a set of representative nodes. This work contributes a fast, scalable, signature-based clustering algorithm that clusters processes exhibiting similar execution behavior. Instead of prior work based on statistical clustering, our approach produces precise results nearly without loss of events or accuracy. The proposed algorithm combines low overhead at the clustering level with log(P) time complexity, and it splits the merge process to make tracing suitable for extreme-scale computing. Overall, this multi-level precise clustering based on signatures further generalizes to a novel multi-metric clustering technique with unprecedented low overhead.

The MPO system for automatic workflow documentation

Data from large-scale experiments and extreme-scale computing is expensive to produce and may be used for critical applications. However, it is not the mere existence of data that is important, but our ability to make use of it. Experience has shown that when metadata is better organized and more complete, the underlying data becomes more useful. Traditionally, capturing the steps of scientific workflows and metadata was the role of the lab notebook, but the digital era has resulted instead in the fragmentation of data, processing, and annotation. This paper presents the Metadata, Provenance, and Ontology (MPO) System, the software that can automate the documentation of scientific workflows and associated information. Based on recorded metadata, it provides explicit information about the relationships among the elements of workflows in notebook form augmented with directed acyclic graphs. A set of web-based graphical navigation tools and Application Programming Interface (API) have been created for searching and browsing, as well as programmatically accessing the workflows and data. We describe the MPO concepts and its software architecture. We also report the current status of the software as well as the initial deployment experience.

Unite and conquer approach for high scale numerical computing

The ability to exploit emerging exascale computational systems will require a careful review and redesign of core numerical algorithms and their implementations to fully exploit multiple levels of concurrency, hierarchical memory structures and heterogeneous processing units that will become available in these computational platforms. This paper presents the “unite and conquer” approach to solve linear systems of equations and eigenvalue problems for extreme scale computing. Indeed, there are two ways to optimize the execution of a restarted method on a large-scale distributed system. The first one is to optimize the number of floating point operations per restart cycle through maximizing the concurrency inside a restart cycle while minimizing latencies. The second way is to accelerate/improve the rate of convergence for a given computational scheme. The unite and conquer restarted approach focuses on decreasing the number of restart cycles by coupling either synchronously or asynchronously several restarted methods called also co-methods. In the end of a restart cycle, each co-method locally gathers available results of all collaborating co-methods and selects the best one in order to create its restarting information. Consequently this permits the global reduction of the number of cycles to convergence. The unite and conquer restarted methods are heterogeneous, fault tolerant, support asynchronous communications and present a big potential of load balancing. Due to these properties, they are well adapted to large-scale multi-level parallel architectures. We show the relevant programming paradigms that allow multi-level parallel expression of these methods and how the software engineering technology can contribute significantly in achieving high scalability. We present some experiments validating the approach for unite and conquer restarted Krylov methods on several parallel and distributed platforms.

Analysis and Practical Use of Flexible BiCGStab

A flexible version of the BiCGStab algorithm for solving a linear system of equations is analyzed. We show that under variable preconditioning, the perturbation to the outer residual norm is of the same order as that to the application of the preconditioner. Hence, in order to maintain a similar convergence behavior to BiCGStab while reducing the preconditioning cost, the flexible version can be used with a moderate tolerance in the preconditioning Krylov solves. We explored the use of flexible BiCGStab in a large-scale reacting flow application, PFLOTRAN, and showed that the use of a variable multigrid preconditioner significantly accelerates the simulation time on extreme-scale computers using $$O(10^4)$$O(104)---$$O(10^5)$$O(105) processor cores.

Extreme-scale computer architecture

As transistor sizes continue to scale down, we are about to witness extraordinary levels of chip integration. Sometime early in the next decade, as we reach 7 nm, we will be able to integrate, for example, 1000 sizable cores and substantial memory on a single die. There are many unknowns as to how to build a general-purpose architecture in such an environment. However, we know that the main challenge will be to make it highly energy efficient. Energy and power consumption have emerged as the main obstacles to designing more capable architectures. Given this energy efficiency challenge, researchers have coined the term ‘Extreme Scale Computer Architecture’ to refer to computer organizations that, loosely speaking, are 100–1000 times more capable than current systems for the same power consumption and physical footprint. For example, these organizations should deliver a datacenter that provides exascale performance (1018 operations per second) for 20 MW, a departmental server that provides petascale performance (1015 operations per second) for 20 KW, and a portable device that provides sustained terascale performance (1012 operations per second) for 20 W. Extreme-scale computing is concerned with technologies that are applicable to all machine sizes—not just highend systems. Extreme-scale computer architectures need to be designed for energy efficiency from the ground up. They need to have efficient support for concurrency, since onlymassive parallelismwill deliver this performance. They should also minimize data transfers—since moving data around is a major source of energy consumption. Finally, they need to leverage new technologies that will be developed in the next few years. These technologies include low supply voltage (Vdd) operation, 3D die stacking, resistive memories, and photonic interconnects. In this paper, we outline some of challenges that appear at different layers of the computing stack, and some of the techniques that can be used to address them.