Compute-intensive Workloads Research Articles

GenArchBench: A genomics benchmark suite for arm HPC processors

Arm usage has substantially grown in the High-Performance Computing (HPC) community. Japanese supercomputer Fugaku, powered by Arm-based A64FX processors, held the top position on the Top500 list between June 2020 and June 2022, currently sitting in the fourth position. The recently released 7th generation of Amazon EC2 instances for compute-intensive workloads (C7 g) is also powered by Arm Graviton3 processors. Projects like European Mont-Blanc and U.S. DOE/NNSA Astra are further examples of Arm irruption in HPC. In parallel, over the last decade, the rapid improvement of genomic sequencing technologies and the exponential growth of sequencing data has placed a significant bottleneck on the computational side. While most genomics applications have been thoroughly tested and optimized for x86 systems, just a few are prepared to perform efficiently on Arm machines. Moreover, these applications do not exploit the newly introduced Scalable Vector Extensions (SVE).This paper presents GenArchBench, the first genome analysis benchmark suite targeting Arm architectures. We have selected computationally demanding kernels from the most widely used tools in genome data analysis and ported them to Arm-based A64FX and Graviton3 processors. Overall, the GenArch benchmark suite comprises 13 multi-core kernels from critical stages of widely-used genome analysis pipelines, including base-calling, read mapping, variant calling, and genome assembly. Our benchmark suite includes different input data sets per kernel (small and large), each with a corresponding regression test to verify the correctness of each execution automatically. Moreover, the porting features the usage of the novel Arm SVE instructions, algorithmic and code optimizations, and the exploitation of Arm-optimized libraries. We present the optimizations implemented in each kernel and a detailed performance evaluation and comparison of their performance on four different HPC machines (i.e., A64FX, Graviton3, Intel Xeon Skylake Platinum, and AMD EPYC Rome). Overall, the experimental evaluation shows that Graviton3 outperforms other machines on average. Moreover, we observed that the performance of the A64FX is significantly constrained by its small memory hierarchy and latencies. Additionally, as proof of concept, we study the performance of a production-ready tool that exploits two of the ported and optimized genomic kernels.

Exploring Instruction Set Architectural Variations: x86, ARM, and RISC-V in Compute-Intensive Applications

As computational demands continue to evolve in the modern era, the choice of hardware architecture plays a pivotal role in optimizing the performance of compute-intensive applications. This research paper delves into the exploration and comparison of three prominent hardware architectures: x86, ARM, and RISC-V, within the context of compute-intensive applications. The study begins with a comprehensive overview of these architectures, highlighting their distinctive features, and strengths. Subsequently, we investigate their suitability and adaptability in diverse compute-intensive workloads. Our analysis encompasses a wide spectrum of parameters, including computational throughput, power efficiency, scalability, and architectural flexibility. We scrutinize the architectural intricacies that impact the execution of compute-intensive tasks, shedding light on both the advantages and limitations of each architecture. We used the gem5 simulator to compare these Instruction Set Architectures (ISA). We run different benchmarks on gem5 with different ISA and different configurations and compare the result. Based on these results we predict which architecture is better in which scenario. Gem5 is not a cycle accurate simulator but it’s a model accurate. In conclusion, "Exploring Architectural Variations: x86, ARM, and RISC-V in Compute-Intensive Applications" offers a comprehensive insight into the nuances of hardware selection for compute-intensive workloads. Our findings aid system architects, researchers, and technology enthusiasts in making informed decisions about the most suitable architectural choice for their specific computeintensive applications, ultimately contributing to advancements in computational performance and efficiency

Serverless High-Performance Computing over Cloud

Abstract HPC clouds may provide fast access to fully configurable and dynamically scalable virtualized HPC clusters to address the complex and challenging computation and storage-intensive requirements. The complex environmental, software, and hardware requirements and dependencies on such systems make it challenging to carry out our large-scale simulations, prediction systems, and other data and compute-intensive workloads over the cloud. The article aims to present an architecture that enables HPC workloads to be serverless over the cloud (Shoc), one of the most critical cloud capabilities for HPC workloads. On one hand, Shoc utilizes the abstraction power of container technologies like Singularity and Docker, combined with the scheduling and resource management capabilities of Kubernetes. On the other hand, Shoc allows running any CPU-intensive and data-intensive workloads in the cloud without needing to manage HPC infrastructure, complex software, and hardware environment deployments.

EXTREM-EDGE—EXtensions To RISC-V for Energy-efficient ML inference at the EDGE of IoT

Artificial intelligence (AI) and machine learning (ML) have emerged as the fastest growing workloads ranging from applications like object detection, natural language processing and facial recognition to self-driving cars. The proliferation of these compute-intensive workloads resulted in numerous hardware accelerators to fill the gap between the performance and energy-efficiency requirements of AI applications and the capabilities of current architectures like CPU and GPU. In most cases these accelerators are specialized for a particular task, are costly to produce, require special programming tools, and can become obsolete as new ML algorithms are introduced. To solve these problems, we present EXTREM-EDGE, a hardware/software co-design approach to add custom extensions to the open-source RISC-V Instruction Set architecture (ISA) for designing a scalable and flexible ML processor architecture. EXTREM-EDGE augments the RISC-V processor with hardware AI functional units (AFU) along with ISA extensions which directly target these AFUs. EXTREM-EDGE is a system-level solution which is easy to program, enables royalty-free production and provides flexibility for future workloads. It enables the designers to quickly adapt to any hardware or ISA/software changes and allows the design-space exploration of various available hardware, instructions and software options. This enables a processor architecture which addresses the requirements of current AI/ML workloads, gives the flexibility to hot-swap AFUs when better hardware is available and scales with new AI instructions in response to rapidly evolving AI algorithms while providing a streamlined development flow for both hardware and software. EXTREM-EDGE provides 1.75x (MAC) to 17.63x (PIM VMM) performance improvements for a GEMV kernel and 1.41x (MAC) to 4.41x (PIM VMM) reductions in processor clock cycles for ResNet-8 neural network model from MLPerf Tiny benchmark depending upon the size of added accelerators and complexity of added instructions.

An Energy-Efficient 3D Cross-Ring Accelerator With 3D-SRAM Cubes for Hybrid Deep Neural Networks

Hybrid deep neural networks (DNNs) are utilized for various embedded applications and composed of convolution layers, fully connected layers and recurrent layers. However, hybrid-DNNs are difficult to be fully deployed to edge devices due to both memory-intensive and computation-intensive workloads. Additionally, the resource utilization of a hybrid-DNN accelerator is less than those of CNN accelerators according to various computation kernels and dataflows. Fortunately, 3D-SRAM cubes by TSV 3D-stacking technologies provide promising solutions and become feasible for on-device DNN accelerators. In this paper, a flexible interconnect architecture, 3D cross-ring, and an efficient dataflow are proposed with 3D-SRAM cubes for an energy-efficient hybrid-DNN accelerator. Micro-routers of 3D cross-rings are designed to decrease the power of on-chip data movement about <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5\times $ </tex-math></inline-formula> compared to conventional 3D routers. Moreover, the efficient dataflow with dynamic workload distribution is designed based on 3D cross-rings for supporting different NN layers and models. The proposed accelerator can achieve higher than 90% PE utilization and reduce the DRAM accesses about <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$6\times $ </tex-math></inline-formula> on different DNN models. This accelerator improves the overall energy efficiency up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$17.4\times $ </tex-math></inline-formula> on VGG-16 compared to other state-of-art CNN accelerators.

Enhancing Performance and Energy Efficiency for Hybrid Workloads in Virtualized Cloud Environment

Virtualization has attained mainstream status in enterprise IT industry. Despite its widespread adoption, it is known that virtualization also introduces non-trivial overhead when tasks are executed on a virtual machine (VM). In particular, a combined effect from device virtualization overhead and CPU scheduling latency can cause performance degradation when computation intensive tasks and I/O intensive tasks are co-located on a VM. Such an interference also causes extra energy consumption. In this paper, we present Hylics, a novel solution that enables efficient data traverse paths for both I/O and computation intensive workloads. This is achieved with the provision of in-memory file system and network service at the hypervisor level. Several important design issues are pinpointed and addressed during our prototype implementation, including efficient intermediate data sharing, network service offloading, and QoS-aware memory usage management. Based on our real-world deployment on KVM, we show that Hylics can significantly improve computation and I/O performance for hybrid workloads. Moreover, this design also alleviates the existing virtualization overhead and naturally optimizes the overall energy efficiency.

Energy-Efficient Accelerator Design With Tile-Based Row-Independent Compressed Memory for Sparse Compressed Convolutional Neural Networks

Deep convolutional neural networks (CNNs) are difficult to be fully deployed to edge devices because of both memory-intensive and computation-intensive workloads. The energy efficiency of CNNs is dominated by convolution computation and off-chip memory (DRAM) accesses, especially for DRAM accesses. In this article, an energy-efficient accelerator is proposed for sparse compressed CNNs by reducing DRAM accesses and eliminating zero-operand computation. Weight compression is utilized for sparse compressed CNNs to reduce the required memory capacity/bandwidth and a large portion of connections. Thus, a tile-based row-independent compression (TRC) method with relative indexing memory is adopted for storing none-zero terms. Additionally, the workloads are distributed based on channels to increase the degree of task parallelism, and all-row-to-all-row non-zero element multiplication is adopted for skipping redundant computation. The simulation results over the dense accelerator show that the proposed accelerator achieves $1.79\times$ speedup and reduces 23.51%, 69.53%, 88.67% on-chip memory size, energy, and DRAM accesses of VGG-16.

Implementing Practical DNN-Based Object Detection Offloading Decision for Maximizing Detection Performance of Mobile Edge Devices

In the last decade, deep neural network (DNN)-based object detection technologies have received significant attention as a promising solution to implement a variety of image understanding and video analysis applications on mobile edge devices. However, the execution of computationally intensive DNN-based object detection workloads in mobile edge devices is insufficient in fulfilling the object detection requirements with high accuracy and low latency, owing to the limited computation capacity. In this paper, we implement and evaluate a DNN-based object detection offloading framework to improve the object detection performance of mobile edge devices by offloading computation-intensive workloads to a remote edge server. However, preliminary experimental results have shown that offloading all object detection workloads of mobile edge devices may lead to worse performance than executing the workloads locally. This degradation is obtained from the inefficient resource utilization in the edge computing architectures, both for the edge server and mobile edge devices. To resolve the aforementioned problem with degradation, we devise a device-aware DNN offloading decision algorithm that is aimed to maximize resource utilization in the edge computing architecture. The proposed algorithm decides whether or not to offload the object detection workloads of edge devices by considering their computing power and network bandwidth, and therefore maximizing their average object detection processing frames per second. Through various experiments conducted in a real-life wireless local area network (WLAN) environment, we verified the effectiveness of the proposed DNN-based object detection offloading framework.

Development of benchmark automation suite and evaluation of various high-performance computing systems

This study aimed to develop a dynamic benchmark automation suite to measure a range of benchmark performances and evaluate the various high-performance computing (HPC) systems. Our suite supports an automated scaling test and profiling data based on hardware performance counters to analyze the system characteristics. We selected four HPC benchmarks—STREAM, High-Performance Linpack, High-Performance Conjugate Gradient, and NAS Parallel Benchmark-for experiments and configured testbeds based on five different systems and an Intel Knights Landing (KNL) cluster with 16 nodes. The Intel KNL system showed both unstable memory and high benchmark performances for a specific input range. Modern Intel systems also exhibited proper characteristics on compute-intensive workloads, whereas the up-to-date AMD system showed high efficiency and proper characteristics on memory-intensive and real application workloads. We also verified that each system has an optimal environment and characteristic for various combinations of experimental variables and profiling data.

ASIC clouds

Planet-scale applications are driving the exponential growth of the Cloud, and datacenter specialization is the key enabler of this trend. GPU- and FPGA-based clouds have already been deployed to accelerate compute-intensive workloads. ASIC-based clouds are a natural evolution as cloud services expand across the planet. ASIC Clouds are purpose-built datacenters comprised of large arrays of ASIC accelerators that optimize the total cost of ownership (TCO) of large, high-volume scale-out computations. On the surface, ASIC Clouds may seem improbable due to high NREs and ASIC inflexibility, but large-scale ASIC Clouds have already been deployed for the Bitcoin cryptocurrency system. This paper distills lessons from these Bitcoin ASIC Clouds and applies them to other large scale workloads such as YouTube-style video-transcoding and Deep Learning, showing superior TCO versus CPU and GPU. It derives Pareto-optimal ASIC Cloud servers based on accelerator properties, by jointly optimizing ASIC architecture, DRAM, motherboard, power delivery, cooling, and operating voltage. Finally, the authors examine the impact of ASIC NRE and when it makes sense to build an ASIC Cloud.

Priority-Based PCIe Scheduling for Multi-Tenant Multi-GPU Systems

Multi-GPU systems are widely used in data centers to provide significant speedups to compute-intensive workloads such as deep neural network training. However, limited PCIe bandwidth between the CPU and multiple GPUs becomes a major performance bottleneck. We observe that relying on a traditional Round-Robin-based PCIe scheduling policy can result in severe bandwidth competition and stall the execution of multiple GPUs. In this article, we propose a priority-based scheduling policy which aims to overlap the data transfers and GPU execution for different applications to alleviate this bandwidth contention. We also propose a dynamic priority policy for semi-QoS management that can help applications to meet QoS requirements and improve overall multi-GPU system throughput. Experimental results show that the system throughput is improved by 7.6 percent on average using our priority-based PCIe scheduling scheme as compared with a Round-Robin-based PCIe scheduler. Leveraging semi-QoS management can help to meet defined QoS goals, while preserving application throughput.

Dynamic multi-user computation offloading for wireless powered mobile edge computing

The combination of mobile edge computing (MEC) and wireless energy transfer (WET) has been recognized as a prominent solution to liberate massive wireless devices from limited battery capacities and compute-intensive workloads in the Internet of Things (IoT) environment. This new paradigm is called wireless powered mobile edge computing (WP-MEC). In this paper, an online computation rate maximization (OCRM) algorithm for multi-user WP-MEC systems is proposed by jointly managing the radio, computational resources, allocating time for energy transfer and data transmission. In each time block, the optimal bandwidth allocation and data transmit powers for offloading are achieved by Bisection method with linear time complexity. In addition, the closed-form solutions of optimal time allocation and CPU-cycle frequencies of wireless devices are obtained. The theoretical performance analysis shows that the time average sum computation rate and task queue backlog obey an O1/V,OV tradeoff. Numerical results verify that, our proposed OCRM algorithm outperforms the baseline schemes without joint optimization, which dramatically improves the computation rate while making the system stable.

Contention-Aware Fair Scheduling for Asymmetric Single-ISA Multicore Systems

Asymmetric single-ISA multicore processors (AMPs), which integrate high-performance big cores and low-power small cores, were shown to deliver higher performance per watt than symmetric multicores. Previous work has demonstrated that the OS scheduler plays an important role in realizing the potential of AMP systems. While throughput optimization on AMPs has been extensively studied, delivering fairness on these platforms still constitutes an important challenge to the OS. To this end, the scheduler must be equipped with a mechanism enabling to accurately track the progress that each application in the workload makes as it runs on the various core types throughout the execution. In turn, this progress largely depends on the benefit (or speedup) that an application derives on a big core relative to a small one, which may differ greatly across applications. While existing fairness-aware schedulers take application relative speedup into consideration when tracking progress, they do not cater to the performance degradation that may occur naturally due to contention on shared resources among cores, such as the last-level cache or the memory bus. In this paper, we propose CAMPS, a contention-aware fair scheduler for AMPs that primarily targets long-running compute-intensive workloads. Unlike other schemes, CAMPS does not require special hardware extensions or platform-specific speedup-prediction models to function. Our experimental evaluation, which leverages real asymmetric hardware and scheduler implementations in the Linux kernel, demonstrates that CAMPS improves fairness by up to 11 percent with respect to a state-of-the-art fairness-aware OS-level scheme, while delivering better system throughput.

Morpheus

In modern computing systems, object deserialization can become a surprisingly important bottleneck-in our test, a set of generalpurpose, highly parallelized applications spends 64% of total execution time deserializing data into objects. This paper presents the Morpheus model, which allows applications to move such computations to a storage device and bypass the overhead on the host system. We use this model to deserialize data into application objects inside storage devices, rather than in the host CPU. Using the Morpheus model for object deserialization avoids unnecessary system overheads, frees up scarce CPU and main memory resources for compute-intensive workloads, saves I/O bandwidth, and reduces power consumption. In heterogeneous, coprocessor- equipped systems, Morpheus allows application objects to be sent directly from a storage device to a co-processor (e.g., a GPU) by peer-to-peer transfer, further improving application performance as well as reducing the CPU and main memory utilizations. This paper implements Morpheus-SSD, an SSD supporting the Morpheus model. Morpheus-SSD improves the performance of object deserialization by 1.66x, reduces power consumption by 7%, uses 42% less energy, and speeds up the total execution time by 1.32x. By using NVMe-P2P that realizes peer-to-peer communication between Morpheus-SSD and a GPU, Morpheus-SSD can speed up the total execution time by 1.39x in a heterogeneous computing platform.

A Quantitative Evaluation of Contemporary GPU Simulation Methodology

Contemporary Graphics Processing Units (GPUs) are used to accelerate highly parallel compute workloads. For the last decade, researchers in academia and industry have used cycle-level GPU architecture simulators to evaluate future designs. This paper performs an in-depth analysis of commonly accepted GPU simulation methodology, examining the effect both the workload and the choice of instruction set architecture have on the accuracy of a widely-used simulation infrastructure, GPGPU-Sim. We analyze numerous aspects of the architecture, validating the simulation results against real hardware. Based on a characterized set of over 1700 GPU kernels, we demonstrate that while the relative accuracy of compute-intensive workloads is high, inaccuracies in modeling the memory system result in much higher error when memory performance is critical. We then perform a case study using a recently proposed GPU architecture modification, Cache-Conscious Wavefront Scheduling. The case study demonstrates that the cross-product of workload characteristics and instruction set architecture choice can affect the predicted efficacy of the technique.

A Quantitative Evaluation of Contemporary GPU Simulation Methodology

Contemporary Graphics Processing Units (GPUs) are used to accelerate highly parallel compute workloads. For the last decade, researchers in academia and industry have used cycle-level GPU architecture simulators to evaluate future designs. This paper performs an in-depth analysis of commonly accepted GPU simulation methodology, examining the effect both the workload and the choice of instruction set architecture have on the accuracy of a widely-used simulation infrastructure, GPGPU-Sim. We analyze numerous aspects of the architecture, validating the simulation results against real hardware. Based on a characterized set of over 1700 GPU kernels, we demonstrate that while the relative accuracy of compute-intensive workloads is high, inaccuracies in modeling the memory system result in much higher error when memory performance is critical. We then perform a case study using a recently proposed GPU architecture modification, demonstrating that the cross-product of workload characteristics and instruction set architecture choice can have an affect on the predicted efficacy of the technique.

A Holistic Approach for Collaborative Workload Execution in Volunteer Clouds

The demand for provisioning, using, and maintaining distributed computational resources is growing hand in hand with the quest for ubiquitous services. Centralized infrastructures such as cloud computing systems provide suitable solutions for many applications, but their scalability could be limited in some scenarios, such as in the case of latency-dependent applications. The volunteer cloud paradigm aims at overcoming this limitation by encouraging clients to offer their own spare, perhaps unused, computational resources. Volunteer clouds are thus complex, large-scale, dynamic systems that demand for self-adaptive capabilities to offer effective services, as well as modeling and analysis techniques to predict their behavior. In this article, we propose a novel holistic approach for volunteer clouds supporting collaborative task execution services able to improve the quality of service of compute-intensive workloads. We instantiate our approach by extending a recently proposed ant colony optimization algorithm for distributed task execution with a workload-based partitioning of the overlay network of the volunteer cloud. Finally, we evaluate our approach using simulation-based statistical analysis techniques on a workload benchmark provided by Google. Our results show that the proposed approach outperforms some traditional distributed task scheduling algorithms in the presence of compute-intensive workloads.

Replicated Computations Results (RCR) Report for “A Holistic Approach for Collaborative Workload Execution in Volunteer Clouds”

“A Holistic Approach for Collaborative Workload Execution in Volunteer Clouds” [3] proposes a novel approach to task scheduling in volunteer clouds. Volunteer clouds are decentralized cloud systems based on collaborative task execution, where clients voluntarily share their own unused computational resources. By using simulation-based statistical analysis techniques—in particular, statistical model checking—the authors show that their approach can outperform existing distributed task scheduling algorithms in the case of computation-intensive workloads. The analysis considered a realistic workload benchmark provided by Google. This replicated computations results report focuses on the prototypical tool implementation used in the article to perform such analysis. The software was straightforward to install and use, and a representative part of the experimental results from the article could be reproduced in reasonable time using a standard laptop.

Moonwalk

Cloud services are becoming increasingly globalized and data-center workloads are expanding exponentially. GPU and FPGA-based clouds have illustrated improvements in power and performance by accelerating compute-intensive workloads. ASIC-based clouds are a promising way to optimize the Total Cost of Ownership (TCO) of a given datacenter computation (e.g. YouTube transcoding) by reducing both energy consumption and marginal computation cost. The feasibility of an ASIC Cloud for a particular application is directly gated by the ability to manage the Non-Recurring Engineering (NRE) costs of designing and fabricating the ASIC, so that it is significantly lower (e.g. 2X) than the TCO of the best available alternative. In this paper, we show that technology node selection is a major tool for managing ASIC Cloud NRE, and allows the designer to trade off an accelerator's excess energy efficiency and cost performance for lower total cost. We explore NRE and cross-technology optimization of ASIC Clouds for four different applications: Bitcoin mining, YouTube-style video transcoding, Litecoin, and Deep Learning. We address these challenges and show large reductions in the NRE, potentially enabling ASIC Clouds to address a wider variety of datacenter workloads. Our results suggest that advanced nodes like 16nm will lead to sub-optimal TCO for many workloads, and that use of older nodes like 65nm can enable a greater diversity of ASIC Clouds.

Moonwalk

Cloud services are becoming increasingly globalized and data-center workloads are expanding exponentially. GPU and FPGA-based clouds have illustrated improvements in power and performance by accelerating compute-intensive workloads. ASIC-based clouds are a promising way to optimize the Total Cost of Ownership (TCO) of a given datacenter computation (e.g. YouTube transcoding) by reducing both energy consumption and marginal computation cost. The feasibility of an ASIC Cloud for a particular application is directly gated by the ability to manage the Non-Recurring Engineering (NRE) costs of designing and fabricating the ASIC, so that it is significantly lower (e.g. 2X) than the TCO of the best available alternative. In this paper, we show that technology node selection is a major tool for managing ASIC Cloud NRE, and allows the designer to trade off an accelerator's excess energy efficiency and cost performance for lower total cost. We explore NRE and cross-technology optimization of ASIC Clouds for four different applications: Bitcoin mining, YouTube-style video transcoding, Litecoin, and Deep Learning. We address these challenges and show large reductions in the NRE, potentially enabling ASIC Clouds to address a wider variety of datacenter workloads. Our results suggest that advanced nodes like 16nm will lead to sub-optimal TCO for many workloads, and that use of older nodes like 65nm can enable a greater diversity of ASIC Clouds.