Heterogeneous Hardware Research Articles

Parallel Pattern Compiler for Automatic Global Optimizations

High-performance computing (HPC) systems enable scientific advances through simulation and data processing. The heterogeneity in HPC hardware and software increases the application complexity and reduces its maintainability and productivity. This work proposes a prototype implementation for a parallel pattern-based source-to-source compiler to address these challenges. The prototype limits the complexity of parallelism and heterogeneous architectures to parallel patterns that are optimized towards a given target architecture. By applying high-level optimizations and a mapping between parallel patterns and execution units during compile time, portability between systems is achieved. The compiler can address architectures with shared memory, distributed memory, and accelerator offloading.The approach shows speedups for seven of the nine supported Rodinia benchmarks, reaching speedups of up to twelve times. Porting LULESH to the Parallel Pattern Language (PPL) shows a compression of code size by 65% (3.4 thousand lines of code) through a more concise expression and a higher level of abstraction. The tool’s limitations include dynamic algorithms that are challenging to analyze statically and overheads during the compile time optimization. This paper is an extended version of a previous PMAM publication (Schmitz et al., 2024).

Ginkgo - A math library designed to accelerate Exascale Computing Project science applications

Large-scale simulations require efficient computation across the entire computing hierarchy. A challenge of the Exascale Computing Project (ECP) was to reconcile highly heterogeneous hardware with the myriad of applications that were required to run on these supercomputers. Mathematical software forms the backbone of almost all scientific applications, providing efficient abstractions and operations that are crucial to harness the performance of computing systems. Ginkgo is one such mathematical software library, nurtured by ECP, providing high-performance, user-friendly, and performance portable interfaces for applications in ECP and beyond. In this paper, we elaborate on Ginkgo’s philosophy of high-performance software that is sustainable, reproducible, and easy to use. We showcase the wide feature set of solvers and preconditioners available in Ginkgo and the central concepts involved in their design. We elaborate on four different ECP software integrations: MFEM, PeleLM + SUNDIALS, XGC, and ExaSGD that use Ginkgo to accelerate their science runs. Performance studies of different problems from these applications highlight the effectiveness of Ginkgo and the benefits incurred by these ECP applications.

Elevating Security in Migration: An Enhanced Trusted Execution Environment-Based Generic Virtual Remote Attestation Scheme

Cloud computing, as the most widely applied and prominent domain of distributed systems, has brought numerous advantages to users, including high resource sharing efficiency, strong availability, and excellent scalability. However, the complexity of cloud computing environments also introduces various risks and challenges. In the current landscape with numerous cloud service providers and diverse hardware configurations in cloud environments, addressing challenges such as establishing trust chains, achieving general-purpose virtual remote attestation, and ensuring secure virtual machine migration becomes a crucial issue that traditional remote attestation architectures cannot adequately handle. Confronted with these issues in a heterogeneous multi-cloud environment, we present a targeted solution—a secure migration-enabled generic virtual remote attestation architecture based on improved TEE. We introduce a hardware trusted module to establish and bind with a Virtual Root of Trust (VRoT), addressing the challenge of trust chain establishment. Simultaneously, our architecture utilizes the VRoT within TEE to realize a general-purpose virtual remote attestation solution across heterogeneous hardware configurations. Furthermore, we design a controller deployed in the trusted domain to verify migration conditions, facilitate key exchange, and manage the migration process, ensuring the security and integrity of virtual machine migration. Lastly, we conduct rigorous experiments to measure the overhead and performance of our proposed remote attestation scheme and virtual machine secure migration process. The results unequivocally demonstrate that our architecture provides better generality and migration security with only marginal overhead compared to other traditional remote attestation solutions.

An Image-Retrieval Method Based on Cross-Hardware Platform Features

Artificial intelligence (AI) models have already achieved great success in fields such as computer vision and natural language processing. However, deploying AI models based on heterogeneous hardware is difficult to ensure accuracy consistency, especially for precision sensitive feature-based image retrieval. In this article, we realize an image-retrieval method based on cross-hardware platform features, aiming to prove that the features of heterogeneous hardware platforms can be mixed, in which the Huawei Atlas 300V and NVIDIA TeslaT4 are used for experiments. First, we compared the decoding differences of heterogeneous hardware, and used CPU software decoding to help hardware decoding improve the decoding success rate. Then, we compared the difference between the Atlas 300V and TeslaT4 chip architectures and tested the differences between the two platform features by calculating feature similarity. In addition, the scaling mode in the pre-processing process was also compared to further analyze the factors affecting feature consistency. Next, the consistency of capture and correlation based on video structure were verified. Finally, the experimental results reveal that the feature results from the TeslaT4 and Atlas 300V can be mixed for image retrieval based on cross-hardware platform features. Consequently, cross-platform image retrieval with low error is realized. Specifically, compared with the Atlas 300V hard and CPU soft decoding, the TeslaT4 hard decoded more than 99% of the image with a decoding pixel maximum difference of +1/−1. From the average of feature similarity, the feature similarity between the Atlas 300V and TeslaT4 exceeds 99%. The difference between the TeslaT4 and Atlas 300V in recall and mAP in feature retrieval is less than 0.1%.

Inference latency prediction for CNNs on heterogeneous mobile devices and ML frameworks

Due to the proliferation of inference tasks on mobile devices, state-of-the-art neural architectures are typically designed using Neural Architecture Search (NAS) to achieve good tradeoffs between machine learning accuracy and inference latency. While measuring inference latency of a huge set of candidate architectures during NAS is not feasible, latency prediction for mobile devices is challenging, because of hardware heterogeneity, optimizations applied by machine learning frameworks, and diversity of neural architectures. Motivated by these challenges, we first quantitatively assess the characteristics of neural architectures (specifically, convolutional neural networks for image classification), ML frameworks, and mobile devices that have significant effects on inference latency. Based on this assessment, we propose an operation-wise framework which addresses these challenges by developing operation-wise latency predictors and achieves high accuracy in end-to-end latency predictions, as shown by our comprehensive evaluations on multiple mobile devices using multicore CPUs and GPUs. To illustrate that our approach does not require expensive data collection, we also show that accurate predictions can be achieved on real-world neural architectures using only small amounts of profiling data.

FedGA: A greedy approach to enhance federated learning with Non-IID data

In the context of federated learning, smart terminal devices inherently exhibit various factors such as personalized user behaviors, regional differences, and heterogeneous hardware configurations due to their unique data capturing capabilities. Consequently, they inevitably present non-independent and identically distributed (Non-IID) data during the training process, making it challenging for traditional federated learning methods to achieve desired model performance and convergence effects when dealing with such complex data distributions. To address this challenge, researchers have explored data augmentation techniques, adaptive optimization algorithms, and improved model aggregation rules. However, these methods often fail to deliver satisfactory performance when confronted with Non-IID problems. In this paper, we propose a federated learning framework based on a greedy algorithm (FedGA). Unlike traditional approaches that rely on global parameter averaging aggregation, FedGA progressively searches for partially optimal models and aggregates them to obtain the global model. We first gather client data information and finely classify all clients, employing two strategies to optimize client data distribution: (1) for clients with imbalanced quantities, we utilize data sampling methods to balance client data; (2) for clients with imbalanced class distributions, we introduce a complementary federated meta-learning method, called FedComMeta, to improve the data distribution of client groups. Experimental results demonstrate that in scenarios involving Non-IID data, FedGA achieves significant improvements in model performance compared to existing methods such as FedAvg, FedProx, and Astraea, validating the effectiveness and superiority of our approach in handling Non-IID data distributions.

An Agile Pathway Towards Carbon-Aware Clouds

Climate change is a pressing threat to planetary well-being that can be addressed only by rapid near-term actions across all sectors. Yet, the cloud computing sector, with its increasingly large carbon footprint, has initiated only modest efforts to reduce emissions to date; its main approach today relies on cloud providers sourcing renewable energy from a limited global pool of options. We investigate how to accelerate cloud computing's efforts. Our approach tackles carbon reduction from a software standpoint by gradually integrating carbon awareness into the cloud abstraction. Specifically, we identify key bottlenecks to software-driven cloud carbon reduction, including (1) the lack of visibility and disaggregated control between cloud providers and users over infrastructure and applications, (2) the immense overhead presently incurred by application developers to implement carbon-aware application optimizations, and (3) the increasing complexity of carbon-aware resource management due to renewable energy variability and growing hardware heterogeneity. To overcome these barriers, we propose an agile approach that federates the responsibility and tools to achieve carbon awareness across different cloud stakeholders. As a key first step, we advocate leveraging the role of application operators in managing large-scale cloud deployments and integrating carbon efficiency metrics into their cloud usage workflow. We discuss various techniques to help operators reduce carbon emissions, such as carbon budgets, service-level visibility into emissions, and configurable-yet-centralized resource management optimizations.

Partitioned scheduling with safety-performance trade-offs in stochastic conditional DAG models

This paper is motivated by robotic systems that solve difficult real-world problems such as search and rescue (SAR) or precision agriculture 11This work was supported by NSF under Grant CNS-1932074.. These applications require robots to operate in complex, uncertain environments while maintaining safe interactions with human teammates within a specified level of performance. In this paper, we study the scheduling of real-time applications on heterogeneous hardware platforms inspired by such contexts. To capture the stochasticity due to unpredictable environments, we propose the stochastic heterogeneous parallel conditional DAG (SHPC-DAG) model, which extends the most recent HPC-DAG model in two regards. First, it uses conditional DAG nodes to model the execution of computational pipelines based on context, while the stochasticity of DAG edges captures the uncertain nature of a system’s environment or the reliability of its hardware. Second, considering the pessimism of deterministic worst-case execution time (WCET), it uses probability distributions to model the execution times of subtasks (DAG nodes). We propose a new partitioning algorithm Least Latency Partitioned (LLP), which considers precedence constraints among nodes during the allocation process. Coupled with a scheduling algorithm that accounts for varying subtask criticality and constraints, the end-to-end latencies of safety-critical paths/nodes are then minimized. We use tasksets inspired by real robotics to demonstrate that our framework allows for efficient scheduling in complex computational pipelines, with more flexible representation of timing constraints, and ultimately, safety-performance tradeoffs.

CXL and the Return of Scale-Up Database Engines

The trend toward specialized processing devices such as TPUs, DPUs, GPUs, and FPGAs has exposed the weaknesses of PCIe in interconnecting these devices and their hosts. Several attempts have been proposed to improve, augment, or downright replace PCIe, and more recently, these efforts have converged into a standard called Compute Express Link (CXL). CXL is already on version 2.0 in terms of commercial availability, but its potential to radically change the conventional server architecture has only just started to surface. For example, CXL can increase the bandwidth and quantity of memory available to any single machine beyond what that machine can originally provide, most importantly, in a manner that is fully transparent to software applications. We argue, however, that CXL can have a broader impact beyond memory expansion and deeply affect the architecture of data-intensive systems. In a nutshell, while the cloud favored scale-out approaches that grew in capacity by adding full servers to a rack, CXL brings back scale-up architectures that can grow by fine-tuning individual resources, all while transforming the rack into a large shared-memory machine. In this paper, we describe why such architectural transformations are now possible, how they benefit emerging heterogeneous hardware platforms for data-intensive systems, and the associated research challenges.

TinyML4D: Scaling Embedded Machine Learning Education in the Developing World

Embedded machine learning (ML) on low-power devices, also known as "TinyML," enables intelligent applications on accessible hardware and fosters collaboration across disciplines to solve real-world problems. Its interdisciplinary and practical nature makes embedded ML education appealing, but barriers remain that limit its accessibility, especially in developing countries. Challenges include limited open-source software, courseware, models, and datasets that can be used with globally accessible heterogeneous hardware. Our vision is that with concerted effort and partnerships between industry and academia, we can overcome such challenges and enable embedded ML education to empower developers and researchers worldwide to build locally relevant AI solutions on low-cost hardware, increasing diversity and sustainability in the field. Towards this aim, we document efforts made by the TinyML4D community to scale embedded ML education globally through open-source curricula and introductory workshops co-created by international educators. We conclude with calls to action to further develop modular and inclusive resources and transform embedded ML into a truly global gateway to embedded AI skills development.

Systematic review of memory assessment in virtual reality: evaluating convergent and divergent validity with traditional neuropsychological measures.

The evaluation of memory is a crucial aspect of both cognitive research and clinical applications, as it offers valuable insights into an individual's cognitive wellbeing and performance. Conventional neuropsychological assessments represent the established method for assessing different aspects of memory. Recent technological advancements, specifically in the field of virtual reality (VR), have introduced novel methods for evaluating memory. This systematic review aims to examine the current state of memory assessment using VR technologies, assessing the degree of convergence and divergence between VR-based memory assessments and conventional neuropsychological tests. A systematic review of the literature was conducted searching PubMed, PsycINFO, Web of Science databases, leading to the incorporation of 24 studies. Studies were grouped according to the examined memory domain (episodic, prospective, spatial domain). Convergence and divergence validity were examined for each, and information on software and hardware features was collected. This review demonstrates a notable alignment between VR-based memory assessments and conventional neuropsychological tests. Moreover, VR tasks have shown to exhibit associations with executive functions and overall cognitive performance. The inclusion of various ecological contexts, such as residential environments, commercial establishments, and simulated scenarios, serves to augment the ecological validity of memory evaluations conducted in VR. The findings indicate that VR assessments demonstrate a functional perspective by effectively capturing the dynamic relationship between memory, executive functions, and overall cognitive performance. Nevertheless, it is imperative to acknowledge and tackle certain constraints that may hinder the widespread adoption and utilization of VR tasks. These limitations encompass factors such as restricted accessibility to VR tasks and the presence of heterogeneity in VR hardware and software. The dynamic and ever-changing nature of VR technology presents a range of potential avenues for future investigation and utilization in the domain of memory evaluation.

Automated deep learning segmentation of high-resolution 7 Tesla postmortem MRI for quantitative analysis of structure-pathology correlations in neurodegenerative diseases.

Postmortem MRI allows brain anatomy to be examined at high resolution and to link pathology measures with morphometric measurements. However, automated segmentation methods for brain mapping in postmortem MRI are not well developed, primarily due to limited availability of labeled datasets, and heterogeneity in scanner hardware and acquisition protocols. In this work, we present a high-resolution dataset of 135 postmortem human brain tissue specimens imaged at 0.3 mm3 isotropic using a T2w sequence on a 7T whole-body MRI scanner. We developed a deep learning pipeline to segment the cortical mantle by benchmarking the performance of nine deep neural architectures, followed by post-hoc topological correction. We evaluate the reliability of this pipeline via overlap metrics with manual segmentation in 6 specimens, and intra-class correlation between cortical thickness measures extracted from the automatic segmentation and expert-generated reference measures in 36 specimens. We also segment four subcortical structures (caudate, putamen, globus pallidus, and thalamus), white matter hyperintensities, and the normal appearing white matter, providing a limited evaluation of accuracy. We show generalizing capabilities across whole-brain hemispheres in different specimens, and also on unseen images acquired at 0.28 mm3 and 0.16 mm3 isotropic T2*w fast low angle shot (FLASH) sequence at 7T. We report associations between localized cortical thickness and volumetric measurements across key regions, and semi-quantitative neuropathological ratings in a subset of 82 individuals with Alzheimer's disease (AD) continuum diagnoses. Our code, Jupyter notebooks, and the containerized executables are publicly available at the project webpage (https://pulkit-khandelwal.github.io/exvivo-brain-upenn/).

Combining machine learning with computational fluid dynamics using OpenFOAM and SmartSim

AbstractCombining machine learning (ML) with computational fluid dynamics (CFD) opens many possibilities for improving simulations of technical and natural systems. However, CFD+ML algorithms require exchange of data, synchronization, and calculation on heterogeneous hardware, making their implementation for large-scale problems exceptionally challenging. We provide an effective and scalable solution to developing CFD+ML algorithms using open source software OpenFOAM and SmartSim. SmartSim provides an Orchestrator that significantly simplifies the programming of CFD+ML algorithms enables scalable data exchange between ML and CFD clients. We show how to leverage SmartSim to effectively couple different segments of OpenFOAM with ML, including pre/post-processing applications, function objects, and mesh motion solvers. We additionally provide an OpenFOAM sub-module with examples that can be used as starting points for real-world applications in CFD+ML.

Neko: A modern, portable, and scalable framework for high-fidelity computational fluid dynamics

Computational fluid dynamics (CFD), in particular applied to turbulent flows, is a research area with great engineering and fundamental physical interest. However, already at moderately high Reynolds numbers the computational cost becomes prohibitive as the range of active spatial and temporal scales is quickly widening. Specifically scale-resolving simulations, including large-eddy simulation (LES) and direct numerical simulations (DNS), thus need to rely on modern efficient numerical methods and corresponding software implementations. Recent trends and advancements, including more diverse and heterogeneous hardware in High-Performance Computing (HPC), are challenging software developers in their pursuit for good performance and numerical stability. The well-known maxim “software outlives hardware” may no longer necessarily hold true, and developers are today forced to re-factor their codebases to leverage these powerful new systems. In this paper, we present Neko, a new portable framework for high-order spectral element discretization, targeting turbulent flows in moderately complex geometries. Neko is fully available as open software. Unlike prior works, Neko adopts a modern object-oriented approach in Fortran 2008, allowing multi-tier abstractions of the solver stack and facilitating hardware backends ranging from general-purpose processors (CPUs) down to exotic vector processors and FPGAs. We show that Neko’s performance and accuracy are comparable to NekRS, and thus on-par with Nek5000’s successor on modern CPU machines. Furthermore, we develop a performance model, which we use to discuss challenges and opportunities for high-order solvers on emerging hardware.

Zero-sided RDMA: Network-driven Data Shuffling for Disaggregated Heterogeneous Cloud DBMSs

In this paper, we present a novel communication scheme called zero-sided RDMA, enabling data exchange as a native network service using a programmable switch. In contrast to one- or two-sided RDMA, in zero-sided RDMA, neither the sender nor the receiver is actively involved in data exchange. Zero-sided RDMA thus enables efficient RDMA-based data shuffling between heterogeneous hardware devices in a disaggregated setup without the need to implement a complete RDMA stack on each heterogeneous device or the need for a CPU that is co-located with the accelerator to coordinate the data transfer. As such, we think that zero-sided RDMA is a major building block to make efficient use of heterogeneous accelerators in future cloud DBMSs. In our evaluation, we show that zero-sided RDMA can outperform existing one-sided RDMA-based schemes for accelerator-to-accelerator communication and thus speed up typical distributed database operations such as joins.

Towards a Docker-based architecture for open multi-agent systems

<p>In open multi-agent systems (OMAS), heterogeneous agents in different environments or models can migrate from one system to another, taking their attributes and knowledge and increasing developing complexity compared to conventional multi-agent systems (MAS). Furthermore, the complexity of opening may be due to the uncertainties and dynamic behavior that the change of agents entails, needing to formulate techniques to analyze this complexity and understand the system’s global behavior. We used Docker to approach these problems and make the architecture flexible to handle distinct types of programming languages and frameworks of agents. This paper presents a Docker-based architecture to aid OMAS development, acting on agent migration between different models running in heterogeneous hardware and software scenarios. We present a simulation scenario with NetLogo’s Open Sugarscape 2 Constant Growback and JaCaMo’s Gold Miners to verify the proposal’s feasibility.</p>

BrickOS: specialized kernels for heterogeneous hardware resources

BrickOS: specialized kernels for heterogeneous hardware resources

RC-LAHR: Road-Side-Unit-Assisted Cloud-Based Location-Aware Hybrid Routing for Software-Defined Vehicular Ad Hoc Networks.

The reliability of the communication link is quite common and challenging to handle as the topology changes frequently in vehicular ad hoc networks (VANETs). Another problem with VANETs is that the vehicles are from different manufacturers. Hence, the heterogeneity of hardware is obvious. These heterogeneity and reliability problems affect the message dissemination in VANETs. This paper aims to address these challenges by proposing a robust routing protocol capable of ensuring reliable, scalable, and heterogeneity-tolerant message dissemination in VANETs. We first introduced a hybrid hierarchical architecture based on software-defined networking (SDN) principles for VANETs, leveraging SDN's inherent scalability and adaptability to heterogeneity. Further, a road-side unit (RSU)-assisted cloud-based location-aware hybrid routing for software-defined VANETs (SD-VANETs) that we call RC-LAHR was proposed. RC-LAHR was rigorously tested and analyzed for its performance in terms of packet delivery ratio (PDR) and end-to-end delay (EED), along with a comprehensive assessment of network traffic and load impacts on cloud infrastructure and RSUs. The routing protocol is compared with state-of-the-art protocols, Greedy Perimeter Stateless Routing (GPSR) and Opportunistic and Position-Based Routing (OPBR). The proposed routing protocol performs well as compared to GPSR and OPBR. The result shows that the EED is reduced to 20% and the PDR is increased to 30%. The network reliability is also increased up to 5% as compared to the OPBR and GPSR.

A Path to Industry 5.0 Digital Twins for Human–Robot Collaboration by Bridging NEP+ and ROS

The integration of heterogeneous hardware and software components to construct human-centered systems for Industry 5.0, particularly human digital twins, presents considerable complexity. Our research addresses this challenge by pioneering a novel approach that harmonizes the techno-centered focus of the Robot Operating System (ROS) with the cross-platform advantages inherent in NEP+ (a human-centered development framework intended to assist users and developers with diverse backgrounds and resources in constructing interactive human–machine systems). We introduce the nep2ros ROS package, aiming to bridge these frameworks and foster a more interconnected and adaptable approach. This initiative can be used to facilitate diverse development scenarios beyond conventional robotics, underpinning a transformative shift in Industry 5.0 applications. Our assessment of NEP+ capabilities includes an evaluation of communication performance utilizing serialization formats like JavaScript Object Notation (JSON) and MessagePack. Additionally, we present a comparative analysis between the nep2ros package and existing solutions, illustrating its efficacy in linking the simulation environment (Unity) and ROS. Moreover, our research demonstrates NEP+’s applicability through an immersive human-in-the-loop collaborative assembly. These findings offer promising prospects for innovative integration possibilities across a broad spectrum of applications, transcending specific platforms or disciplines.

A Hybrid Fault Diagnosis Method for Autonomous Driving Sensing Systems Based on Information Complexity

In the context of autonomous driving, sensing systems play a crucial role, and their accuracy and reliability can significantly impact the overall safety of autonomous vehicles. Despite this, fault diagnosis for sensing systems has not received widespread attention, and existing research has limitations. This paper focuses on the unique characteristics of autonomous driving sensing systems and proposes a fault diagnosis method that combines hardware redundancy and analytical redundancy. Firstly, to ensure the authenticity of the study, we define 12 common real-world faults and inject them into the nuScenes dataset, creating an extended dataset. Then, employing heterogeneous hardware redundancy, we fuse MMW radar, LiDAR, and camera data, projecting them into pixel space. We utilize the “ground truth” obtained from the MMW radar to detect faults on the LiDAR and camera data. Finally, we use multidimensional temporal entropy to assess the information complexity fluctuations of LiDAR and the camera during faults. Simultaneously, we construct a CNN-based time-series data multi-classification model to identify fault types. Through experiments, our proposed method achieves 95.33% accuracy in detecting faults and 82.89% accuracy in fault diagnosis on real vehicles. The average response times for fault detection and diagnosis are 0.87 s and 1.36 s, respectively. The results demonstrate that the proposed method can effectively detect and diagnose faults in sensing systems and respond rapidly, providing enhanced reliability for autonomous driving systems.