Virtual Shared Memory Research Articles

Distributed Shared Memory : A Page-Based Approach

In 1986, Kai Li's PhD thesis, "Shared Virtual Memory on Loosely Coupled Microprocessors," introduced the concept of Distributed Shared Memory (DSM) systems, which has since facilitated further research in this field. One approach to implementing DSM in distributed systems is the page-based method, which utilizes virtual memory techniques to map pages of a process' address space onto the physical memory of multiple nodes. The system automatically moves pages between nodes as needed, ensuring coherence and consistency for shared page updates. Page-based DSM offers several advantages, including improved performance due to reduced communication overhead and better data locality, simplified programming models that resemble centralized systems, and improved scalability by allowing additional nodes to join the system and expand memory capacity. However, its suitability for certain distributed systems depends on their specific requirements and characteristics. Page-based DSM has been applied in various distributed systems, including high- performance computing, cloud computing, and distributed databases. Nonetheless, remote memory access overheads, cache coherence issues, and the necessity of a well-connected network remain potential limitations of this approach. Key words: Distributed memory, Shared memory, Paging, Virtual memory, memory coherence, memory consistency

Softmicrogrid: a Software Assisted Microgrid for Optimal Prosumer Satisfaction

These days electrical grid makes use of information and communication technology. Information technology is primarily used for providing intelligence in the electrical grid. Solution presented in this article is a step further in this direction. All the prosumers have photovoltaic cells and energy storage batteries. The cyber-physical energy infrastructure solves many contentious issues of traditional electrical grid. This research article addresses the issues related to energy sharing among prosumers of a microgrid. The issues are namely, (i) intermittent nature of renewable energy resources, (ii) energy loss due to physical central storage, and (iii) charging/discharging scheme of the central storage in the presence of real-time pricing. The concept of central physical energy storage system is inefficient. Therefore, it is necessary to have an efficient energy sharing mechanism to reduce energy penetration to the grid and reduce energy loss. This paper proposes a cyber-physical energy sharing structure which introduces the idea of virtual shared distributed energy storage analogous to the virtual shared memory in distributed computing systems. This concept, named as softmicrogrid, considerably reduces the subsequent charging/discharging hurdles and energy loss of microgrid. This concept is supplemented with an algorithm for charging the virtual shared distributed energy storage.

Co-Scheduling on Fused CPU-GPU Architectures With Shared Last Level Caches

Fused CPU-GPU architectures integrate a CPU and general-purpose GPU on a single die. Recent fused architectures even share the last level cache (LLC) between CPU and GPU. This enables hardware-supported byte-level coherency. Thus, CPU and GPU can execute computational kernels collaboratively, but novel methods to co-schedule work are required. This paper contributes three dynamic co-scheduling methods. Two of our methods implement workers that autonomously acquire work from a common set of independent work items (similar to bag-of-tasks scheduling). The third method, host-side profiling , uses a fraction of the total work of a kernel to determine a ratio of how to distribute work to CPU and GPU based on profiling. The resulting ratio is used for the following executions of the same kernel. Our methods are realized using OpenCL 2.0, which introduces fine-grained shared virtual memory (SVM) to allocate coherent memory between CPU and GPU. We port the Rodinia Benchmark Suite, a standard suite for heterogeneous computing, to fine-grained SVM and fused CPU-GPU architectures ( Rodinia-SVM ). We evaluate the overhead of fine-grained SVM and analyze the suitability of OpenCL 2.0’s new features for co-scheduling. Our host-side profiling method performs competitively to the optimal choice of executing kernels either on CPU or GPU (hypothetical xor-Oracle). On average, it achieves 97% of xor-Oracle’s performance and a $1.43\times$ speedup over using the GPU alone (standard in Rodinia). We show, however, that in most cases it is not beneficial to split the work of a kernel between CPU and GPU compared to exclusively running it on the most suitable single compute device. For a fixed amount of work per device, cache-related stalls can increase by up to $1.75\times$ when both devices are used in parallel instead of exclusively while cache misses remain the same. Thus, not the cost of cache conflicts, but inefficient cache coherence is a major performance bottleneck for current fused CPU-GPU Intel architectures with shared LLC.

Formal Specification of Memory Coherence Protocol

Memory coherence is the most fundamental re-quirement in a shared virtual memory system where there are concurrent as well as loosely coupled processes. These processes can demand a page for reading or writing. The memory is called coherent if the last update in a page remains constant for each process until the owner of that page does not change it. The ownership is transferred to a process interested to update that page. In [Kai LI, and Paul Hudak. Memory Coherence in Shared Virtual Memory Systems, 1986. Proc. of Fifth Annual ACM Symposium on Principles of Distributed Computing.], algorithms ensuring memory coherence are given. We formally specify these protocols and report the improvements through formal analysis. The protocols are specified in UPPAAL, i.e., a tool for modeling, validation and verification of real-time systems.

Soft Memory Box: A Virtual Shared Memory Framework for Fast Deep Neural Network Training in Distributed High Performance Computing

Deep learning is one of the major promising machine learning methodologies. Deep learning is widely used in various application domains, e.g., image recognition, voice recognition, and natural language processing. In order to improve learning accuracy, deep neural networks have evolved by: 1) increasing the number of layers and 2) increasing the number of parameters in massive models. This implies that distributed deep learning platforms need to evolve to: 1) deal with huge/complex deep neural networks and 2) process with high-performance computing resources for massive training data. This paper proposes a new virtual shared memory framework, called Soft Memory Box (SMB), which enables sharing the memory of remote node among distributed processes in the nodes so as to improve communication performance via parameter sharing. According to data-intensive performance evaluation results, the communication time of deep learning using the proposed SMB is 2.1 times faster than that using the massage passing interface (MPI). In addition, the communication time of the SMB-based asynchronous parameter update becomes 2–7 times faster than that using the MPI depending on deep learning models and the number of deep learning workers.

Enabling PoCL-based runtime frameworks on the HSA for OpenCL 2.0 support

Abstract The heterogeneous system architecture (HSA), announced by the HSA Foundation, is an approach to integrate central processing unit (CPU) and graphics processing unit (GPU) architectures. The open computing language (OpenCL) is a programming framework that can help utilize heterogeneous architectures. The well-known OpenCL framework, currently in version 1.2, provides programming models for heterogeneous computing. The proposed specifications of OpenCL 2.0 can help utilize HSA features, such as shared virtual memory (SVM). In previous work, we helped enable Portable Computing Language (PoCL)-based OpenCL 1.2 runtime frameworks on the HSA. In this paper, we further extend the PoCL-based runtime on the HSA to support OpenCL 2.0 features. In addition, this is the first work, to our best knowledge, to support PoCL-based OpenCL 2.0 features on HSA. Compared with the widely used OpenCL 1.2, OpenCL 2.0 will support SVM, nested parallelism, pipes, and atomic operations. It can further support parallel design patterns such as tree searches, pointer-based programming and nested parallelism models. Note that PoCL is a widely used open source implementation of OpenCL. Our design flow can help academics to enable OpenCL 2.0 flow on the HSA and benefit further from advanced academic research. The experimental results indicate that our framework provides adequate features to support advanced research.

Efficient Virtual Memory Sharing via On-Accelerator Page Table Walking in Heterogeneous Embedded SoCs

Shared virtual memory is key in heterogeneous systems on chip (SoCs) that combine a general-purpose host processor with a many-core accelerator, both for programmability and performance. In contrast to the full-blown, hardware-only solutions predominant in modern high-end systems, lightweight hardware-software co-designs are better suited in the context of more power- and area-constrained embedded systems and provide additional benefits in terms of flexibility and predictability. As a downside, the latter solutions require the host to handle in software synchronization in case of page misses as well as miss handling. This may incur considerable run-time overheads. In this work, we present a novel hardware-software virtual memory management approach for many-core accelerators in heterogeneous embedded SoCs. It exploits an accelerator-side helper thread concept that enables the accelerator to manage its virtual memory hardware autonomously while operating cache-coherently on the page tables of the user-space processes of the host. This greatly reduces overhead with respect to host-side solutions while retaining flexibility. We have validated the design with a set of parameterizable benchmarks and real-world applications covering various application domains. For purely memory-bound kernels, the accelerator performance improves by a factor of 3.8 compared with host-based management and lies within 50% of a lower-bound ideal memory management unit.

A Comparative Study of Programming Environments Exploiting Heterogeneous Systems

This paper compares programming environments that exploit heterogeneous systems to process a large amount of data efficiently. Our motivation is to investigate the feasibility of the adaptive, transparent migration of intensive computation for a large amount of data across heterogeneous programming languages and processors for high performance and programmability. We compare a variety of programming environments composed of programming languages, such as Java and C, memory space models, such as distinct and shared memory, and parallel processors, such as general-purpose CPUs and graphics processing units (GPUs) to examine their performance-programmability tradeoffs. In addition, we introduce a software-based shared virtual memory that creates a view of the host memory inside GPU kernels to enable seamless computation offloading from the host to the device. This paper reveals a programmability-performance hierarchy in which programs increase their performance at the cost of decreasing programmability. The experimental results suggest the desirability of a well-balanced system.

Distributed Attack Graph Generation

Attack graphs show possible paths that an attacker can use to intrude into a target network and gain privileges through series of vulnerability exploits. The computation of attack graphs suffers from the state explosion problem occurring most notably when the number of vulnerabilities in the target network grows large. Parallel computation of attack graphs can be utilized to attenuate this problem. When employed in online network security evaluation, the computation of attack graphs can be triggered with the correlated intrusion alerts received from sensors scattered throughout the target network. In such cases, distributed computation of attack graphs becomes valuable. This article introduces a parallel and distributed memory-based algorithm that builds vulnerability-based attack graphs on a distributed multi-agent platform. A virtual shared memory abstraction is proposed to be used over such a platform, whose memory pages are initialized by partitioning the network reachability information. We demonstrate the feasibility of parallel distributed computation of attack graphs and show that even a small degree of parallelism can effectively speed up the generation process as the problem size grows. We also introduce a rich attack template and network model in order to form chains of vulnerability exploits in attack graphs more precisely.

A framework for general sparse matrix–matrix multiplication on GPUs and heterogeneous processors

General sparse matrix–matrix multiplication (SpGEMM) is a fundamental building block for numerous applications such as algebraic multigrid method (AMG), breadth first search and shortest path problem. Compared to other sparse BLAS routines, an efficient parallel SpGEMM implementation has to handle extra irregularity from three aspects: (1) the number of nonzero entries in the resulting sparse matrix is unknown in advance, (2) very expensive parallel insert operations at random positions in the resulting sparse matrix dominate the execution time, and (3) load balancing must account for sparse data in both input matrices.In this work we propose a framework for SpGEMM on GPUs and emerging CPU–GPU heterogeneous processors. This framework particularly focuses on the above three problems. Memory pre-allocation for the resulting matrix is organized by a hybrid method that saves a large amount of global memory space and efficiently utilizes the very limited on-chip scratchpad memory. Parallel insert operations of the nonzero entries are implemented through the GPU merge path algorithm that is experimentally found to be the fastest GPU merge approach. Load balancing builds on the number of necessary arithmetic operations on the nonzero entries and is guaranteed in all stages.Compared with the state-of-the-art CPU and GPU SpGEMM methods, our approach delivers excellent absolute performance and relative speedups on various benchmarks multiplying matrices with diverse sparsity structures. Furthermore, on heterogeneous processors, our SpGEMM approach achieves higher throughput by using re-allocatable shared virtual memory.

A parallel computational model based on mobile agents for high performance computing

Abstract The goal of this paper is to present a new massively parallel virtual machine model, designed for parallel and high performance computing on distributed systems. The proposed model allows us to build a polymorphic grid computing assigned to solve fine grained parallel problems over different well known, SIMD (Single Instruction Multiple Data), SPMD (Single Program Multiple Data), MIMD (Multiple Instruction Multiple Data) ) and MPMD (Multiple Program Multiple Data) machine structures. This model is built using dynamic distributed Virtual Processors Units (VPU). Each VPU corresponds to a mobile agent deployed in a physical processing unit. Each physical processing unit is assigned to a node of the considered distributed system. These units may be heterogeneous machines, they are characterized by their physical performance indices. The proposed model is able to include different computing unit technologies, such as supercomputers, simple desktop or smart phones etc. The VPUs are designed to communicate with each other asynchronously by exchanging, in local or remote manner, the ACL messages (Agent Communication Language) containing data, instructions or any task to be performed. In this model a special agent is designed to represent the host of the parallel virtual machine and to manage the following activities: the life cycle of the VPUs, the load balancing control and the parallel programs to run. In this model VPUs can also use a virtual shared memory represented by hierarchical mobile agents. All the properties offered to the proposed model, are easily designed thanks to the flexibility and the mobility of the multi agent systems. This leads to a strong computation model for high perform-

A new perspective for efficient virtual-cache coherence

Coherent shared virtual memory (cSVM) is highly coveted for heterogeneous architectures as it will simplify programming across different cores and manycore accelerators. In this context, virtual L1 caches can be used to great advantage, e.g., saving energy consumption by eliminating address translation for hits. Unfortunately, multicore virtual-cache coherence is complex and costly because it requires reverse translation for any coherence request directed towards a virtual L1. The reason is the ambiguity of the virtual address due to the possibility of synonyms. In this paper, we take a radically different approach than all prior work which is focused on reverse translation. We examine the problem from the perspective of the coherence protocol. We show that if a coherence protocol adheres to certain conditions, it operates effortlessly with virtual caches, without requiring reverse translations even in the presence of synonyms. We show that these conditions hold in a new class of simple and efficient request-response protocols that use both self-invalidation and self-downgrade. This results in a new solution for virtual-cache coherence, significantly less complex and more efficient than prior proposals. We study design choices for TLB placement under our proposal and compare them against those under a directory-MESI protocol. Our approach allows for choices that are particularly effective as for example combining all per-core TLBs in a single logical TLB in front of the last level cache. Significant area, energy, and performance benefits ensue as a result of simplifying the entire multicore memory organization.

Design and Implementation for the Secure ID Management System Based on Virtualization

In this paper, we study a solution to solve the authentication problem in the ID management system (IDMS). Our solution is based on the virtualization technology which allows data transfer between two servers in the absence of the network connection. In this paper, we propose a Virtual ID management system (VIDMS) that logically divides the login server and the authentication server in a physical system. The ID authentication process minimizes the security threat in the network since it can be performed using the virtual shared memory in the absence of the network connection. As compared with the existing IDMS, our solution improves the authentication performance (in terms of the speed).

ZIVM: A Zero-Copy Inter-VM Communication Mechanism for Cloud Computing

With the advent of virtualization technology and its propagation to the infrastructure of Cloud distributed systems, there is an emergent request for more effective means of communication between virtual machines (VMs) lying on distributed memory than traditional message based communication means. This paper presents a distributed virtual shared memory mechanism called ZIVM (Zero-copy Inter-VM) to ease the programming of inter-VM communications in a transparent way and at the same time to achieve comparable and bearable execution performance. ZIVM has been implemented as a virtual cluster on a hosted virtual machine using the KVM hypervisor. Experimental results have shown near native performance in terms of latency and bandwidth.

Optimizing a shared virtual memory system for a heterogeneous CPU-accelerator platform

The client computing platform is moving towards a heterogeneous architecture that combines scalar-oriented CPU cores and throughput-oriented accelerator cores. Recognizing that existing programming models for such heterogeneous platforms are still difficult for most programmers, we advocate a shared virtual memory programming model to improve programmability. In this paper, we focus on performance, and demonstrate that users need not sacrifice performance for programmability. We describe our approaches, experiences, and results in optimizing MYO on a heterogeneous platform consisting of a CPU and an Aubrey Isle accelerator. Our efforts involve the whole system software stack including the OS, runtime, and application.

Bothnia

In this paper, we introduce Bothnia, an extension to the Intel production graphics driver to support a shared virtual memory heterogeneous multithreading programming model. With Bothnia, the Intel graphics device driver can support both the traditional 3D graphics rendering software stack and a new class of heterogeneous multithreaded applications, which can use both IA (Intel Architecture) CPU cores and Intel integrated Graphics and Media Accelerator (GMA) cores in the same virtual address space. We describe the necessary architectural supports in both IA CPU and the GMA cores and present a reference Bothnia implementation. For a set of GPU accelerated media applications on a PC platform with Intel Core 2 Duo CPU and the Intel integrated GMA X3000 running under the Windows XP operating system, Bothnia achieves an average speedup of 3.6x compared to using the GPU as a device, primarily due to Bothnia's support for creation of shared virtual address space between heterogeneous threads of the same application spread on both IA CPU and GMA cores.

On a Virtual Shared Memory Cluster System with Virtual Machines

In this paper, an architecture with a load balancing model and a fault tolerant model for virtual shared memory clusters is proposed. The centralized dynamic load balancing model uses manager worker concept with a virtualized compute server. The virtual server can be expanded on the fly for load balancing by adding virtual machines temporarily. The fault tolerant model controlled by a virtual machine monitor describes the checkpointing based recovery methods. The performance evaluation results show that the proposed system achieves significant speedup in terms of execution time and checkpoint time .

Data race avoidance and replay scheme for developing and debugging parallel programs on distributed shared memory systems

Distributed shared memory (DSM) allows parallel programs to run on distributed computers by simulating a global virtual shared memory, but data racing bugs may easily occur when the threads of a multi-threaded process concurrently access the physically distributed memory. Earlier tools to help programmers locate data racing bugs in non-DSM parallel programs are not easily applied to DSM systems. This study presents the data race avoidance and replay scheme (DRARS) to assist debugging parallel programs on DSM or multi-core systems. DRARS is a novel tool which controls the consistency protocol of the target program, automatically preventing a large class of data racing bugs when the parallel program is subsequently run, obviating much of the need for manual debugging. For data racing bugs that cannot be avoided automatically, DRARS performs a deterministic replay-type function on DSM systems, faithfully reproducing the behavior of the parallel program during run time. Because one class of data racing bugs has already been eliminated, the remaining manual debugging task is greatly simplified. Unlike previous debugging methods, DRARS does not require that the parallel program be written in a specific style or programming language. Moreover, DRARS can be implemented in most consistency protocols. In this paper, DRARS is realized and verified in real experiments using the eager release consistency protocol on a DSM system with various applications.

On the Performance of the Parallel Implementation of the Shallow Water Model on Distributed Memory Architectures

This paper presents a study of the impact of memory architectures, distributed memory (DM) and virtual shared memory (VSM), in the solution of parallel numerical algorithms on a multi-processor cluster. A parallel implementation of the shallow water equations to model a Tsunami is chosen as the case study. Data is partitioned into sub-domains, namely a four by three grid scheme and an eight by six grid scheme which are used for the parallel implementation of this model. There are four versions of the parallel algorithm for each grid scheme: distributed memory without threads, distributed memory with threads, virtual shared memory without threads, and virtual shared memory with threads. These four parallel versions have been implemented on a high performance cluster, connected to the "Nordugrid". Experiments are realized using the Message Passing Interface (MPI) library, the C/Linda, and the Linux pthreads. Subject to the availability of memory, the virtual shared memory version without threads performs best, and as the task is scaled up, the threaded version becomes efficient in both DM and VSM implementations. Index Terms: MPI, Linda, multi processors, Shallow water equations, tsunami model.DOI: 10.4038/icter.v2i1.1399The International Journal on Advances in ICT for Emerging Regions 2009 02 (01): 3-10

Guest Editor Introduction: Special Issue on High Performance Computing for High Productivity Environments

Rapid prototyping and application development environments are often the tool of choice for many developers and users. These environments and languages include MATLAB®, perl, python, R and Ruby. They are popular, in large part, because theymake computing accessible to “accidental programmers” (scientists, statisticians, economists and others not specifically trained as programmers) and because they deliver on their promise of rapid prototyping to more seasoned programmers. Users of these systems often face a dilemma as their problems grow: livewithin the performance and resource constraints of these systems, or leave these comfortable environments behind and take up tools that are traditionally associated with high(er) performance computing (C/C++/FORTRAN, MPI, OpenMP). This special issue presents six projects aimed at resolving this dilemma in favor of remaining in the environment of choice, but augmenting it so that larger problems can be handled with relative ease (although, perhaps, not with absolutely optimal performance). The projects presented cover a good sampling of popular environments: MATLAB, python, R and Ruby, with a touch of Perl on the side. The enhancement strategies span a range of approaches: easy to use interfaces for accessing high performance libraries that were written the “old fashioned” way, parallel versions of apply/map functions, data parallel frameworks, cleanly wrapped RPC systems, and virtual shared memory. Some approaches are specific to one environment; others are applicable to many or most. Taken together, they are indicative of the interest in and the importance of creative responses to the challenge of making HPC accessible to a broad range of users. Sharma and Martin discuss parallel constructs for MATLAB, one of the most popular commercial systems in this area. They address two related challenges: (i) develop parallel functionality that fits well with the MATLAB computational model;