Software Distributed Shared Memory Research Articles

High order accurate simulation of compressible flows on GPU clusters over Software Distributed Shared Memory

The advent of multicore processors during the past decade and especially the recent introduction of many-core Graphics Processing Units (GPUs) open new horizons to large-scale, high-resolution simulations for a broad range of scientific fields. Residing at the forefront of advancements in multiprocessor technology, GPUs are often chosen as co-processors when intensive parts of applications need to be computed. Among the various domains, the scientific area of Computational Fluid Dynamics (CFD) is a potential candidate that could significantly benefit from the utilization of many-core GPUs. In order to investigate this possibility, we herein evaluate the performance of a high order accurate method for the simulation of compressible flows.Targeting computer systems with multiple GPUs, the current implementation and the respective performance evaluation are taking place on a GPU cluster. With respect to using these GPUs, this paper offers an alternative to the mainstream approach of message passing by considering shared memory abstraction. In the implementations presented in this paper, the updates on shared data are not explicitly coded by the programmer across the simulation phases, but are propagated through Software Distributed Shared Memory (SDSM). This way, we intend to preserve a unified memory view that extends the memory hierarchy from the node level to the cluster level. Such an extension could significantly facilitate the porting of multithreaded codes at GPU clusters. Our results indicate that the presented approach is competitive with the message passing paradigm and they lay grounds for further research on the use of shared memory abstraction for future GPU clusters.

Consistency Model and Synchronization Primitives in SDSMS

Abstract —consistency model like Strict Consistency, Sequential Consistency, Processor Consistency, Weak Consistency etc. Then the techniques for implementing distributed shared memory Systems and Synchronization Primitives in Software Distributed Shared Memory Systems are discussed. The analysis involves the performance measurement of the protocol concerned that is Multiple Writer Protocol. Each protocol has pros and cons. So, the problems that are associated with each protocol is discussed and other related things are explored. Keywords — Distributed System, Single owner protocol, Multiple owner protocolI. I NTRODUCTION distributed system is an application that executes a collection of protocols to coordinate the actions of multiple processes on a network, such that all components cooperate together to perform a single or small set of related tasks. II. M EMORY C ONSISTENCY M ODEL A memory consistency model is a contract between programmers and shared memory which specifies how the memory operations of a program will be executed. Computer scientists have proposed different memory models to enhance distributed shared memory systems. In this section, we will give an introduction to those memory models and some terminology used in this paper.

Overcoming performance bottlenecks in using OpenMP on SMP clusters

This paper presents a new parallel programming environment called ParADE to enable easy, portable, and high-performance computing for SMP clusters. Different from the prior studies, ParADE separates the programming model from the execution model: it enables shared-address-space programming while it realizes hybrid execution of message-passing and shared-address-space. To overcome the poor performance of conventional OpenMP on SDSM (Software Distributed Shared Memory), ParADE implements an intelligent OpenMP translator supporting efficient mutual exclusion and efficient page transmission. The experimental results on a Linux cluster demonstrate that ParADE reduces mutual exclusion overhead and overall execution time.

Integrating coordinated checkpointing and recovery mechanisms into DSM synchronization barriers

Distributed shared memory (DSM) creates an abstraction of a physical shared memory that parallel programmers can access. Most recent software DSM systems provide relaxed-memory models that guarantee consistency only at synchronization operations, such as locks and barriers. As the main goal of DSM systems is to provide support for long-term computation-intensive applications, checkpointing and recovery mechanisms are highly desirable. This article presents and evaluates the integration of a coordinated checkpointing mechanism to the barrier primitive that is usually provided with many DSM systems. Our results on some popular benchmarks and a real parallel application show that the overhead introduced during the failure-free execution is often small.

A grid-enabled software distributed shared memory system on a wide area network

This study implements a grid-enabled software distributed shared memory (SDSM) system called Teamster-G on a wide area network (WAN). With the support of Teamster-G, users can develop applications on computational grids by means of shared variables. When they wish to execute their applications, Teamster-G provides a transparent resource allocation service for the execution of the programs. To minimize the turnaround time, Teamster-G employs a session-oriented protocol to reduce the cost of resource allocation, and a two-level consistency protocol to minimize the cost of maintaining data consistency over the WAN. This paper presents the framework of Teamster-G and discusses its performance.

Adapting workload distribution on software DSM clusters

AbstractAchieving an appropriate workload distribution is essential if user applications are to achieve a satisfactory performance on software distributed shared memory (SDSM) clusters. To address this problem, the present study develops a novel method for distributing program threads onto the individual computers of SDSM clusters. In contrast to alternative methods, the proposed approach takes account not only of processor speed and data‐sharing aspects, but also memory availability, such that application performance can be enhanced through the implementation of appropriate workload distributions. In addition, when distributing the program threads, the proposed method specifically chooses only those computers that can enhance the performance of the application, rather than simply distributing the threads to all the available nodes in the cluster. This location policy makes it possible to specify the appropriate node combinations that optimize program performance while simultaneously maximizing resource utilization. The proposed method is implemented on a testbed referred to as Teamster. The experimental results demonstrate that, compared to alternative methods, the proposed approach delivers a 20–30% improvement in the performance of the chosen test applications. Importantly, it is shown that the proposed method can efficiently specify proper node combinations for the applications. Copyright © 2006 John Wiley & Sons, Ltd.

OpenMP extension to SMP clusters

This article discusses the approaches to apply the OpenMP programming model to symmetric multiprocessor (SMP) clusters using software distributed shared memory (SDSM). The major focus of this article is on the challenges that the prior studies faced and on their solution techniques. Exploiting message-passing primitives explicitly for the openMP synchronization and work-sharing directives enables light interprocess synchronizations. The studies on loop scheduling for SMP clusters will promise significant improvement in system performance. Finally, OpenMP is considered as promising programming model

Towards a more efficient implementation of OpenMP for clusters via translation to global arrays

This paper discusses a novel approach to implementing OpenMP on clusters. Traditional approaches to do so rely on Software Distributed Shared Memory systems to handle shared data. We discuss these and then introduce an alternative approach that translates OpenMP to Global Arrays (GA), explaining the basic strategy. GA requires a data distribution. We do not expect the user to supply this; rather, we show how we perform data distribution and work distribution according to the user-supplied OpenMP static loop schedules. An inspector–executor strategy is employed for irregular applications in order to gather information on accesses to potentially non-local data, group non-local data transfers and overlap communications with local computations. Furthermore, a new directive INVARIANT is proposed to provide information about the dynamic scope of data access patterns. This directive can help us generate efficient codes for irregular applications using the inspector–executor approach. We also illustrate how to deal with some hard cases containing reshaping and strided accesses during the translation. Our experiments show promising results for the corresponding regular and irregular GA codes.

Running OpenMP applications efficiently on an everything-shared SDSM

Traditional software distributed shared memory (SDSM) systems modify the semantics of a real hardware shared memory system by relaxing the coherence semantic and by limiting the memory regions that are actually shared. These semantic modifications are done to improve performance of the applications using it. In this paper, we will show that a SDSM system that behaves like a real shared memory system (without the afore-mentioned relaxations) can also be used to execute OpenMP applications and achieve similar speedups as the ones obtained by traditional SDSM systems. This performance can be achieved by encouraging the cooperation between the SDSM and the OpenMP runtime instead of relaxing the semantics of the shared memory. In addition, techniques like boundaries alignment and page presend are demonstrated as very useful to overcome the limitations of the current SDSM systems.

Shared memory computing on clusters with symmetric multiprocessors and system area networks

Cashmere is a software distributed shared memory (S-DSM) system designed for clusters of server-class machines. It is distinguished from most other S-DSM projects by (1) the effective use of fast user-level messaging, as provided by modern system-area networks, and (2) a “two-level” protocol structure that exploits hardware coherence within multiprocessor nodes. Fast user-level messages change the tradeoffs in coherence protocol design; they allow Cashmere to employ a relatively simple directory-based coherence protocol. Exploiting hardware coherence within SMP nodes improves overall performance when care is taken to avoid interference with inter-node software coherence.We have implemented Cashmere on a Compaq AlphaServer/Memory Channel cluster, an architecture that provides fast user-level messages. Experiments indicate that a one-level, version of the Cashmere protocol provides performance comparable to, or slightly better than, that of TreadMarks' lazy release consistency. Comparisons to Compaq's Shasta protocol also suggest that while fast user-level messages make finer-grain software DSMs competitive, VM-based systems continue to outperform software-based access control for applications without extensive fine-grain sharing.Within the family of Cashmere protocols, we find that leveraging intranode hardware coherence provides a 37% performance advantage over a more straightforward one-level implementation. Moreover, contrary to our original expectations, noncoherent hardware support for remote memory writes, total message ordering, and broadcast, provide comparatively little in the way of additional benefits over just fast messaging for our application suite.

JIACKPT: A Recoverable Software Distributed Shared Memory System

Software distributed shared memory (DSM) system has constructed a virtual shared memory abstract on cluster, which combines the programmability of shared memory and fine scalability of cluster. So it is widely studied. Software DSM system is easy to fail because it is a distributed system, some kinds of fault tolerance are necessary for it to be more practical. A recoverable and portable software DSM system, JIACKPT (JIAjia with ChecKPoinTing), has been designed and implemented to tolerate the fault of system. JIACKPT, based on JIAJIA, has adopted the checkpointing technology. By maintaining the strict global consistent state and using some optimization techniques, JIACKPT has gotten high performance. The experimental results on an 8-node PC cluster show that the checkpoint overhead is less than 10% of the whole execution time when checkpoint is done once per minute. JIACKPT also has good portability and can run on several operating systems, such as Linux, Solaris, etc.

CAS-DSM: A Compiler Assisted Software Distributed Shared Memory

Traditional software Distributed Shared Memory (DSM) systems rely on the virtual memory management mechanisms to detect accesses to shared memory locations and maintain their consistency. The resulting involvement of the OS (kernel) and the associated overhead which is significant, can be avoided by careful compile time analysis and code instrumentation. In this paper, we propose such a Compiler Assisted Software support approach (CAS-DSM). In the CAS-DSM implementation, the involvement of the OS kernel is avoided by instrumenting the application code at the source level. The overhead caused by the execution of the instrumented code is reduced through several aggressive compile time optimizations. Finally, we also address the issue of reducing certain overheads in polling-based implementation of receiving asynchronous messages. We used SUIF, a public domain compiler tool, to implement compile time analysis, instrumentation and optimizations. We modified CVM, a publicly available software DSM to support the instrumentation inserted by the compiler. Detailed performance evaluation of CAS-DSM is reported using a set of Splash/Splash2 parallel application benchmarks on a distributed memory IBM SP-2 machine. CAS-DSM achieved moderate to good performance improvements for most of the applications compared to the original CVM implementation. Reducing the overheads in polling-based implementation improves the performance of CAS-DSM significantly resulting in an overall improvement of 12-52% over the original CVM implementation.

Memory management for multi-threaded software DSM systems

When software distributed shared memory (SDSM) systems provide multithreading to exploit cluster of symmetric multiprocessors (SMPs), a challenge is how to preserve memory consistency in a thread-safe way, which is known as “atomic page update problem”. In this paper, we show that this problem can be solved by creating two independent access paths to a physical page and by assigning different access permissions to them. Especially, we propose three new methods using System V shared memory inter-process communication (IPC), a new mdup() system call, and a fork() system call in addition to a known method using file mapping. The main contribution of this paper is to introduce various solutions to the atomic page update problem and to compare their characteristics extensively. Experiments carried out on a Linux-based cluster of SMPs and an IBM SP Night Hawk system show that the proposed methods overcome the drawbacks of the file mapping method such as high initialization cost and buffer cache flushing overhead. In particular, the method using a fork() system call preserves the whole address space to the application.

Priority Based Messaging for Software Distributed Shared Memory

Software Distributed Shared Memory (DSM) systems can be used to provide a coherent shared address space on multicomputers and other parallel systems without support for shared memory in hardware. The coherency software automatically translates shared memory accesses to explicit messages exchanged among the nodes in the system. Many applications exhibit a good performance on such systems but it has been shown that, for some applications, performance critical messages can be delayed behind less important messages because of the enqueuing behavior in the communication libraries used in current systems. We present in this paper a new portable communication library that supports priorities to remedy this situation. We describe an implementation of the communication library and a quantitative model that is used to estimate the performance impact of priorities for a typical situation. Using the model, we show that the use of high-priority communication reduces the latency of performance critical messages substantially over a wide range of network design parameters. The latency is reduced with up to 10–25% for each delaying low priority message in the queue ahead.

An Effective Logical Cache for a Clustered LRC-Based DSM System

As local-area workstation networks are widely available, the idea of offering a software distributed shared memory (SDSM) system across interconnects of clusters is quite an attractive alternative for compute-intensive applications. However, the higher cost of sending a message over an inter-cluster link compared to an intra-cluster one can limit applications' performance on a multi-cluster SDSM system. In this paper, we present the extensions that we have added to the SDSM TreadMarks, which provides the i>lazy release consistency (LRC) memory model, in order to adapt it to a loosely-coupled cluster-based platform. We have implemented a logical per-cluster cache that exploits cluster locality. By accessing the cache of its cluster, a processor can share data previously requested by a second processor of its cluster, thereby, minimizing, the cost of inter-cluster communication.

A Modular OpenMP Implementation for Clusters of Multiprocessors

This paper presents a prototype runtime system that serves as the backend of a source-to-source OpenMP compiler and enables the execution of OpenMP Fortran programs on SMPs, though hardware memory, and clusters of shared-memory multiprocessors, either through the hybrid (MPI+OpenMP) programming model or directly on top of Software Distributed Shared Memory (SDSM). Fork-Join multilevel parallelism is supported without modifications to the OpenMP model, since its shared by default philosophy has been adopted for the generated by the OpenMP compiler code. Specifically, the user-level thread stacks and the Fortran common blocks are allocated explicitly, though transparently to the programmer, in virtual memory. The internal runtime system operations are based on the combination of explicit communication and hardware shared-memory. The runtime system extends the Single Program Multiple Data (SPMD) execution model of existing SDSM libraries in a modular way, allowing their integration in our OpenMP execution environment. We report experimental results of OpenMP programs on both architectural platforms and two different operating systems.

OpenMP: Experiences, Implementations and Applications

The rapid emergence of OpenMP as a preferred parallel programming paradigm for small-to-medium scale parallelism is attracting the scientific community. OpenMP is an industry standard interface for shared-memory programming of parallel computer applications. This standard offers a way to write portable applications to a wide range of shared-memory machines or, through hybrid (OpenMP + MPI) programming model, to cluster of share-memory nodes. This special issue features six papers that explore the experiences of application developers in the use of the language and parallelization of applications with OpenMP. The first paper, MPI and OpenMP Paradigms on Cluster of SMP Architectures: the Vacancy Tracking Algorithm for Multi-Dimensional Array Transposition by Yun He and Chris H. Q. Ding, investigates remapping multi-dimensional arrays on a cluster of SMP architectures under OpenMP, MPI, and hybrid paradigms. The traditional method of array transpose needs an auxiliary array of the same size and a copy-back stage. The authors developed an in-place method using vacancy tracking cycles. The vacancy tracking algorithm outperforms the traditional 2-array method as demonstrated by extensive comparisons. In the second, paper Comparing MPI and OpenMP implementations of the 0-1 Knapsack Problem, Isabel Dorta, Coromoto Leon and Casiano Rodriguez present and compare two parallel implementations of the 0-1 Knapsack Problem. Using the C ++ programming language, two exact algorithms based on the Branch-and-Bound technique have been implemented. MPI has been used to develop the Message Passing algorithm and for the Shared Memory the OpenMP has been chosen. V. Blanco, L. Garcia, J. A Gonzalez, C. Rodriguez and G. Rodriguez, in their paper, A Performance Model for the Analysis of OpenMP Programs, attempt to build a bridge between theoretical complexity models and the performance analysis and tuning of OpenMP programs. They start with the assumption that the theoretical analysis of an OpenMP code produces a complexity function that gives an approach to the asymptotic number of operations performed by the algorithm. The shared-memory platform is characterized by a number of parameters that are computed through a benchmarking program. The authors propose an algorithm that determines the algorithmic constants associated with the complexity formula in terms of the architecture parameters. To predict the performance for a new computing platform, they use the parameters characterizing the platform to solve a random linear equation system. P. E. Handjidoukas, E. D. Polychronopoulos, and T. S. Papatheodorou present in their paper, A Modular OpenMP Implementation for Clusters of Multiprocessors, a prototype runtime system that provides support at the backend of the NANOS OpenMP compiler. This enables the execution of unmodified OpenMP Fortran programs on both SMPs and clusters of multiprocessors, either through the hybrid (MPI + OpenMP) programming model or directly on top of Software Distributed Shared Memory. F. Garcia Lopez and N. L. Frais Arrocha propose in their paper, Expanding the Synchronization Model for OpenMP, an extension to the OpenMP model consisting of a new and efficient synchronization model based on the Monitor model. The motivation behind this proposal is to support an application based on backtracking, branch and bound, and dynamic programming approaches. Finally, Mohammad Towhidul Islam, Parimala Thulasiraman, and Ruppa K. Thulasiram, in their paper Implementation of Ant Colony Optimization Algorithm for Mobile Ad Hoc Network Applications: OpenMP Experiences, present experiences in design, development and implementation of a parallel algorithm for mobile ad hoc networks (MANETs) using the Ant Colony Optimization technique on a shared memory architecture with OpenMP. They have experimented with three scheduling policies provided by OpenMP for varying data sizes and compare performance results with MPI. I would like to thank each of the participating reviewers for their help in ensuring the quality of the papers in this issue. Ami Marowka School of Computer Science and Engineering The Computer Aided Design Laboratory The Hebrew University of Jerusalem

Transparent adaptation of sharing granularity in MultiView‐based DSM systems

AbstractIn this paper we propose a mechanism that provides distributed shared memory (DSM) systems with a flexible sharing granularity. The size of the shared memory units is dynamically determined by the system during runtime. This size can range from that of a single variable up to the size of the entire shared memory space. During runtime, the DSM transparently adapts the granularity to the memory access pattern of the application in each phase of its execution. This adaptation, called ComposedView, provides efficient data sharing in software DSM while preserving sequential consistency. Neither complex code analysis nor annotation by the programmer or the compiler are required. Our experiments indicate a substantial performance boost (up to 80% speed‐up improvement) when running a large set of applications using our method, compared to running these benchmark applications with the best fixed granularity. Copyright © 2001 John Wiley & Sons, Ltd.

Accurate data redistribution cost estimation in software distributed shared memory systems

Distributing data is one of the key problems in implementing efficient distributed-memory parallel programs. The problem becomes more difficult in programs where data redistribution between computational phases is considered. The global data distribution problem is to find the optimal distribution in multi-phase parallel programs. Solving this problem requires accurate knowledge of data redistribution cost. We are investigating this problem in the context of a software distributed shared memory (SDSM) system, in which obtaining accurate redistribution cost estimates is difficult. This is because SDSM communication is implicit: It depends on access patterns, page locations, and the SDSM consistency protocol. We have developed integrated compile- and run-time analysis for SDSM systems to determine accurate redistribution cost estimates with low overhead. Our resulting system, SUIF-Adapt, can efficiently and accurately estimate execution time, including redistribution, to within 5% of the actual time in all of our test cases and is often much closer. These precise costs enable SUIF-Adapt to find efficient global data distributions in multiple-phase programs.

Dynamic data prefetching in home-based software DSMs

A major overhead in software DSM (Distributed Shared Memory) is the cost of remote memory accesses necessitated by the protocol as well as induced by false sharing. This paper introduces a dynamic prefetching method implemented in the JIAJIA software DSM to reduce system overhead caused by remote accesses. The prefetching method records the interleaving string of INV (invalidation) and GETP (getting a remote page) operations for each cached page and analyzes the periodicity of the string when a page is invalidated on a lock or barrier. A prefetching request is issued after the lock or barrier if the periodicity analysis indicates that GETP will be the next operation in the string. Multiple prefetching requests are merged into the same message if they are to the same host. Performance evaluation with eight well-accepted benchmarks in a cluster of sixteen Power PC workstations shows that the prefetching scheme can significantly reduce the page fault overhead and as a result achieves a performance increase of 15%–20% in three benchmarks and around 8%–10% in another three. The average extra traffic caused by useless prefetches is only 7%–13% in the evaluation.