SPMD Applications Research Articles

Scalable performance analysis method for SPMD applications

The analysis of parallel scientific applications allows us to understand their computational and communication behavior. One way of obtaining performance information is through performance tools. One such tool is parallel application signatures for performance prediction (PAS2P), based on parallel application repeatability, focusing on performance analysis and prediction. The same resources that execute the parallel application are used to perform its analysis, creating a machine independent model of the application and identifying its common patterns. However, the analysis is costly in terms of execution time due to the high number of synchronization communications performed by PAS2P, degrading performance as the number of processes increases. To solve this problem, we propose a model that reduces data dependency between processes, reducing the number of communications performed by PAS2P in the analysis stage and taking advantage of the characteristics of single program, multiple sata applications. Our analysis proposal allows us to decrease the analysis time by 29 times when the application scales to 256 processes, while keeping error levels below 11% in the runtime prediction. It is important to mention that the analysis time is not considerably affected by increasing the number of application processes.

ASYMPTOTIC PEAK UTILISATION IN HETEROGENEOUS PARALLEL CPU/GPU PIPELINES: A DECENTRALISED QUEUE MONITORING STRATEGY

Widespread heterogeneous parallelism is unavoidable given the emergence of General-Purpose computing on graphics processing units (GPGPU). The characteristics of a Graphics Processing Unit (GPU)—including significant memory transfer latency and complex performance characteristics—demand new approaches to ensuring that all available computational resources are efficiently utilised. This paper considers the simple case of a divisible workload based on widely-used numerical linear algebra routines and the challenges that prevent efficient use of all resources available to a naive SPMD application using the GPU as an accelerator. We suggest a possible queue monitoring strategy that facilitates resource usage with a view to balancing the CPU/GPU utilisation for applications that fit the pipeline parallel architectural pattern on heterogeneous multicore/multi-node CPU and GPU systems. We propose a stochastic allocation technique that may serve as a foundation for heuristic approaches to balancing CPU/GPU workloads.

Juggle: addressing extrinsic load imbalances in SPMD applications on multicore computers

We investigate proactive dynamic load balancing on multicore systems, in which threads are continually migrated to reduce the impact of processor/thread mismatches. Our goal is to enhance the flexibility of the SPMD-style programming model and enable SPMD applications to run efficiently in multiprogrammed environments. We present Juggle, a practical decentralized, user-space implementation of a proactive load balancer that emphasizes portability and usability. In this paper we assume perfect intrinsic load balance and focus on extrinsic imbalances caused by OS noise, multiprogramming and mismatches of threads to hardware parallelism. Juggle shows performance improvements of up to 80 % over static load balancing for oversubscribed UPC, OpenMP, and pthreads benchmarks. We also show that Juggle is effective in unpredictable, multiprogrammed environments, with up to a 50 % performance improvement over the Linux load balancer and a 25 % reduction in performance variation. We analyze the impact of Juggle on parallel applications and derive lower bounds and approximations for thread completion times. We show that results from Juggle closely match theoretical predictions across a variety of architectures, including NUMA and hyper-threaded systems.

A tool for efficient execution of SPMD applications on multicore clusters

Abstract A challenge for parallel programmers is to efficiently execute traditional MPI applications, designed to be run in a cluster of single core nodes, on a multicore cluster. Multicore clusters include communication heterogeneities which have to be handled carefully to improve efficiency and speedup. This research presents an execution tool developed for SPMD applications which is focused on managing communications heterogeneities, distributing the workload among cores and enhancing parallel performance on multicore clusters. Our tool has been designed through using an execution methodology which includes mapping and scheduling strategies. The tool integrates five modules which give programmers a method to execute their applications efficiently. This tool is centered on improving SPMD applications designed to use MPI for communications. These applications were selected because they are the most commonly used in parallel computing. Also, these applications are chosen due to their data synchronization and communications volumes which can generate communication imbalance issues. The novel contribution of this tool is to permit programmers to find a minimum execution time, while the efficiency level is maintained over a defined threshold. Our tool has been tested in different multicore clusters and with a set of scientific applications. The results obtained show a considerable improvement in the applications efficiency when the tool is applied.

A Heuristic Approach for the Automatic Insertion of Checkpoints in Message-Passing Codes

Abstract: Checkpointingtoolsmaybetypically implementedattwodifferentabstrac-tion levels: at the system level or at the application level. The latter has become a morepopular alternative due to its flexibility and the possibility of operating in different en-vironments. However, application-level checkpointing tools often require the user tomanually insert checkpoints in order to ensure that certain requirements are met (e.g.forcing checkpoints to be taken at the user code and not inside kernel routines). Theapproach presented in this work is twofold. First, a spatial coordination protocol forcheckpointing parallel SPMD applications is proposed, based on forcing checkpoints tobe taken at the same places in the application code by all processes. Thus, global con-sistency is achieved without adding any new runtime communications or piggybackeddata, and without the need to use specific fault-tolerant message-passing implemen-tations. Second, the paper also introduces a compilation technique for the automaticinsertion of checkpoints using the spatial coordination protocol, based on a static anal-ysis of communications and a heuristic analysis of computational load. These analysescan also be used to achieve automatic checkpoint insertion in approaches based onclassical protocols, such as uncoordinated checkpointing or distributed snapshots.Key Words: Fault tolerance, checkpointing, parallel programming, message-passing,compiler-supportCategory: C.4, D.1.3

Background Level of Pops in Ground Water Assessed on Chemical and Toxicity Analysis of Exposed Semipermeable Membrane Devices

Persistent compounds are present around almost the entire world. The level of contamination in very old groundwater sources (Cennoman bedrock Mesozoic, approximately 100 millions year old) was assessed. This offers an information about realistic natural background. Together with chemical analysis a toxicity evaluation of sampled sites was performed. Semipermeable membrane devices were applied as a sampling system. Exposed SPMDs were analyzed both for chemical contain of POPs and toxicity properties. The chemical analyses of PAHs were made by HPLC-FLD, PCBs and OCPs were analysed by GC/MS/MS on GCQ or PolarisQ (Thermoquest). Toxicity bioassays on alga Desmodesmus subspicatus, bacteria Vibrio fischeri and crustacean Daphnia magna was performed. The results show very low contamination of groundwater with POPs with concentrations close to detection limits of applied analytical tools. Even this low contamination was possible to rank based on the obtained toxicity data. Toxicity proved to be a good parameter for determination of relative POPs contamination where concentration is near to detection limits and thus correct determination of all POPs cannot be undertaken. Although contamination levels were found to be very low, a secondary contamination of PCBs through the bedrock was observed. Organochlorine pesticides were found at a sampling site near a mouth of the ground watershed. Applied toxicity tests confirmed the presence of toxic substances and marked sites of higher contamination. Application of toxicological parameter Vtox allowed the ranking of assessed sites by their contamination level even in cases where concentrations of pollutants were near or under detection limits and it was not therefore possible to rank the sites on the basis of chemical parameters. Toxicity response of bioassays obtained on SPMDs exposed in clean groundwater can be used as a background toxicity values for further SPMD applications. Secondary contamination with PCBs and pesticides was detected in Cennoman groundwater. Toxicity evaluation of SPMD extract can be used as an effective tool for ranking of general level of water contamination.

Integrated performance models for spmd applications and mimd architectures

This paper introduces queuing network models for the performance analysis of SPMD applications executed on general-purpose parallel architectures such as MIMD and clusters of workstations. The models are based on the pattern of computation, communication, and I/O operations of typical parallel applications. Analysis of the models leads to the definition of speedup surfaces which capture the relative influence of processors and I/O parallelism and show the effects of different hardware and software components on the performance. Since the parameters of the models correspond to measurable program and hardware characteristics, the models can be used to anticipate the performance behavior of a parallel application as a function of the target architecture (i.e., number of processors, number of disks, I/O topology, etc).

Prophet: automated scheduling of SPMD programs in workstation networks

Obtaining efficient execution of parallel programs in workstation networks is a difficult problem for the user. Unlike dedicated parallel computer resources, network resources are shared, heterogeneous, vary in availability, and offer communication performance that is still an order of magnitude slower than parallel computer interconnection networks. Prophet, a system that automatically schedules data parallel SPMD programs in workstation networks for the user, has been developed. Prophet uses application and resource information to select the appropriate type and number of workstations, divide the application into component tasks and data across these workstations, and assign tasks to workstations. This system has been integrated into the Mentat parallel processing system developed at the University of Virginia. A suite of scientific Mentat applications has been scheduled using Prophet on a heterogeneous workstation network. The results are promising and demonstrate that scheduling SPMD applications can be automated with good performance. Copyright © 1999 John Wiley & Sons, Ltd.

Para++: A High Level C++ Interface for Message Passing

This paper describes a high level C++ interface for message passing applications. Our interface is built on top of PVM and MPI. The two main contributions are to allow a quicker design of parallel applications without any important drop of performances. We introduce two levels of tasks and use C++ streams for communications. We also present a performance study over both PVM and MPI to show the overhead of our implementation. Finally, we detail two applications based on the heat equation to explain howPara++ can be used for SPMD and MPMD applications.

Scheduling parallel applications in distributed networks

Prophet is a run-time scheduling system designed to support the efficient execution of parallel applications written in the Mentat programming language (Grimshaw, 1993). Prior results demonstrated that SPMD applications could be scheduled automatically in an ethernet-based local-area workstation network with good performance (Weissman and Grimshaw, 1994 and 1995). This paper describes our recent efforts to extend Prophet along several dimensions: improved overhead control, greater resource sharing, greater resource heterogeneity, wide-area scheduling, and new application types. We show that both SPMD and task parallel applications can be scheduled effectively in a shared heterogeneous LAN environment containing ethernet and ATM networks by exploiting the application structure and dynamic run-time information.

Web‐based framework for distributed computing

Parallel and distributed computing on a cluster of workstations is being increasingly applied to a variety of large size computational problems. Several software systems have been developed that make distributed computing available to an application programmer. However, these systems either are not Web-based or lack a collaborative environment. The increasing use of Web technology for Internet and Intranet applications is making the Web an attractive framework for solving distributed applications, in particular, because the interface can be made platform-independent. In this paper we describe JAVADC, a Web–Java-based framework for the execution of parallel SPMD applications which use PVM, pPVM and MPI. We also discuss the design of a collaborative, distributed computing environment, Arcade, focused on more general programming paradigms for multidisciplinary applications. Arcade is a Web-based integrated environment which provides support in all phases of the development of general multidisciplinary applications including the design, execution, monitoring and control of such applications. © 1997 John Wiley & Sons, Ltd.