Execution Of Parallel Applications Research Articles

Performance Analysis and Optimization of the OP2 Framework on Many-Core Architectures

This paper presents a benchmarking, performance analysis and optimization study of the OP2 ‘active’ library, which provides an abstraction framework for the parallel execution of unstructured mesh applications. OP2 aims to decouple the scientific specification of the application from its parallel implementation, and thereby achieve code longevity and near-optimal performance through re-targeting the application to execute on different multi-core/many-core hardware. Runtime performance results are presented for a representative unstructured mesh application on a variety of many-core processor systems, including traditional X86 architectures from Intel (Xeon based on the older Penryn and current Nehalem micro-architectures) and GPU offerings from NVIDIA (GTX260, Tesla C2050). Our analysis demonstrates the contrasting performance between the use of CPU (OpenMP) and GPU (CUDA) parallel implementations for the solution of an industrial-sized unstructured mesh consisting of about 1.5 million edges. Results show the significance of choosing the correct partition and thread-block configuration, the factors limiting the GPU performance and insights into optimizations for improved performance.

Expanding the volunteer computing scenario: A novel approach to use parallel applications on volunteer computing

Nowadays, high performance computing is being improved thanks to different platforms like clusters, grids, and volunteer computing environments. Volunteer computing is a type of distributed computing paradigm in which a large number of computers, volunteered by members of the general public, provide computing and storage resources for the execution of scientific projects.Currently, volunteer computing is used for high throughput computing, but it is not used for parallel applications based on MPI due to the difficulty in communicating among computing peers. As Sarmenta and Hirano (1999) [2] said, there are several research issues to be solved in volunteer computing platforms, one of them is ‘implementing other types of applications’. In fact, the BOINC team (the well known volunteer computing development platform) is requesting ‘MPI-like support’ for volunteer computing as one of its hot topics.This paper proposes an extension of the usual capabilities of volunteer computing and desktop grid systems by allowing the execution of parallel applications, based on MPI, on these types of platforms. The goal of this paper is to describe a method to transform and adapt MPI applications in order to allow the execution of this type of applications on volunteer computing platforms and desktop grids. The proposed approach does not require an MPI installation on the client’s side, which keeps the software to be installed on the client quite lightweight. The paper describes the prototype built, and also shows the evaluation made of this prototype, in order to test the viability of the idea proposed in this article, with promising results.

Reliable management of checkpointing and application data in opportunistic grids

Abstract Opportunistic computational grids use idle processor cycles from shared machines to enable the execution of long-running parallel applications. Besides computational power, these applications may also consume and generate large amounts of data, requiring an efficient data storage and management infrastructure. In this article, we present an integrated middleware infrastructure that enables the use of not only idle processor cycles, but also unused disk space of shared machines. Our middleware enables the reliable distributed storage of application data in the shared machines in a redundant and fault-tolerant way. A checkpointing-based mechanism monitors the execution of parallel applications, saves periodical checkpoints in the shared machines, and in case of node failures, supports the application migration across heterogeneous grid nodes. We evaluate the feasibility of our middleware using experiments and simulations. Our evaluation shows that the proposed middleware promotes important improvements in grid data management reliability while imposing a low performance overhead.

Performance evaluation of an application-level checkpointing solution on grids

In recent years there has been a significant effort to develop middleware that facilitates the execution of applications on Grid infrastructures. However, support for fault-tolerant execution continues to be scarce. The CPPC-G framework is a service-based architecture designed to provide efficient fault-tolerant mechanisms for the execution of sequential and parallel applications on grids. Applications to be managed by CPPC-G are expected to be preprocessed with CPPC (ComPiler for Portable Checkpointing), a tool for automatically inserting portable checkpoint instrumentation into the code of parallel applications. Built on top of existing Globus services, CPPC-G services are in charge of submitting and monitoring CPPC applications, managing generated checkpoint files, detecting failures and automatically restarting failed executions. In this paper the feasibility of this approach is assessed by measuring the performance of CPPC-G, quantitatively addressing its impact on application performance. Results show that the increase in overall throughput and availability comes with minor performance degradation.

The Scalasca performance toolset architecture

AbstractScalasca is a performance toolset that has been specifically designed to analyze parallel application execution behavior on large‐scale systems with many thousands of processors. It offers an incremental performance‐analysis procedure that integrates runtime summaries with in‐depth studies of concurrent behavior via event tracing, adopting a strategy of successively refined measurement configurations. Distinctive features are its ability to identify wait states in applications with very large numbers of processes and to combine these with efficiently summarized local measurements. In this article, we review the current toolset architecture, emphasizing its scalable design and the role of the different components in transforming raw measurement data into knowledge of application execution behavior. The scalability and effectiveness of Scalasca are then surveyed from experience measuring and analyzing real‐world applications on a range of computer systems. Copyright © 2010 John Wiley & Sons, Ltd.

Application execution management on the InteGrade opportunistic grid middleware

The InteGrade project is a multi-university effort to build a novel grid computing middleware based on the opportunistic use of resources belonging to user workstations. The InteGrade middleware currently enables the execution of sequential, bag-of-tasks, and parallel applications that follow the BSP or the MPI programming models. This article presents the lessons learned over the last five years of the InteGrade development and describes the solutions achieved concerning the support for robust application execution. The contributions cover the related fields of application scheduling, execution management, and fault tolerance. We present our solutions, describing their implementation principles and evaluation through the analysis of several experimental results.

Automatic design of application-specific reconfigurable processor extensions with UPaK synthesis kernel

This article presents a new tool for automatic design of application-specific reconfigurable processor extensions based on UPaK (Abstract U nified Pa tterns Based Synthesis K ernel for Hardware and Software Systems). We introduce a complete design flow that identifies new instructions, selects specific instructions and schedules a considered application on the newly created reconfigurable architecture. The identified extensions are implemented as specialized sequential or parallel instructions. These instructions are executed on a reconfigurable unit implementing all merged patterns. Our method uses specially developed algorithms for subgraph isomorphism that are implemented as graph matching constraints. These constraints together with separate algorithms are able to efficiently identify computational patterns and carry out application mapping and scheduling. Our methods can handle both time-constrained and resource-constrained scheduling. Experimental results show that the presented method provides high coverage of application graphs with small number of patterns and ensures high application execution speedup both for sequential and parallel application execution with reconfigurable processor extensions implementing selected patterns.

Triva: Interactive 3D visualization for performance analysis of parallel applications

The successful execution of parallel applications in grid infrastructures depends directly on a performance analysis that takes into account the grid characteristics, such as the network topology and resources location. This paper presents Triva, a software analysis tool that implements a novel technique to visualize the behavior of parallel applications. The proposed technique explores 3D graphics to show the application behavior together with a description of the resources, highlighting communication patterns, the network topology and a visual representation of a logical organization of the resources. We have used a real grid infrastructure to execute and trace applications composed of thousands of processes.

Clock synchronization in Cell/B.E. traces

AbstractCell/B.E. is a heterogeneous multicore processor that was designed for the efficient execution of parallel and vectorizable applications with high computation and memory requirements. The transition to multicores introduces the challenge of providing tools that help programmers tune the code running on these architectures. Tracing tools, in particular, often help locate performance problems related to thread and process communication. A major impediment to implementing tracing on Cell is the absence of a common clock that can be accessed at low cost from all cores. The OS clock is costly to access from the auxiliary cores and the hardware timers cannot be simultaneously set on all the cores. In this paper, we describe an offline trace analysis algorithm that assigns wall‐clock time to trace records based on their thread‐local time stamps and event order. Our experiments on several Cell SDK workloads show that the indeterminism in assigning wall‐clock time to events is low, on average 20–40 clock ticks (translating into 1.4–2.8 µs on the system used in our experiments). We also show how various practical problems, such as the imprecision of time measurement, can be overcome. Copyright © 2009 John Wiley & Sons, Ltd.

Special Issue: Advanced Strategies in Grid Environments—Models and Techniques for Scheduling and Programming

Special Issue: Advanced Strategies in Grid Environments—Models and Techniques for Scheduling and Programming

A distributed Key Message algorithm to optimize the communication in clusters

The Key Message (KM) approach prioritizes communications along the critical path to speed up the execution of parallel applications in a cluster environment. However, the current KM approaches (i.e, the Static KM (SKM) and the dynamic KM with a central predictor (CDKM)) either lack the capability of adapting to dynamic environment or suffer from the expensive centralized control to predict dynamic critical path rendering them to be suboptimal. In this paper, we introduce a dynamic and distributed KM algorithm (DDKM) that takes this dynamic nature into account while avoiding overheads of central control, thus, overcoming the drawbacks of its predecessors. To evaluate the DDKM algorithm, we implement it and simulate it in a cluster environment whose communication network (e.g., shared bus) can be modeled as an M/D/1. We compare DDKM with current KM approaches. The performance of DDKM is close to that of CDKM, and much better than SKM up to 28.9% when the network is highly congested. These results demonstrate that DDKM is a promising optimization in a real cluster environment.

Robust resource allocation in a cluster based imaging system

Recently there has been an increased demand for imaging systems in support of high-speed digital printing. The required increase in performance in support of such systems can be accomplished through an effective parallel execution of image processing applications in a distributed cluster computing environment. The output of the system must be presented to a raster based display at regular intervals, effectively establishing a hard deadline for the production of each image. Failure to complete a rasterization task before its deadline will result in an interruption of service that is unacceptable. The goal of this research was to derive a metric for measuring robustness in this environment and to design a resource allocation heuristic capable of completing each rasterization task before its assigned deadline, thus, preventing any service interruptions. We present a mathematical model of such a cluster based raster imaging system, derive a robustness metric for evaluating heuristics in this environment, and demonstrate using the metric to make resource allocation decisions. The heuristics are evaluated within a simulation of the studied raster imaging system. We clearly demonstrate the effectiveness of the heuristics by comparing their results with the results of a resource allocation heuristic commonly used in this type of system.

A novel strategy for building interoperable MPI environment in heterogeneous high performance systems

Breakthrough advances in microprocessor technology and efficient power management have altered the course of development of processors with the emergence of multi-core processor technology, in order to bring higher level of processing. The utilization of many-core technology has boosted computing power provided by cluster of workstations or SMPs, providing large computational power at an affordable cost using solely commodity components. Different implementations of message-passing libraries and system softwares (including Operating Systems) are installed in such cluster and multi-cluster computing systems. In order to guarantee correct execution of message-passing parallel applications in a computing environment other than that originally the parallel application was developed, review of the application code is needed. In this paper, a hybrid communication interfacing strategy is proposed, to execute a parallel application in a group of computing nodes belonging to different clusters or multi-clusters (computing systems may be running different operating systems and MPI implementations), interconnected with public or private IP addresses, and responding interchangeably to user execution requests. Experimental results demonstrate the feasibility of this proposed strategy and its effectiveness, through the execution of benchmarking parallel applications.

WSPE: a peer‐to‐peer grid programming environment

AbstractGrid programming environments are tools designed to isolate users from issues such as heterogeneity, scalability and adaptability, thus simplifying the use of a Grid infrastructure. This paper presents work stealing programming environment (WSPE), a Grid programming environment for Grid‐unaware applications. WSPE can be defined as a programming environment, since it is a software tool that presents both a programming model and an execution system. The WSPE consists of a simple programming interface and a fully decentralized runtime system following a peer‐to‐peer organization, both presented in this paper. It's runtime system employs a new scheduling mechanism, known as round stealing, inspired by the idea of work stealing. The main focus of our work is to research on methods to achieve efficient execution of parallel applications in a Grid computing infrastructure. By simulation, we show that our scheduling mechanism outperforms a more traditional mechanism in a Grid environment. We also demonstrate how an appropriate choice for a network overlay mechanism can further improve the execution efficiency. Copyright © 2009 John Wiley & Sons, Ltd.

Parallel computing on clusters and enterprise grids: practice and experience

We assert that companies can make more money and research institutions can improve their performance if inexpensive clusters and enterprise grids are exploited. In this paper, we have demonstrated that our claim is valid by showing the study of how programming environments, tools and middleware could be used for the execution of parallel and sequential applications, multiple parallel applications executing simultaneously on a non-dedicated cluster, and parallel applications on an enterprise grid and that the execution performance was improved. For this purpose an execution environment, and parallel and sequential benchmark applications selected for, and used in, the experiments were characterised.

Malleable iterative MPI applications

AbstractMalleability enables a parallel application's execution system to split or merge processes modifying granularity. While process migration is widely used to adapt applications to dynamic execution environments, it is limited by the granularity of the application's processes. Malleability empowers process migration by allowing the application's processes to expand or shrink following the availability of resources. We have implemented malleability as an extension to the process checkpointing and migration (PCM) library, a user‐level library for iterative message passing interface (MPI) applications. PCM is integrated with the Internet Operating System, a framework for middleware‐driven dynamic application reconfiguration. Our approach requires minimal code modifications and enables transparent middleware‐triggered reconfiguration. Experimental results using a two‐dimensional data parallel program that has a regular communication structure demonstrate the usefulness of malleability. Copyright © 2008 John Wiley & Sons, Ltd.

A framework for characterizing overlap of communication and computation in parallel applications

Effective overlap of computation and communication is a well understood technique for latency hiding and can yield significant performance gains for applications on high-end computers. In this paper, we propose an instrumentation framework for message-passing systems to characterize the degree of overlap of communication with computation in the execution of parallel applications. The inability to obtain precise time-stamps for pertinent communication events is a significant problem, and is addressed by generation of minimum and maximum bounds on achieved overlap. The overlap measures can aid application developers and system designers in investigating scalability issues. The approach has been used to instrument two MPI implementations as well as the ARMCI system. The implementation resides entirely within the communication library and thus integrates well with existing approaches that operate outside the library. The utility of the framework is demonstrated by analyzing communication-computation overlap for micro-benchmarks and the NAS benchmarks, and the insights obtained are used to modify the NAS SP benchmark, resulting in improved overlap.

PARALLELIZATION CONCEPTS AND APPLICATIONS FOR THM COUPLED FINITE ELEMENT PROBLEMS

Numerical analysis of thermo-hydro-mechanical coupled finite element problems in fractured or porous media involves significant computational resources and parallel computing is a way to enable the use of the necessary computer power of SMP machines or clusters. An interdisciplinary collaboration is essential to port large scale and complex software applications efficiently to parallel platforms with shared or distributed memory. A comprehensive and continuously developed scientific software concept is required to treat the aspects of parallelization. We describe a resourceful and practicable software environment that enables the execution of serial or parallel applications, and which makes a transparent administration of the source code for a scientific group possible. Such software environments are difficult to design and implement, especially when solution adaptive techniques are used and three-dimensional problems on complex geometries are faced, which yield the use of unstructured meshes. In this paper, we present object-oriented software concepts and parallelization methods for serial/parallel runs and coupled/non-coupled schemes on single or multiple processes. Applications of different numerical simulations on different parallel computer platforms are described and compared to demonstrate the computational efficiency of the developed structures and procedures.

A study of the concurrent execution of parallel and sequential applications on a non-dedicated cluster

Computers of a non-dedicated cluster are often idle (users attend meetings, have lunch or coffee breaks) or lightly loaded (users carry out simple computations to support problem solving activities). These underutilised computers can be employed to execute parallel applications. Thus, these computers can be shared by parallel and sequential applications, which could lead to the improvement of their execution performance. However, there is a lack of experimental study showing the applications’ performance and the system utilization of executing parallel and sequential applications concurrently and concurrent execution of multiple parallel applications on a non-dedicated cluster. Here we present the result of an experimental study into load balancing based scheduling of mixtures of NAS Parallel Benchmarks and BYTE sequential applications on a very low cost non-dedicated cluster. This study showed that the proposed sharing provided performance boost as compared to the execution of the parallel load in isolation on a reduced number of computers and better cluster utilization. The results of this research were used not only to validate other researchers’ result generated by simulation but also to support our research mission of widening the use of non-dedicated clusters. Our promising results obtained could promote further research studies to convince universities, business and industry, which require a large amount of computing resources, to run parallel applications on their already owned non-dedicated clusters.

The Performance of a Parallel TSP Program and Byte Sequential Benchmarks Executing on a Shared Cluster

Although individual PCs of a cluster are used by their owners to run sequential applications (local jobs), the cluster as a whole or its subset can also be employed to run parallel applications (cluster jobs) even during working hours. This implies that these computers have to be shared by parallel and sequential applications, which could lead to the improvement of the execution performance and resource utilization. However, there is a lack of experimental study showing the behavior and performance of executing parallel and sequential applications concurrently on a non-dedicated cluster. The result of such research would be beneficial for the development of new global scheduling algorithms. We present the result of an experimental study into scheduling of a mixture of parallel and sequential applications on a non-dedicated cluster. The aim of this study is to learn how the concurrent execution of a communication intensive parallel application and sequential applications influences their execution performance and utilization of the cluster.