Bulk Synchronous Parallel Applications Research Articles

MigPF: Towards on self-organizing process rescheduling of Bulk-Synchronous Parallel applications

Migration is a known technique to offer process rescheduling, being especially pertinent for Bulk Synchronous Parallel (BSP) programs. Such programs are organized in a set of supersteps, in which the slowest process always determines the synchronization time. This approach motivated us to develop a first model called MigBSP, which combines computation, communication, and migration costs metrics for process rescheduling decisions. In this paper, a new model named MigPF enhances our previous work in three aspects: (i) a different algorithm for detecting imbalance situations, which considers the performance of each processor, including its enclosed processes, instead of individual times of each process; (ii) an improvement on the rescheduling reactivity through shortening the interval for the next migration call when imbalance situations arise; (iii) a new algorithm for self-organizing the migratable processes and their destinations. Particularly, this third item represents our main scientific contribution, not only in terms of the MigBSP context, but also in a broader one that covers the entire BSP landscape. The algorithm assembles n possible rescheduling plans (where n is the number of processes) and provides a prediction function (pf) that chooses the most profitable rescheduling plan when compared to the current mapping. We developed a MigPF prototype with the Adaptive MPI (AMPI) library, which offers a standard framework for implementing migration-based load balancing policies. We tested this prototype against other built-in AMPI rescheduling policies with two scientific applications: shear sorting and fractal image compression. The results revealed performance gains up to 41% and an overhead limited to 5% when migrations do not take place.

Flexible Management on BSP Process Rescheduling: Offering Migration at Middleware and Application Levels

This article describes the rationales for developing jMigBSP - a Java programming library that offers object rescheduling. It was designed to work on grid computing environments and offers an interface that follows the BSP (Bulk Synchronous Parallel) style. jMigBSP’s main contribution focuses on the rescheduling facility in two different ways: (i) by using migration directives on the application coded irectly and (ii) through automatic load balancing at middleware level. Especially, this second idea is feasible thanks to the Java’s inheritance feature, in which transforms a simple jMigBSP application in amigratable one only by changing a single line of code. In addition, the presented library makes the object interaction easier by providing one-sided message passing directives and hides network latency through asynchronous communications. Finally, we developed three BSP applications: (i) Prefix Sum; (ii) Fractal Image Compression (FIC) and; (iii) Fast Fourier Transform (FFT).They show our library as viable solution to offer load balancing on BSP applications. Specially, the FIC results present gains up to 37% when applying migration directives inside the code. Finally, the FFT tests emphasize strength of jMigBSP. In this situation, it outperforms a native library denoted BSPlib when migration facilities take place. Keywords: Bulk Synchronous Parallel, rescheduling, Java, adaptation, object migration, grid computing.

OBSERVING THE IMPACT OF MULTIPLE METRICS AND RUNTIME ADAPTATIONS ON BSP PROCESS RESCHEDULING

Process rescheduling is an useful mechanism to offer runtime load balancing, mainly in dynamic and heterogeneous environments. In this context, we developed a model called MigBSP which controls the process migration on BSP (Bulk Synchronous Parallel) applications. A BSP application is divided in one or more supersteps, each one containing both computation and communication phases followed by a barrier synchronization. Since the barrier waits for the slowest process, MigBSP's final objective is to adjust the processes location in order to reduce the supersteps' times. Its novel ideas are twofold. The former is represented by the combination of three metrics - Memory, Computation and Communication - in order to measure the Potential of Migration of each BSP process. The second idea consists in offering efficient adaptations that work on the rescheduling frequency. Both ideas turn MigBSP a viable model for getting performance on BSP applications. Meanwhile, it provides a low overhead on application execution when migrations do not take place. This paper presents MigBSP's algorithms, the parallel machine organization, some experimental results and related work.

Checkpointing BSP parallel applications on the InteGrade Grid middleware

AbstractInteGrade is a Grid middleware infrastructure that enables the use of idle computing power from user workstations. One of its goals is to support the execution of long‐running parallel applications that present a considerable amount of communication among application nodes. However, in an environment composed of shared user workstations spread across many different LANs, machines may fail, become inaccessible, or may switch from idle to busy very rapidly, compromising the execution of the parallel application in some of its nodes. Thus, to provide some mechanism for fault tolerance becomes a major requirement for such a system. In this paper, we describe the support for checkpoint‐based rollback recovery of Bulk Synchronous Parallel applications running over the InteGrade middleware. This mechanism consists of periodically saving application state to permit the application to restart its execution from an intermediate execution point in case of failure. A precompiler automatically instruments the source code of a C/C++ application, adding code for saving and recovering application state. A failure detector monitors the application execution. In case of failure, the application is restarted from the last saved global checkpoint. Copyright © 2005 John Wiley & Sons, Ltd.

Effective distributed scheduling of parallel workloads

We present a distributed algorithm for time-sharing parallel workloads that is competitive with coscheduling. Implicit scheduling allows each local scheduler in the system to make independent decisions that dynamically coordinate the scheduling of cooperating processes across processors. Of particular importance is the blocking algorithm which decides the action of a process waiting for a communication or synchronization event to complete. Through simulation of bulk-synchronous parallel applications, we find that a simple two-phase fixed-spin blocking algorithm performs well; a two-phase adaptive algorithm that gathers run-time data on barrier wait-times performs slightly better. Our results hold for a range of machine parameters and parallel program characteristics. These findings are in direct contrast to the literature that states explicit coscheduling is necessary for fine-grained programs. We show that the choice of the local scheduler is crucial, with a priority-based scheduler performing two to three times better than a round-robin scheduler. Overall, we find that the performance of implicit scheduling is near that of coscheduling (+/- 35%), without the requirement of explicit, global coordination.