BSP Cost Model Research Articles

An Efficient CGM-Based Parallel Algorithm Solving the Matrix Chain Ordering Problem

This study focuses on the parallel resolution of the matrix chain ordering problem and the optimal convex polygon triangulation problem on the Coarse grain multicomputer model (CGM for short). There has been intensive work on the parallelization of these dynamic programming problems in PRAM, including the use of systolic arrays, but a BSP/CGM solution is necessary for ease of implementation and portability. Our CGM algorithm is based on Yao's sequential solution running in O(n2) time and O(n2) space. This CGM algorithm uses p processors, each with O(n/p) local memory. It requires at most O(S/p×n2) running time with S communication rounds and with S/p<1. Our algorithm performs better than the algorithm proposed in 2012 by Dilson and Marco when S is less than n/p. We offer several ways of partitioning the problem to solve and study the impact of each partitioning algorithm performance. A CGM solution exists based on Yao's algorithm, but the subdivision of tasks is defined according to the BSP cost model. In this paper, we propose a solution based only on the CGM model specifications. Note that S is the number of super-steps of the CGM algorithm.

PREFACE: Special Issue on High-Level Parallel Programming and Applications

Many applications of computing require performance levels attainable only on parallel architectures. Such systems are now readily available as their price/performance ratio continues to improve. Despite this, corresponding software has not developed to the same levels of capability, price, and flexibility as those of sequential software. There are deep technical reasons for this, which research has been addressing for two decades. Sequential programming has long benefited from high-level programming techniques and tools that have made today's immense range of software economically viable. Research into high-level parallel programming is producing methods and tools that improve the price/performance ratio of parallel software, and broaden the range of target applications. This special issue of Parallel Processing Letters presents recent work of researchers in this field. One promising approach to developing portable parallel programs lies in the use of algorithmic skeletons templates for common patterns of parallel computation. Jocelyn SEROT demonstrates the use of dataflow techniques for efficient implementation of nested dynamic skeletons. Andrea ZAVANELLA applies the BSP cost model to data-parallel skeletons, allowing architecture-independent optimizations. The BSP computation model is also the subject of contributions by Alexandre TISKIN, who shows how threads may be used to allow divide-and-conquer algorithms to be expressed in BSP. Frederic LOULERGUE, who presents a functional calculus of BSP programs ; and Yifeng CHEN & J. W. SANDERS who define a refinement calculus for deriving BSP programs from specifications. Paul H. J. KELLY, Olav BECKMANN, Tony FIELD & Scott B. BADEN present the THEMIS model for dynamic optimizations of run-time parallel program behavior. Hans-Wolfgang LOIDL, Philip W. TRINDER & Carsten BUTZ introduce a way of improving task granularity and data locality in parallel Haskell programs based on systematic transformations of a data class. Zineb H ABB AS, Michael KRAJECKI & Daniel SINGER show a simple decomposition strategy for distributing a constraint satisfaction problem among machines. These papers were first presented at the International Workshop on High-Level Parallel Programming and Applications in Orleans in March 2001. This special

A calculus of functional BSP programs

An extension of the λ-calculus called BS λ is introduced as a formal basis for functional languages expressing bulk synchronous parallel algorithms. A confluence result is shown. The application of the calculus is illustrated by examples of program proofs and the associated notion of parallel reduction. The reduction process is interpreted in the BSP cost model.

Predictable communication on unpredictable networks: implementing BSP over TCP/IP and UDP/IP

The BSP cost model measures the cost of communication using a single architectural parameter, g, which measures permeability of the network to continuous traffic. Architectures such as networks of workstations pose particular problems for high-performance communication because it is hard to achieve high communication throughput, and even harder to do so predictably. Yet both of these are required for BSP to be effective. We present a technique for controlling applied communication load that achieves both. Traffic is presented to the communication network at a rate chosen to maximise throughput and minimise its variance. Significant performance improvements can be achieved compared to unstructured communication over the same transport protocols as in the case of, for example, MPI. Copyright © 1999 John Wiley & Sons, Ltd.

Using the BSP cost model to optimise parallel neural network training

We derive cost formulae for three different parallelisation techniques for training both supervised and unsupervised networks. These formulae are parameterised by properties of the target computer architecture. It is therefore possible to decide both which technique is best for a given parallel computer, and which parallel computer best suits a given technique. One technique, exemplar parallelism, is far superior to almost all parallel computer architectures. Formulae also take into account optimal batch learning as the overall training approach. Cost predictions are made for several of today's popular parallel computers.