Algorithmic Skeletons Research Articles

Basic skeletons in llc

llc is a high-level parallel language that provides support for some of the most widely used algorithmic skeletons. The language has a syntax based on OpenMP-like directives and the compiler uses direct translation to MPI to produce parallel code. To evaluate the performance of our prototype compiler we present computational results for some of the skeletons available in the language on different architectures. Particularly in the presence of coarse-grain parallelism, the results reflect similar performances for the llc compiled version and ad hoc MPI or OpenMP implementations. In all cases, the performance loss with respect to a direct MPI implementation is clearly compensated by a significantly smaller development effort.

Autonomous mobility skeletons

To manage load on large and dynamic networks we have developed Autonomous Mobile Programs (AMPs) that periodically use a cost model to decide where to execute. A disadvantage of directly programming AMPs is that the cost model, mobility decision function, and network interrogation are all explicit in the program. This paper proposes autonomous mobility skeletons that encapsulate self-aware mobile coordination for common patterns of computation over collections. Autonomous mobility skeletons are akin to algorithmic skeletons in being polymorphic higher order functions, but where algorithmic skeletons abstract over parallel coordination, autonomous mobility skeletons abstract over autonomous mobile coordination. We present the automap, autofold and autoiter autonomous mobility skeletons, together with performance measurements of Jocaml, Java Voyager, and JavaGo implementations on small networks. autoiter is an unusual skeleton, abstracting over the Iterator interface commonly used with Java collections.

Algorithmic skeletons meeting grids

In this work, we discuss an extension of the set of principles that should guide the future design and development of skeletal programming systems, as defined by Cole in his “pragmatic manifesto” paper. The three further principles introduced are related to the ability to exploit existing sequential code as well as to the ability to target typical modern architectures, those made out of heterogeneous processing elements with dynamically varying availability, processing power and connectivity features such as grids or heterogeneous, non-dedicated clusters. We outline two skeleton based programming environments currently developed at our university and we discuss how these environments adhere to the proposed set of principles. Eventually, we outline how some other relevant, well-known skeleton environments conform to the same set of principles.

Skeletons and Asynchronous RPC for Embedded Data and Task Parallel Image Processing

Developing embedded parallel image processing applications is usually a very hardware-dependent process, often using the single instruction multiple data (SIMD) paradigm, and requiring deep knowledge of the processors used. Furthermore, the application is tailored to a specific hardware platform, and if the chosen hardware does not meet the requirements, it must be rewritten for a new platform. We have proposed the use of design space exploration [9] to find the most suitable hardware platform for a certain application. This requires a hardware-independent program, and we use algorithmic skeletons [5] to achieve this, while exploiting the data parallelism inherent to low-level image processing. However, since different operations run best on different kinds of processors, we need to exploit task parallelism as well. This paper describes how we exploit task parallelism using an asynchronous remote procedure call (RPC) system, optimized for low-memory and sparsely connected systems such as smart cameras. It uses a futures [16]-like model to present a normal imperative C-interface to the user in which the skeleton calls are implicitly parallelized and pipelined. Simulation provides the task dependency graph and performance numbers for the mapping, which can be done at run time to facilitate data dependent branching. The result is an easy to program, platform independent framework which shields the user from the parallel implementation and mapping of his application, while efficiently utilizing on-chip memory and interconnect bandwidth.

Efficient parallel LAN/WAN algorithms for optimization. The mallba project

The mallba project tackles the resolution of combinatorial optimization problems using generic algorithmic skeletons implemented in C++. A skeleton in the mallba library implements an optimization method in one of the three families of generic optimization techniques offered: exact, heuristic and hybrid. Moreover, for each of those methods, mallba provides three different implementations: sequential, parallel for Local Area Networks, and parallel for Wide Area Networks. This paper introduces the architecture of the mallba library, details some of the implemented skeletons, and offers computational results for some classical optimization problems to show the viability of our library. Among other conclusions, we claim that the design used to develop the optimization techniques included in the library is generic and efficient at the same time.

TOWARDS MOBILITY SKELETONS

In mobile languages programmers control the placement of code or computations in open networks, e.g., a program can migrate between locations. Mobile computations are typically stateful and interact with the state at each location in the network. We propose mobility skeletons, a library of parameterisable functions that encapsulate common patterns of mobile computation. Mobility skeletons are analogous to, but very different from, algorithmic skeletons — parameterisable functions encapsulating common patterns of parallel computation. We have identified three common patterns of mobile computation, and implemented them as a library of higher-order functions in mobile Haskell. Each mobility skeleton is defined and illustrated with an example. We show how mobility skeletons can be composed and nested, and illustrate their use in a non-trivial case study: a distributed meeting planner. Mobility skeletons are extensible: there is a small set of mobility primitives, and medium-level abstractions such as remote evaluation can be defined using them. New mobility skeletons can be defined using the medium and low level abstractions.

A parallel SML compiler based on algorithmic skeletons

Algorithmic skeletons are abstractions from common patterns of parallel activity which offer a high degree of reusability for developers of parallel algorithms. Their close association with higher order functions (HOFs) makes functional languages, with their strong transformational properties, excellent vehicles for skeleton-based parallel program development. However, using HOFs in this way raises substantial problems of identification of useful HOFs within a given application and of resource allocation on target architectures. We present the design and implementation of a parallelising compiler for Standard ML which exploits parallelism in the familiar $map$ and $fold$ HOFs through skeletons for processor farms and processor trees, respectively. The compiler extracts parallelism automatically and is target architecture independant. HOF execution within a functional language can be nested in the sense that one HOF may be passed and evaluated during the execution of another HOF. We are able to exploit this by nesting our parallel skeletons in a processor topology which matches the structure of the Standard ML source. However, where HOF arguments result from partially applied functions, free variable bindings must be identified and communicated through the corresponding skeleton hierarchy to where those arguments are actually applied. We describe the analysis leading from input Standard ML through HOF instantiation and backend compilation to an executable parallel program. We also present an overview of the runtime system and the execution model. Finally, we give parallel performance figures for several example programs, of varying computational loads, on the Linux-based Beowulf, IBM SP/2, Fujitsu AP3000 and Sun StarCat 15000 MIMD parallel machines. These demonstrate good cross-platform consistency of parallel code behaviour.

Features from functional programming for a C++ skeleton library

Message passing based on libraries such as MPI is typically used to program parallel machines with distributed memory. This is efficient, but error prone. Algorithmic skeletons are intended to simp...

Managing Algorithmic Skeleton Nesting Requirements in Realistic Image Processing Applications: The Case of the SKiPPER-II Parallel Programming Environment's Operating Model

SKiPPER is a Skeleton-based Parallel Programming EnviRonment being developed since 1996 and running at LASMEA Laboratory, the Blaise-Pascal University, France. The main goal of the project was to demonstrate the applicability of skeleton-based parallel programming techniques to the fast prototyping of reactive vision applications. This paper deals with the special features embedded in the latest version of the project: algorithmic skeleton nesting capabilities and a fully dynamic operating model. Throughout the case study of a complete and realistic image processing application, in which we have pointed out the requirement for skeleton nesting, we are presenting the operating model of this feature. The work described here is one of the few reported experiments showing the application of skeleton nesting facilities for the parallelisation of a realistic application, especially in the area of image processing. The image processing application we have chosen is a 3D face-tracking algorithm from appearance.

Discovering applications of higher order functions through proof planning

Abstract. The close association between higher order functions (HOFs) and algorithmic skeletons is a promising source of automatic parallelisation of programs. A theorem proving approach to discovering HOFs in functional programs is presented. Our starting point is proof planning, an automated theorem proving technique in which high-level proof plans are used to guide proof search. We use proof planning to identify provably correct transformation rules that introduce HOFs. The approach has been implemented in the λ Clam proof planner and tested on a range of examples. The work was conducted within the context of a parallelising compiler for Standard ML.

Scheduling Skeleton-Based Grid Applications Using PEPA and NWS

Any scheduling scheme for grid applications must make implicit or explicit assumptions about both the future behaviour of the application and the future availability and performance of grid resources. This paper describes an approach in which the future application behaviour is constrained by the use of algorithmic skeletons, facilitating modelling with a performance oriented process algebra, and future grid resource performance is predicted by the Network Weather Service (NWS) tool. The concept is illustrated through a case study involving Pipeline and Deal skeletons. A tool is presented which automatically generates and solves a set of models which are parameterised with information obtained from NWS. Some numerical results and timing information on the use of the tool are provided, illustrating the efficacy of this approach.

Features from functional programming for a C++ skeleton library

AbstractMessage passing based on libraries such as MPI is typically used to program parallel machines with distributed memory. This is efficient, but error prone. Algorithmic skeletons are intended to simplify parallel programming by increasing expressive power. The idea is to offer typical parallel programming patterns as polymorphic higher‐order functions which are efficiently implemented in parallel. The present paper describes how C++ templates and operator overloading can be used in order to provide the main features needed for algorithmic skeletons, namely higher‐order functions, partial applications and parametric polymorphism. Experimental results based on a draft implementation of our C++ skeleton library show that higher expressive power can be gained without a significant performance penalty. Copyright © 2005 John Wiley & Sons, Ltd.

Evaluating the performance of pipeline-structured parallel programs with skeletons and process algebra

We show in this paper how to evaluate the performance of pipeline-structured parallel programs with skeletons and process algebra. Since many applications follow some commonly used algorithmic skeletons, we identify such skeletons and model them with process algebra in order to get relevant information about the performance of the application, and to be able to take good scheduling decisions. This concept is illustrated through the case study of the pipeline skeleton, and a tool which generates automatically a set of models and solves them is presented. Some numerical results are provided, proving the efficacy of this approach.

PREFACE: Special Issue on High-Level Parallel Programming and Applications

Many applications of computing require performance levels attainable only on parallel architectures. Such systems are now readily available as their price/performance ratio continues to improve. Despite this, corresponding software has not developed to the same levels of capability, price, and flexibility as those of sequential software. There are deep technical reasons for this, which research has been addressing for two decades. Sequential programming has long benefited from high-level programming techniques and tools that have made today's immense range of software economically viable. Research into high-level parallel programming is producing methods and tools that improve the price/performance ratio of parallel software, and broaden the range of target applications. This special issue of Parallel Processing Letters presents recent work of researchers in this field. One promising approach to developing portable parallel programs lies in the use of algorithmic skeletons templates for common patterns of parallel computation. Jocelyn SEROT demonstrates the use of dataflow techniques for efficient implementation of nested dynamic skeletons. Andrea ZAVANELLA applies the BSP cost model to data-parallel skeletons, allowing architecture-independent optimizations. The BSP computation model is also the subject of contributions by Alexandre TISKIN, who shows how threads may be used to allow divide-and-conquer algorithms to be expressed in BSP. Frederic LOULERGUE, who presents a functional calculus of BSP programs ; and Yifeng CHEN & J. W. SANDERS who define a refinement calculus for deriving BSP programs from specifications. Paul H. J. KELLY, Olav BECKMANN, Tony FIELD & Scott B. BADEN present the THEMIS model for dynamic optimizations of run-time parallel program behavior. Hans-Wolfgang LOIDL, Philip W. TRINDER & Carsten BUTZ introduce a way of improving task granularity and data locality in parallel Haskell programs based on systematic transformations of a data class. Zineb H ABB AS, Michael KRAJECKI & Daniel SINGER show a simple decomposition strategy for distributing a constraint satisfaction problem among machines. These papers were first presented at the International Workshop on High-Level Parallel Programming and Applications in Orleans in March 2001. This special

A data and task parallel image processing environment

The paper presents a data and task parallel low-level image processing environment for distributed memory systems. Image processing operators are parallelized by data decomposition using algorithmic skeletons. Image processing applications are parallelized by task decomposition, based on the image application task graph. In this way, an image processing application can be parallelized both by data and task decomposition, and thus better speed-ups can be obtained. We validate our method on the multi-baseline stereo vision application.

Fast prototyping of parallel-vision applications using functional skeletons

We present a design methodology for real-time vision applications aiming at significantly reducing the design-implement-validate cycle time on dedicated parallel platforms. This methodology is based upon the concept of algorithmic skeletons, i.e., higher order program constructs encapsulating recurring forms of parallel computations and hiding their low-level implementation details. Parallel programs are built by simply selecting and composing instances of skeletons chosen in a predefined basis. A complete parallel programming environment was built to support the presented methodology. It comprises a library of vision-specific skeletons and a chain of tools capable of turning an architecture-independent skeletal specification of an application into an optimized, deadlock-free distributive executive for a wide range of parallel platforms. This skeleton basis was defined after a careful analysis of a large corpus of existing parallel vision applications. The source program is a purely functional specification of the algorithm in which the structure of a parallel application is expressed only as combination of a limited number of skeletons. This specification is compiled down to a parametric process graph, which is subsequently mapped onto the actual physical topology using a third-party CAD software. It can also be executed on any sequential platform to check the correctness of the parallel algorithm. The applicability of the proposed methodology and associated tools has been demonstrated by parallelizing several realistic real-time vision applications both on a multi-processor platform and a network of workstations. It is here illustrated with a complete road-tracking algorithm based upon white-line detection. This experiment showed a dramatic reduction in development times (hence the term fast prototyping), while keeping performances on par with those obtained with the handcrafted parallel version.

Parallel Standard ML with Skeletons

We present an overview of our system for automatically extracting parallelism from Standard ML programs using algorithmic skeletons . This system identifies a small number of higher-order functions as sites of parallelism and the compiler uses profiling and transformation techniques to exploit these.

Empirical Parallel Performance Prediction From Semantics-Based Profiling

The PMLS parallelizing compiler for Standard ML is based upon the automatic instantiation of algorithmic skeletons at sites of higher order function (HOF) use. Rather than mechanically replacing HOFs with skeletons, which in general leads to poor parallel performance, PMLS seeks to predict run-time parallel behaviour to optimise skeleton use. Static extraction of analytic cost models from programs is undecidable, and practical heuristic approaches are intractable. In contrast, PMLS utilises a hybrid approach by combining static analytic cost models for skeletons with dynamic information gathered from the sequential instrumentation of HOF argument functions. Such instrumentation is provided by an implementation independent SML interpreter, based on the language's Structural Operational Semantics (SOS), in the form of SOS rule counts. PMLS then tries to relate the rule counts to program execution times through numerical techniques. This paper considers the design and implementation of the PMLS approach to parallel performance prediction. The formulation of a general rule count cost model as a set of over-determined linear equations is discussed, and their solution by single value decomposition, and by a genetic algorithm, are presented.

NESTED ALGORITHMIC SKELETONS FROM HIGHER ORDER FUNCTIONS

Algorithmic skeletons provide a promising basis for the automatic utilisation of parallelism at sites of higher order function use through static program analysis. However, decisions about whether or not to realise particular higher order function instances as skeletons must be based on information about processing resources available at runtime In principle, nested higher order functions may be realised as nested skeletons. However, where higher order function arguments result from partially applied functions, free-variable bindings must be identified and communicated through the corresponding skeleton hierarchy to where those arguments are actually applied Here, a skeleton based parallelising compiler for Standard ML is presented. Hybrid skeletons, which can change from parallel to serial evaluation at runtime, are considered and mechanisms for their nesting are discussed. The main compilation stages are illustrated for simple examples. A nested higher order function based algorithm for multiplying matrices of arbitrary length integers is presented along with performance figures for compiled code running on a Fujitsu AP3000.

Generating parallel programs from skeleton based specifications

The Static Iterative Transformation Specification System (SITSS) is a portable parallel programming system. This paper describes the techniques used in SITSS to generate parallel programs from user specifications of algorithms. The paper first outlines SITSS, the algorithmic skeleton on which SITSS is based, and the user interface. The techniques for analysing the specification selecting a data partitioning strategy, and generating parallel code are described. A comparison of SITSS produced code against hand-written code is then presented, followed by a discussion of areas for future work to enhance the system.