High-level Programming Model Research Articles

Parallel asynchronous computations for image analysis

Many works have been done for parallelizing low-level image analysis computations. However the task is harder for higher levels, as the data manipulations are complex, and there is a wide range of algorithms to encompass. To allow concurrently speed and programmability, a high-level programming model that can be efficiently implemented on parallel architectures is required. To achieve this goal, we propose the associative nets model, a parallel computing model for image analysis based on simple data-parallelism paradigms, providing special features, such as graph-based data structures to handle irregular data, virtual data-structures to ease hierarchical image descriptions, and specific primitives (dirassoc) to compute on the interpixels relation graph. For implementation purposes, the dirassoc computing primitive performs asynchronous local computations until it reaches stability. Asynchronism has many advantages for hardware (speed, power consumptions, and chip size) as well as in software (less synchronization barriers). However to insure completion of the asynchronous operation, the dirassoc must use a set of specific operators (r-operators) introduced by Ducourthial. In this paper we emphasize on the interest of the r-operators and of the asynchronous computations for image analysis algorithms. We give applications in distance transforms, contour closing, Voronoi segmentation, watershed segmentation, and mathematical morphology. Hence, we show that asynchronous computations are powerful tools for image analysis on interpixel graphs.

Airport simulation using CORBA and DIS

This paper presents the SEEDS simulation environment for the evaluation of distributed traffic control systems. Starting with an overview of the general simulator architecture, performance measurements of the simulation environment carried out with a prototype for airport ground-traffic simulation are described. The main aspects of the performance analysis are the attained application performance using CORBA and DIS as communication middleware, and the scalability of the overall approach. The evaluation shows that CORBA and DIS are well suited for distributed interactive simulation purposes because of their adequate performance, high scalability, and the high-level programming model which allows to rapidly develop and maintain complex distributed applications.

Architectural Support and Mechanisms for Object Caching in Dynamic Multithreaded Computations

High-level parallel programming models supporting dynamic fine-grained threads in a global object space are becoming increasingly popular for expressing irregular applications based on sophisticated adaptive algorithms and pointer-based data structures. However, implementing these multithreaded computations on scalable parallel machines poses significant challenges, particularly with respect to object caching. Object caching techniques must be able to tolerate unresponsive processors and protocol handler occupancy delays. This paper examines whether these challenges can be offset by leveraging responsive general-purpose communication architectural features (such as remote memory access and atomic operations), possibly compensating for the lack of more sophisticated hardware primitives by relying upon increased involvement of the run-time system and the compiler. A detailed performance analysis of four irregular applications, using the Illinois Concert System on the Cray T3D and the SGI Origin 2000, finds that existing software distributed shared memory (DSM) systems are capable of delivering good performance only in the presence of a high level of responsive communication architecture support (specifically, support for remote atomic operations). Recognizing that this situation stems from the synchronous request–reply nature of DSM protocols, we present a composable object caching framework, called view caching, which exploits knowledge of application data access semantics to construct custom protocols that require reduced processor synchronization. View caching protocols are more tolerant to responsiveness and occupancy delays and are able to exploit even lower level responsive communication primitives (such as nonatomic remote memory accesses) for a performance benefit.

A high level FPGA-based abstract machine for image processing

Image processing requires high computational power, plus the ability to experiment with algorithms. Recently, reconfigurable hardware devices in the form of field programmable gate arrays (FPGAs) have been proposed as a way of obtaining high performance at an economical price. At present, however, users must program FPGAs at a very low level and have a detailed knowledge of the architecture of the device being used. They do not therefore facilitate easy development of, or experimentation with, image processing algorithms. To try to reconcile the dual requirements of high performance and ease of development, this paper reports on the design and realisation of an FPGA based image processing machine and its associated high level programming model. This abstract programming model allows an application developer to concentrate on the image processing algorithm in hand rather than on its hardware implementation. The abstract machine is based on a PC host system with a PCI-bus add-on card containing Xilinx XC6200 series FPGA(s). The machine's high level instruction set is based on the operators of image algebra. XC6200 series FPGA configurations have been developed to implement each high level instruction.

Evaluating high level parallel programming support for irregular applications in ICC++

Object-oriented techniques have been proffered as aids for managing complexity, enhancing reuse, and improving readability of irregular parallel applications. However, as performance is the major reason for employing parallelism, programmability and high performance must be delivered together. Using a suite of seven challenging irregular applications and the mature Illinois Concert system (a high-level concurrent object-oriented programming model) and an aggressive implementation (whole program compilation plus microsecond threading and communication primitives in the runtime), we evaluate what programming efforts are required to achieve high performance. For all seven applications, we achieve performance comparable to the best achievable via low-level programming means on large-scale parallel systems. In general, a high-level concurrent object-oriented programming model supported by aggressive implementation techniques can eliminate programmer management of many concerns – procedure and computation granularity, namespace management, and low-level concurrency management. Our study indicates that these concerns are fully automated for these applications. Decoupling these concerns makes managing the remaining fundamental concerns – data locality and load balance – much easier. In several cases, data locality and load balance for the complex algorithm and pointer data structures is automatically managed by the compiler and runtime, but in general programmer intervention was required. In a few cases, more detailed control is required, specifically explicit task priority, data consistency, and task placement. Our system integrates the expression of such information cleanly into the programming interface. Finally, only small changes to the sequential code were required to express concurrency and performance optimizations, less than 5 per cent of the source code lines were changed in all cases. This bodes well for supporting both sequential and parallel performance in a single code base. © 1998 John Wiley & Sons, Ltd.

High-level programming of massively parallel computers based on shared virtual memory

Highly parallel machines needed to solve compute-intensive scientific applications are based on the distribution of physical memory across the compute nodes. The drawback of such systems is the necessity to write applications in the message passing programming model. Therefore, a lot of research is going on in higher-level programming models and supportive hardware, operating system techniques, languages. The research direction outlined in this article is based on shared virtual memory systems, i.e., scalable parallel systems with a global address space which support an adaptive mapping of global addresses to physical memories. We introduce programming concepts and program optimizations for SVM systems in the context of the SVM-Fortran programming environment which is based on a shared virtual memory system implemented on Intel Paragon. The performance results for real applications proved that this environment enables users to obtain a similar or better performance than by programming in HPF.

Concurrency control and view notification algorithms for collaborative replicated objects

This paper describes algorithms for implementing a high-level programming model for synchronous distributed groupware applications. In this model, several application data objects may be atomically updated, and these objects automatically maintain consistency with their replicas using an optimistic algorithm. Changes to these objects may be optimistically or pessimistically observed by view objects by taking consistent snapshots. The algorithms for both update propagation and view notification are based upon optimistic guess propagation principles adapted for fast commit by using primary copy replication techniques. The main contribution of the paper is the synthesis of these two algorithmic techniques-guess propagation and primary copy replication-for implementing a framework that is easy to program to and is well suited for the needs of groupware applications.

Benchmark kernels as a tool for performance evaluation of MPP‘s

The performance achieved by a parallel architecture over a complete application is determined by the combination of the hardware and software modules. When we talk about hardware we mean node processing power and network parameters, while software entails all from the optimization capabilities of the compiler to the high level programming model. They interact in a non-simple way delivering variable results for different problem sizes and making the task of predicting performance a very difficult one. Performance is predictable once, given an algorithm, you can parameterize it in terms of floating-point operations needed, bandwidth and latency requirements, granularity of the problem itself and few parameters, obviously machine dependent. We attack the issue of predicting performance for a large class of regular synchronous problems on rectangular grids (only 2D in this paper). The aim of the paper is to determine, by means of dedicated small benchmarking kernels, all the machine dependent parameters. These will be used to predict and compare, over a very wide range of data set sizes, the performances of the Connection Machine CM-5, the Cray T3D and the IBM SP2 for a simple but complete application like the Conjugate Gradient solution for the Poisson equation. We show that the parameterization can be done quite accurately for all of the studied platforms, thus predicting, from measurements performed on extremely simple kernels and some algorithmic understanding, the behavior of an MPP over a very wide range of parameters. We argue in favor of adopting this methodology to produce meaningful benchmarks of MPP platforms. © 1998 John Wiley & Sons, Ltd.

Optimal and Near–Optimal Solutions for Hard Compilation Problems

An optimizing compiler typically uses multiple program representations at different levels of program and performance abstractions in order to be able to perform transformations that – at least in the majority of cases – will lead to an overall improvement in program performance. The complexities of the program and performance abstractions used to formulate compiler optimization problems have to match the complexities of the high–level programming model and of the underlying target system. Scalable parallel systems typically have multi–level memory hierarchies and able to exploit coarse–grain and fine–grain parallelism. Most likely, future systems will have even deeper memory hierarchies and more granularities of parallelism. As a result, future compiler optimizations will have to use more and more complex, multi–level computation and performance models in order to keep up with the complexities of their future target systems. Most of the optimization problems encountered in highly optimizing compilers are already NP–hard, and there is little hope that most newly encountered optimization formulations will not be at least NP–hard as well. To face this "complexity crisis", new methods are needed to evaluate the benefits of a compiler optimization formulation. A crucial step in this evaluation process is to compute the optimal solution of the formulation. Using ad–hoc methods to compute optimal solutions to NP–complete problems may be prohibitively expensive. Recent improvements in mixed integer and 0–1 integer programming suggest that this technology may provide the key to efficient, optimal and near–optimal solutions to NP–complete compiler optimization problems. In fact, early results indicate that integer programming formulations may be efficient enough to be included in not only evaluation prototypes, but in production programming environments or even production compilers. This paper discusses the potential benefits of integer programming as a tool to deal with NP–complete compiler optimization formulations in compilers and programming environments.

An integrated performance analysis tool for SPMD data-parallel programs

A new generation of data parallel languages have been proposed whereby a user specifies how data structures are to be distributed amongst the processor nodes of a distributed-memory machine. Based on this information, the compiler then generates code for the parallel application. Although this approach significantly simplifies the development of the initial version of a parallel application, selection of good data distributions leading to efficient computations is often quite difficult. Therefore, performance debuggers are needed to yield insights into the data distribution effects. On the other hand, most of the existing approaches to performance debugging are very general and thus do not provide the user feedback in terms of the high level programming model or the source code of the parallel application. In this paper, we describe a novel approach to performance debugging of data parallel programs. The design and implementation of a visual performance debugger is described that is specifically targeted to meet the performance debugging requirements of a data-parallel programming model based on user-specified data distributions. The performance debugger is part of an integrated programming environment, called EPPP, which also supports a data parallel compiler and a parallel architecture simulator. Thus development and performance debugging of an application may be done either on the real hardware or by using the simulator. The code for EPPP can be obtained free of cost (contact web address, http://www.crim.ca/apar).

Konrad Zuse's Plankalkül: the first high-level, "non von Neumann" programming language

Konrad Zuse was the first person in history to build a working digital computer, a fact that is still not generally acknowledged. Even less known is that in the years 1943-1945, Zuse developed a high-level programming model and, based on it, an algorithmic programming language called Plankalku/spl uml/l (Plan Calculus). The Plankalku/spl uml/l features binary data structure types, thus supporting a loop-free programming style for logical or relational problems. As a language for numerical applications, Plankalku/spl uml/l already had the essential features of a "von Neumann language", though at the level of an operator language. Consequently, the Plankalku/spl uml/l is in some aspects equivalent and in others more powerful than the von Neumann programming model that came to dominate programming for a long time. To find language concepts similar to those of the Plankalku/spl uml/l, one has to look at "non-von Neumann languages" such as APL or the relational algebra. This paper conveys the syntactic and semantic flavor of the Plankalku/spl uml/l, without presenting all its syntactic idiosyncrasies. Rather, it points out that the Plankalku/spl uml/l was not only the first high-level programming language but in some aspects conceptually ahead of the high-level languages that evolved a decade later.

Coordination models and languages as software integrators

Early concurrent languages supported interactions through shared variables. Languages based on message passing were also a very early idea that gained popularity as soon as software engineers started to design and build software for distributed computer systems. A more recent approach consists of using higher-level programming models and languages for distributed programming, called coordination models and languages. “A coordination model is the glue that binds separate activities into an ensemble” [Carriero and Gelernter 1992]. In other words, a coordination model provides a framework in which the interaction of active and independent entities called agents can be expressed. A coordination model should cover the issues of creation and destruction of agents, communication among agents, and spatial distribution of agents, as well as synchronization and distribution of their actions over time. A constructive approach to define coordination models consists of identifying the components out of which they are built:

Polymorphic processor arrays

Polymorphic processor arrays (PPAs), two-dimensional mesh-connected arrays of processors in which each processor is equipped with a switch able to interconnect its four NEWS ports, are discussed. The main features of PPA are that it models a realistic class of parallel computers, it supports the definition of high level programming models, it supports virtual parallelism, and it supports low complexity algorithms in a number of application fields. Both the PPA computation model and the PPA programming model are presented. It is shown that the PPA computation model is realistic by relating it to the design of the polymorphic torus (PT) chip. It is also shown that the PPA programming model is scalable by demonstrating that any algorithm having O(p) complexity on a virtual PPA of size square root m* square root m, has O(k p) complexity on a PPA of size square root n* square root n, with m k n and k integers. Some application algorithms in the area of numerical analysis and graph processing are presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>