Construction Of Parallel Programs Research Articles

OPTIMIZING THE OPERATION OF FRAGMENTED PROGRAMS BASED ON TRACES

This article analyzes a fragmented algorithm to solve systems of linear algebraic equations and optimize its operation. A sequential and parallel algorithm of the generalized minimal residuals method is described. A fragmented algorithm has been developed and implemented in the LuNA language. The LuNA system automatically constructs parallel programs, thereby allowing researchers not to waste efforts on developing a parallel program, in particular, allows not to take into consideration the communication processes between computer nodes and information about which nodes store distributed data, perform calculations. To optimize the operation of the fragmented program, the trace playback module was used, using data from the previous operation of the program to bypass additional costs in the further construction of parallel programs. The program was tested on the cluster of the Information and Computing Center of Novosibirsk State University and the results were analyzed. With the help of trace playback, at best, an acceleration of 27 times was obtained.

Parallel programs execution optimization using behavior control in LuNA system

In the paper, the problem of efficient parallel execution of numerical algorithms for supercomputers in the LuNA system is concerned. With LuNA, an application algorithm is represented in a hardware-independent high-level form. This allows implementing the algorithm by automatic construction of various parallel programs, which possess different non-functional properties, such as execution time, memory consumption, network workload. In the LuNA system, the efficiency problem of automatically constructed parallel programs is dealt with through the behavior concept. The presented approach allows controlling parallel program behavior without low-level programming of the desired behavior.

GVF snake algorithm-a parallel approach

Multicore architecture is an emerging computer technology where multiple processing element will be acting as independent processing cores by sharing a common memory. Digital image segmentation is a widely used medical imaging application to extract regions of interest. GVF Active Contour is a region based segmentation technique which extracts curved and irregular shaped regions by diffusing gradient vectors and by the influence of internal and external forces. This requires prior knowledge on the geometric position and anatomical structures to locate the specific region defined within an image domain. This process requires complex mathematical calculations which in turn results in the immense consumption of CPU processing time. This may adversely affect the overall performance efficiency of the process. With the advancements in multicore technology, this processing time delay can be reduced by adapting parallelization in the computation of GVF field to the specific region of interest which is to be segmented. OpenMP is a shared memory parallel programming construct, which could implement multicore parallelism with extensive and powerful APIs thereby supporting the functionalities required to attain parallelism. This article provides a high level overview of OpenMP, its effectiveness and ease of implementation in adapting parallelism to existing traditional sequential methods using instruction, data and loop level parallelism. Performance comparison could be done with sequential versions of the program written in Matlab, Java and C languages with the proposed parallelized version of OpenMP. The result is also comparable with different operating systems like Windows and Linux.

Scalable distributed data allocation in LuNA fragmented programming system

The paper presents a scalable distributed algorithm for static and dynamic data allocation in LuNA fragmented programming system. LuNA is intended for automation of construction of parallel programs, which implement large-scale numerical models for multicomputers with large number of computing nodes. The proposed algorithm takes into account data structure of the numerical model implemented, provides static and dynamic load balancing and can be used with various network topologies.

Implementation of a three dimensional three-phase fluid flow (“oil–water–gas”) numerical model in LuNA fragmented programming system

To overcome the difficulties of efficient scalable numerical algorithms implementation on multicomputers with a large number of computing nodes, LuNA system is being developed. LuNA automatically generates a code related to communications, resources management and dynamic workload balancing, thus simplifying construction of parallel programs. To examine the efficiency of numerical model implementation, created with LuNA, a real application of a three-phase (oil, water and gas) fluid filtration simulation is studied. The application algorithm is parallelized and implemented using LuNA and conventional approach using MPI (Message Passing Interface). A comparative performance testing of the two implementations is done. The results show that LuNA implementation is easier to develop, but its efficiency is significantly lower than the efficiency of MPI program, but manual tuning of the program execution makes its efficiency comparable to that of MPI program.

SU‐E‐I‐83: Parallel Programming Upgrades for the Control Acquisition, Processing and Image Display System (CAPIDS) of the Micro Angiographic Fluoroscope (MAF)

Purpose:CAPIDS is a unique software platform designed to control and acquire images from the high‐resolution MAF detector, process them and display them in a clinical environment. The images are then stored for optional playback at a later stage. CAPIDS also acquires and records the exposure parameters from the x‐ray unit. We present new parallel programming modifications using the host computer system's Graphics Processing Unit (GPU) and Central Processing unit (CPU) to improve the system performance for the various MAF imaging tasks.Methods:Multicore CPU's allow for concurrent tasks to be executed at the same time in parallel. During runtime, CAPIDS has three concurrent tasks: image acquisition and processing, image saving, and exposure parameter acquisition. Parallel programming constructs from LabVIEW allow each tasks to be executed on a separate core.GPU's allow for the same task to be performed on independent data sets in parallel. During runtime, all the image processing including flat‐field correction, digital image subtraction, image averaging, and temporal recursive filtering are performed on the GPU.Results:The new version of CAPIDS with all the parallel programming updates was successfully used for the first time to control the MAF, acquire the images, process the images and display the images during an actual clinical intervention. The images were acquired under fluoroscopy, digital subtraction angiography, and roadmap modes.Conclusion:Distributing concurrent tasks to different cores of a multicore CPU results in an efficient utilization of resources, efficient power management and increases operation speed. Use of GPU's for image processing further enhances the speed of operation.Supported by NIH Grant: 2R01EB002873 and an equipment grant from Toshiba Medical Systems Corporation

A parallel log-barrier method for mesh quality improvement and untangling

The development of parallel algorithms for mesh generation, untangling, and quality improvement is of high importance due to the need for large meshes with millions to billions of elements and the availability of supercomputers with hundreds to thousands of cores. There have been prior efforts in the development of parallel algorithms for mesh generation and local mesh quality improvement in which only one vertex is moved at a time. But for global mesh untangling and for global mesh quality improvement, where all vertices are simultaneously moved, parallel algorithms have not yet been developed. In our earlier work, we developed a serial global mesh optimization algorithm and used it to perform mesh untangling and mesh quality improvement. Our algorithm moved the vertices simultaneously to optimize a log-barrier objective function that was designed to untangle meshes as well as to improve the quality of the worst quality mesh elements. In this paper, we extend our work and develop a parallel log-barrier mesh untangling and mesh quality improvement algorithm for distributed-memory machines. We have used the algorithm with an edge coloring-based algorithm for synchronizing unstructured communication among the processes executing the log-barrier mesh optimization algorithm. The main contribution of this paper is a generic scheme for global mesh optimization, whereby the gradient of the objective function with respect to the position of some of the vertices is communicated among all processes in every iteration. The algorithm was implemented using the OpenMPI 2.0 parallel programming constructs and shows greater strong scaling efficiency compared to an existing parallel mesh quality improvement technique.

As-if-serial exception handling semantics for Java futures

Exception handling enables programmers to specify the behavior of a program when an exceptional event occurs at runtime. Exception handling, thus, facilitates software fault tolerance and the production of reliable and robust software systems. With the recent emergence of multi-processor systems and parallel programming constructs, techniques are needed that provide exception handling support in these environments that are intuitive and easy to use. Unfortunately, extant semantics of exception handling for concurrent settings is significantly more complex to reason about than their serial counterparts. In this paper, we investigate a similarly intuitive semantics for exception handling for the future parallel programming construct in Java. Futures are used by programmers to identify potentially asynchronous computations and to introduce parallelism into sequential programs. The intent of futures is to provide some performance benefits through the use of method-level concurrency while maintaining as-if-serial semantics that novice programmers can easily understand — the semantics of a program with futures is the same as that for an equivalent serial version of the program. We extend this model to provide as-if-serial exception handling semantics. Using this model our runtime delivers exceptions to the same point it would deliver them if the program was executed sequentially. We present the design and implementation of our approach and evaluate its efficiency using an open source Java virtual machine.

On Data Distributions in the Construction of Parallel Programs

Data distributions have a serious impact on time complexity of parallel programs, developed based on domain decomposition. A new kind of distributions—set distributions, based on set-valued mappings, is introduced. These distributions assign a data object to more than one process. The set distributions can be used especially when the number of processes is greater than the data input size, but, sometimes using set distributions can lead to efficient general parallel algorithms. The work-load properties of these distributions and their impact on the number of communications are discussed. In order to illustrate the implications of data distributions in the construction of parallel programs, some examples are presented. Two parallel algorithms for computation of Lagrange interpolation polynomial are developed, starting from simple distributions and set distributions.

Concurrent Programming in C Made Easy

Technology limitations forces us to investigate other methods of achieving high performance computer systems. Parallel computing is one such approach. Parallel programming is a relatively new and recent addition to traditional programming languages like C. While constructs have been suggested in literature to support Parallel programming, their implementation details have been left vague and undocumented. Also, in order to use the traditional programming constructs to implement the suggested Parallel programming constructs required costly time and effort on part of the programmer, especially in debugging since the effect of programs based on parallel programming are difficult to reproduce. This paper investigates selection of method of expression of concurrent operation, and design and implementation of a preprocessor so that such an effort is minimized, while maintaining the syntax and keywords used by pioneering investigators. These constructs are especially useful for teachers and students who wish to teach/study Operating System concepts.

A transformation approach to derive efficient parallel implementations

The construction of efficient parallel programs usually requires expert knowledge in the application area and a deep insight into the architecture of a specific parallel machine. Often, the resulting performance is not portable, i.e., a program that is efficient on one machine is not necessarily efficient on another machine with a different architecture. Transformation systems provide a more flexible solution. They start with a specification of the application problem and allow the generation of efficient programs for different parallel machines. The programmer has to give an exact specification of the algorithm expressing the inherent degree of parallelism and is released from the low-level details of the architecture. We propose such a transformation system with an emphasis on the exploitation of the data parallelism combined with a hierarchically organized structure of task parallelism. Starting with a specification of the maximum degree of task and data parallelism, the transformations generate a specification of a parallel program for a specific parallel machine. The transformations are based on a cost model and are applied in a predefined order, fixing the most important design decisions like the scheduling of independent multitask activations, data distributions, pipelining of tasks, and assignment of processors to task activations. We demonstrate the usefulness of the approach with examples from scientific computing.

Limits of task-based parallelism in irregular applications

Tomorrow's microprocessors will be able to handle multiple flows of control. Applications that exhibit task level parallelism (TLP) and can be decomposed into parallel tasks will perform well on these platforms. TLP arises when a task is independent of its neighboring code. Traditional parallel compilers exploit one variety of TLP, loop level parallelism (LLP), where loop iterations are executed in parallel. LLP can overwhelming be found in numeric, typically FORTRAN programs with regular patterns of data accesses. In contrast, irregular applications, typified by general purpose integer applications, exhibit little LLP as they tend to access data in irregular patterns through pointers. Without pointer disambiguation to analyze data access dependences, traditional parallel compilers cannot parallelize these irregular applications and ensure correct execution.We focus on a different variety of TLP, namely Speculative Task Parallelism (STP). STP arises when a task (either a leaf-procedure, a non-leaf procedure or an entire loop) is control- and memory-independent of its preceding code, and thus could be executed in parallel. Two sections of code are memory-independent when neither contains a store to a memory location that the other accesses. To exploit STP, we assume a hypothetical speculative machine that supports speculative futures (a parallel programming construct that executes a task early on a different thread or processor) with mechanisms for resolving incorrect speculation when the task is not, after all, independent. This allows us to speculatively parallelize code when there is a high probability of independence, but no guarantee.Figure 1 illustrates STP, showing a task Y in the dynamic instruction stream of an irregular application that has no memory access conflicts with a group of instructions, X, that precede Y. The shorter of X and Y determines the overlap of memory-independent instructions as seen in Figures 1(b) and 1(c). In the absence of any register dependences, X and Y may be executed in parallel, resulting in shorter execution time. It is hard for traditional parallel compilers of pointer-based languages to expose this parallelism.The goals of this paper are to identify such regions as X and Y within irregular applications and to find the number of instructions that may thus be removed from the critical path. This number represents the maximum STP when the cost of exploiting STP is zero.Because the biggest barrier to detecting independence in irregular codes is memory disambiguation, we identify memory-independent tasks using a profile-based approach and measure the amount of STP by estimating the amount of memory-independent instructions those tasks expose. We vary the level of control dependence and memory dependence to investigate their effect on the amount of memory-independence we find. We profile at different memory granularities and introduce synchronization to expose higher levels of memory-independence. Across this variety of speculation assumptions, 7 to 22% of dynamic instructions are within tasks that are found to be memory-independent. This was on the SPECint95 benchmarks, a set of irregular applications for which traditional methods of parallelization are ineffective.

An optical bus-based distributed dynamic barrier mechanism

Barrier synchronization is a useful parallel programming construct for ensuring that all processors are at a particular location in the code before any processor is allowed to continue. Barrier synchronization is integral to programming models such as the Bulk Synchronous Parallel model. Specialized hardware is often used to improve the performance of a barrier synchronization operation. With continued improvement in processor performance, more efficient synchronization mechanisms are required to counter the rising relative cost of synchronization operations. A high-speed, distributed barrier synchronization mechanism has been developed for broadcast-based optical interconnection networks. This mechanism avoids multiple conversions between optical and electrical signals by having each processor locally decide whether the barrier in which it is participating has been satisfied. It also allows arbitrary sized partitions to be built dynamically during the execution of a program. Simulations of the current hardware design estimate that the barrier synchronization requires less than 300 ns for a 128-processor system.

Behaviour specification of parallel active objects

The development of parallel programs is primarily concerned with application speed. This has led to the development of parallel applications in which software engineering aspects play only subordinate roles. In order to increase software quality in parallel applications, we motivate the construction of parallel programs by composing active objects which interact by means of an object-oriented coordination model. This paper presents a formalism for specifying the behaviour of parallel active objects and a corresponding notion of behavioural types which can be used for verifying whether certain active objects conform to a specified behaviour. Our approach is based on high-level Petri nets which enable (besides other benefits) automated analysis, in particular for automated type checking of active objects. We illustrate the usefulness of our approach by presenting reusable active objects for a manager/worker architecture. Their correct interaction is shown by automated checking of behavioural types.

Refining multiset transformers

Gamma is a minimal language based on local multiset rewriting with an elegant chemical reaction metaphor. The virtues of this paradigm in terms of systematic program construction and design of parallel programs have been argued in previous papers. Gamma can also be seen as a notation for coordinating independent programs in a larger application. In this paper, we study a notion of refinement for programs involving parallel and sequential composition operators, and derive a number of programming laws. The calculus thus obtained is applied in the development of a generic “pipelining” transformation, which enables certain sequential compositions to be refined into parallel compositions.

Evaluating the Effect of Coherence Protocols on the Performance of Parallel Programming Constructs

The different implementations of parallel programming constructs interact heavily with a multiprocessor's coherence protocol and thus may have a significant impact on performance. The form and extent of this interaction have not been established so far however, particularly in the case of update-based coherence protocols. In this paper we study the running time and communication behavior of ticket and MCS spin locks; centralized, dissemination, and tree-based barriers; parallel and sequential reductions; linear broadcasting and producer and consumer-driven logarithmic broadcasting; and centralized and distributed task queues, under pure and competitive update coherence protocols on a scalable multiprocessor; results for a write invalidate protocol are presented mostly for comparison purposes. Our experiments indicate that parallel programming techniques that are well-established for write invalidate protocols, such as MCS locks and task queues, are often inappropriate for update-based protocols. In contrast, techniques such as dissemination and tree barriers achieve superior performance under update-based protocols. Our results also show that update-based protocols sometimes lead to different design decisions than write invalidate protocols. Our main conclusion is that indeed the interaction of the parallel programming constructs with the multiprocessor's coherence protocol has a significant impact on performance. The implementation of these constructs must be carefully matched to the coherence protocol if ideal performance is to be achieved.

The interaction of parallel programming constructs and coherence protocols

Some of the most common parallel programming idioms include locks, barriers, and reduction operations. The interaction of these programming idioms with the multiprocessor's coherence protocol has a significant impact on performance. In addition, the advent of machines that support multiple coherence protocols prompts the question of how to best implement such parallel constructs, i.e. what combination of implementation and coherence protocol yields the best performance. In this paper we study the running time and communication behavior of (1) centralized (ticket) and MCS spin locks, (2) centralized, dissemination, and tree-based barriers, and (3) parallel and sequential reductions, under pure and competitive update coherence protocols; results for write-invalidate protocol are presented mostly for comparison purposes. Our experiments indicate that parallel programming techniques that are well-established for write invalidate protocols, such as MCS locks and parallel reductions, are often inappropriate for update-based protocols. In contrast, techniques such as dissemination and tree barriers achieve superior performance under update-based protocols. Our results also show that the implementation of parallel programming idioms must take the coherence protocol into account, since update-based protocols often lead to different design decisions than write invalidate protocols. Our main conclusion is that protocol-conscious implementation of parallel programming structures can significantly improve application performance; for multiprocessors that can support more than one coherence protocol both the protocol and implementation should betaken into account when exploiting parallel constructs.

Abstract Level Parallelization of Finite Difference Methods

A formalism is proposed for describing finite difference calculations in an abstract way. The formalism consists of index sets and stencils, for characterizing the structure of sets of data items and interactions between data items (“neighbouring relations”). The formalism provides a means for lifting programming to a more abstract level. This simplifies the tasks of performance analysis and verification of correctness, and opens the way for automaticcode generation. The notation is particularly useful in parallelization, for the systematic construction of parallel programs in a process/channel programming paradigm (e.g., message passing). This is important because message passing, unfortunately, still is the only approach that leads to acceptable performance for many more unstructured or irregular problems on parallel computers that have non‐uniform memory access times. It will be shown that the use of index sets and stencils greatly simplifies the determination of which data must be exchanged between different computing processes.

Implementation of GAMMA on a massively parallel computer

The GAMMA paradigm is recently proposed by Banâtre and Metayer to describe the systematic construction of parallel programs without introducing artificial sequentially. This paper presents two synchronous execution models for GAMMA and discusses how to implement them on MasPar MP-1, a massively data parallel computer. The results show that GAMMA paradigm can be implemented very naturally on data parallel machines, and very high level language, such as GAMMA in which parallelism is left implicit, is suitable for specifying massively parallel applications.

Linear logic automata

A Linear Logic automaton is a hybrid of a finite automaton and a non-deterministic Petri net. LL automata commands are represented by propositional Horn Linear Logic formulas. Computations performed by LL automata directly correspond to cut-free derivations in Linear Logic. A programming language of LL automata is developed in which typical sequential, non-deterministic and parallel programming constructs are expressed in the natural way. All non-deterministic computations, e.g. computations performed by programs built up of guarded commands in the Dijkstra's approach to non-deterministic programming, are directly simulated within the framework of Linear Logic automata, and thereby within the Horn framework of Linear Logic.