Parallel Skeletons Research Articles

A highly optimized skeleton for unbalanced and deep divide-and-conquer algorithms on multi-core clusters

AbstractEfficiently implementing the divide-and-conquer pattern of parallelism in distributed memory systems is very relevant, given its ubiquity, and difficult, given its recursive nature and the need to exchange tasks and data among the processors. This task is noticeably further complicated in the presence of multi-core systems, where hybrid parallelism must be exploited to attain the best performance, and when unbalanced and deep workloads are considered, as additional measures must be taken to load balance and avoid deep recursion problems. In this manuscript a parallel skeleton that fulfills all these requirements while providing high levels of usability is presented. In fact, the evaluation shows that our proposal is on average 415.32% faster than MPI codes and 229.18% faster than MPI + OpenMP benchmarks, while offering an average improvement in the programmability metrics of 131.04% over MPI alternatives and 155.18% over MPI + OpenMP solutions.

Multiaxis wire and arc additive manufacturing for overhangs based on conical substrates

PurposeThis paper aims to present a series of approaches for three-related issues in multiaxis in wire and arc additive manufacturing (WAAM) as follows: how to achieve a stable and robust deposition process and maintain uniform growth of the part; how to maintain consistent formation of a melt pool on the surface of the workpiece; and how to fabricate an overhanging structure without supports.Design/methodology/approachThe principal component analysis-based path planning approach is proposed to compute the best scanning directions of slicing contours for the generation of filling paths, including zigzag paths and parallel skeleton paths. These printing paths have been experimented with in WAAM. To maintain consistent formation of a melt pool at overhanging regions, the authors introduce definitions for the overhanging point, overhanging distance and overhanging vector, with which the authors can compute and optimize the multiaxis motion. A novel fabricating strategy of depositing the overhanging segments as a support for the deposition of filling paths is presented.FindingsThe second principal component of a planar contour is a reasonable scanning direction to generate zigzag filling paths and parallel skeleton filling paths. The overhanging regions of a printing layer can be supported by pre-deposition of overhanging segments. Large overhangs can be successfully fabricated by the multiaxis WAAM process without supporting structures.Originality/valueAn intelligent approach of generating zigzag printing paths and parallel skeleton printing paths. Optimizations of depositing zigzag paths and parallel skeleton paths. Applications of overhanging point overhanging distance and overhanging vector for multiaxis motion planning. A novel fabricating strategy of depositing the overhanging segments as a support for the deposition of filling paths.

A Parallel Skeleton for Divide-and-conquer Unbalanced and Deep Problems

The Divide-and-conquer (D&C) pattern appears in a large number of problems and is highly suitable to exploit parallelism. This has led to much research on its easy and efficient application both in shared and distributed memory parallel systems. One of the most successful approaches explored in this area consists of expressing this pattern by means of parallel skeletons which automate and hide the complexity of the parallelization from the user while trying to provide good performance. In this paper, we tackle the development of a skeleton oriented to the efficient parallel resolution of D&C problems with a high degree of imbalance among the subproblems generated and/or a deep level of recurrence. The skeleton achieves in our experiments average speedups between 11 and 18% higher than those of other solutions, reaching a maximum speedup of 78% in some tests. Nevertheless, the new proposal requires an average of between 13 and 29% less programming effort than the usual alternatives.

Программная поддержка модели BSF

The paper is devoted to the software support of the parallel computation model BSF (Bulk Synchronous Farm) intended to iterative algorithms with high computational complexity, which developed for multiprocessor systems with distributed memory of exaflops performance level. Software support includes parallel BSF-skeleton and BSF-Studio web-application. The definition and classification of parallel skeletons are given. The logic and file structure of new BSF-skeleton are described. The BSF-skeleton is a collection of C++ source code files used for BSF-programs development. A detailed description of the design and implementation of the BSF-Studio web-application, which is a visual constructor of BSF-programs, is given. BSF-Studio provides compilation and execution of program code.

Simultaneous CPU–GPU Execution of Data Parallel Algorithmic Skeletons

Parallel programming has become ubiquitous; however, it is still a low-level and error-prone task, especially when accelerators such as GPUs are used. Thus, algorithmic skeletons have been proposed to provide well-defined programming patterns in order to assist programmers and shield them from low-level aspects. As the complexity of problems, and consequently the need for computing capacity, grows, we have directed our research toward simultaneous CPU–GPU execution of data parallel skeletons to achieve a performance gain. GPUs are optimized with respect to throughput and designed for massively parallel computations. Nevertheless, we analyze whether the additional utilization of the CPU for data parallel skeletons in the Muenster Skeleton Library leads to speedups or causes a reduced performance, because of the smaller computational capacity of CPUs compared to GPUs. We present a C\(++\) implementation based on a static distribution approach. In order to evaluate the implementation, four different benchmarks, including matrix multiplication, N-body simulation, Frobenius norm, and ray tracing, have been conducted. The ratio of CPU and GPU execution has been varied manually to observe the effects of different distributions. The results show that a speedup can be achieved by distributing the execution among CPUs and GPUs. However, both the results and the optimal distribution highly depend on the available hardware and the specific algorithm.

Program Transformation to Identify List-Based Parallel Skeletons

Algorithmic skeletons are used as building-blocks to ease the task of parallel programming by abstracting the details of parallel implementation from the developer. Most existing libraries provide implementations of skeletons that are defined over flat data types such as lists or arrays. However, skeleton-based parallel programming is still very challenging as it requires intricate analysis of the underlying algorithm and often uses inefficient intermediate data structures. Further, the algorithmic structure of a given program may not match those of list-based skeletons. In this paper, we present a method to automatically transform any given program to one that is defined over a list and is more likely to contain instances of list-based skeletons. This facilitates the parallel execution of a transformed program using existing implementations of list-based parallel skeletons. Further, by using an existing transformation called distillation in conjunction with our method, we produce transformed programs that contain fewer inefficient intermediate data structures.

Automatic Dynamic Data Structures Recognition to Support the Migration of Applications to the Cloud

The work presented in this manuscript describes a methodology for the recognition of Dynamic Data structures, with a focus on Queues, Pipes and Lists. The recognition of such structures is used as a basis for the mapping of sequential code to Cloud Services, in order to support the semi-automatic restructuring of source software. The goal is to develop a complete methodology and a framework based on it to ease the efforts needed to port native applications to a Cloud Platform and simplify the relative complex processes. In order to achieve such an objective, the proposed technique exploits an intermediate representation of the code, consisting in parallel Skeletons and Cloud Patterns. Logical inference rules act on a knowledge base, built during the analysis of the source code, to guide the recognition and mapping processes. Both the inference rules and knowledge base are expressed in Prolog. A prototype tool for the automatic analysis of sequential source code and its mapping to a Cloud Pattern is also presented.

PSkel: A stencil programming framework for CPU‐GPU systems

SummaryThe use of Graphics Processing Units (GPUs) for high‐performance computing has gained growing momentum in recent years. Unfortunately, GPU‐programming platforms like Compute Unified Device Architecture (CUDA) are complex, user unfriendly, and increase the complexity of developing high‐performance parallel applications. In addition, runtime systems that execute those applications often fail to fully utilize the parallelism of modern CPU‐GPU systems. Typically, parallel kernels run entirely on the most powerful device available, leaving other devices idle. These observations sparked research in two directions: (1) high‐level approaches to software development for GPUs, which strike a balance between performance and ease of programming; and (2) task partitioning to fully utilize the available devices. In this paper, we propose a framework, called PSkel, that provides a single high‐level abstraction for stencil programming on heterogeneous CPU‐GPU systems, while allowing the programmer to partition and assign data and computation to both CPU and GPU. Our current implementation uses parallel skeletons to transparently leverage Intel Threading Building Blocks (Intel Corporation, Santa Clara, CA, USA) and NVIDIA CUDA (Nvidia Corporation, Santa Clara, CA, USA). In our experiments, we observed that parallel applications with task partitioning can improve average performance by up to 76% and 28% compared with CPU‐only and GPU‐only parallel applications, respectively. Copyright © 2015 John Wiley & Sons, Ltd.

Automatic Task-Based Code Generation for High Performance Domain Specific Embedded Language

Providing high level tools for parallel programming while sustaining a high level of performance has been a challenge that techniques like Domain Specific Embedded Languages try to solve. In previous works, we investigated the design of such a DSEL--NT$$^2$$2--providing a Matlab -like syntax for parallel numerical computations inside a C++ library. In this paper, we show how NT$$^2\!$$2 has been redesigned for shared memory systems in an extensible and portable way. The new NT$$^2\!$$2 design relies on a tiered Parallel Skeleton system built using asynchronous task management and automatic compile-time taskification of user level code. We describe how this system can operate various shared memory runtimes and evaluate the design by using two benchmarks implementing linear algebra algorithms.

Разработка и применение вычислительной модели типовых решений. Пример использования «портфеля задач» для обучения нейронной сети HRBF

The article proposes a solution to the problem of mapping an algorithm from the field of Computational Mathematics on the target computing environment. The solution is based on a formal method for constructing parallel skeletons. The method comprises a specification of concurrency with the directed graphs and a formula for interpretation of dynamic behavior of such graphs. This interpretation is based on Temporal Logic of Actions approach proposed by Leslie Lamport. To illustrate the use of the method the “bag-oftasks” parallel skeleton is discussed hereinafter. We present graphically basic skeleton operations with the proposed computational model. After that we specify a learning algorithm of hyper-radial basis function neural network in the terms of skeleton operations as a case study. This made it possible to parallelize the leaning algorithm and map it on desired computing environments with predefined run-time libraries. Computational experiments confirming that our approach does not reduce the performance of the resulting programs are presented. The approach is suitable for researchers not familiar with parallel computing. It helps to get a reliable and effective supercomputer application both for SMP and distributed architectures.

An Algorithm Template for Domain-Based Parallel Irregular Algorithms

The parallelization of irregular algorithms has not been as widely studied as the one of regular codes. In particular, while there are many proposals of parallel skeletons and libraries very well suited to regular algorithms, this is not the case for irregular ones. This is probably due to the complexity of finding common patterns, behaviors and semantics in these algorithms. This is unfortunate, as the parallelization of irregular algorithms would benefit even more than that of regular codes from the higher degree of abstraction provided by skeletons. This work proposes to exploit the concept of domain defined on some property of the elements to process in order to enable the simple and effective parallelization of irregular applications. Namely, we propose to use such domains both to decompose the computations in parallel tasks and to detect and avoid conflicts between these tasks. A generic C++ library providing a skeleton for multicore systems built on this idea is described and evaluated. Our experimental results show that this library is a very practical tool for the parallelization of irregular algorithms with little programming effort.

An Automatic Fusion Mechanism for Variable-Length List Skeletons in SkeTo

Skeletal parallel programming is a promising approach to easy parallel programming in which users can build parallel programs simply by combining parts of a given set of ready-made parallel computation patterns called skeletons. There is a trade-off for this easiness in the form of an efficiency problem caused by the compositional style of the programming. One solution to this problem is fusion transformation that optimizes naively composed skeleton programs by eliminating redundant intermediate data structures. Several parallel skeleton libraries have automatic fusion mechanisms. However, there have been no automatic fusion mechanisms proposed for variable-length list (VLL) skeletons, even though such skeletons are useful for practical problems. The main difficulty is that previous fusion mechanisms are not applicable to VLL skeletons, and so the fusion cannot be completed. In this paper, we propose a novel fusion mechanism for VLL skeletons that can achieve both an easy programming interface and complete fusion. The proposed mechanism has been implemented in our skeleton library, SkeTo, by using the expression templates technique, experimental results have shown that it is very effective.

Efficient Abstractions for GPGPU Programming

General purpose (GP)GPU programming demands to couple highly parallel computing units with classic CPUs to obtain a high performance. Heterogenous systems lead to complex designs combining multiple paradigms and programming languages to manage each hardware architecture. In this paper, we present tools to harness GPGPU programming through the high-level OCaml programming language. We describe the SPOC library that allows to handle GPGPU subprograms (kernels) and data transfers between devices. We then present how SPOC expresses GPGPU kernel: through interoperability with common low-level extensions (from Cuda and OpenCL frameworks) but also via an embedded DSL for OCaml. Using simple benchmarks as well as a real world HPC software, we show that SPOC can offer a high performance while efficiently easing development. To allow better abstractions over tasks and data, we introduce some parallel skeletons built upon SPOC as well as composition constructs over those skeletons.

Eden's Bees: Parallelizing Artificial Bee Colony in a Functional Environment

Due to the proliferation of multicore computers, most users own hardware that allows to speedup the execution of programs by running them in parallel. However, in practice it is not trivial to take advantage of such parallel architectures, because the programmer needs to take care of too many low level details. This is also true in problems with a high degree of inherent parallelism, like many bioinspired metaheuristics. In this paper we simplify the parallelization of one of such metaheuristics, namely Artificial Bee Colony, by using a functional language, Eden, to implement a parallel skeleton to deal with it. Once the skeleton is defined, the user only needs to provide a concrete fitness function, while all the low level parallel details are done automatically by the skeleton.

Magneto-structural study and synthesis optimization of a phosphovanadate copper complex, [Cu(VO)2(PO4)2(H2O)4] n

The optimized synthesis of a phospho-vanadyl compound, [Cu(VO)2(PO4)2(H2O)4] n , is presented and its structural analysis is performed and discussed in relation to magnetic properties. The structure consists of a 2-D skeleton parallel to (100), made up by two independent VO6 octahedra linked by phosphates. Neighboring 2-D structures stack along [100] and are connected by aqua-mediated H-bonds. The spatial disposition of paramagnetic ions gives rise to weak antiferromagnetic behavior, with J = −17 cm−1, which is consistent with a model of three paramagnetic centers in an irregular triangular arrangement. An optimization of the synthesis process, looking for a response surface with optimum experimental values, was performed using a multivariate analysis.

AUTO-TUNING PARALLEL SKELETONS

Parallel skeletons are a structured parallel programming abstraction that provide programmers with a predefined set of algorithmic templates that can be combined, nested and parameterized with sequential code to produce complex programs. The implementation of these skeletons is currently a manual process, requiring human expertise to choose suitable implementation parameters that provide good performance. This paper presents an empirical exploration of the optimization space of the FastFlow parallel skeleton framework. We performed this using a Monte Carlo search of a random subset of the space, for a representative set of platforms and programs. The results show that the space is program and platform dependent, non-linear, and that automatic search achieves a significant average speedup in program execution time of 1.6× over a human expert. An exploratory data analysis of the results shows a linear dependence between two of the parameters, and that another two parameters have little effect on performance. These properties are then used to reduce the size of the space by a factor of 6, reducing the cost of the search. This provides a starting point for automatically optimizing parallel skeleton programs without the need for human expertise, and with a large improvement in execution time compared to that achievable using human expert tuning.

Easy composition of symbolic computation software using SCSCP: A new Lingua Franca for symbolic computation

We present the results of the first four years of the European research project SCIEnce—Symbolic Computation Infrastructure in Europe (http://www.symbolic-computing.org), which aims to provide key infrastructure for symbolic computation research. A primary outcome of the project is that we have developed a new way of combining computer algebra systems using the Symbolic Computation Software Composability Protocol (SCSCP), in which both protocol messages and data are encoded in the OpenMath format. We describe the SCSCP middleware and APIs, outline implementations for various Computer Algebra Systems (CAS), and show how SCSCP-compliant components may be combined to solve scientific problems that cannot be solved within a single CAS, or may be organised into a system for distributed parallel computations. Additionally, we present several domain-specific parallel skeletons that capture commonly used symbolic computations. To ease use and to maximise inter-operability, these skeletons themselves are provided as SCSCP services and take SCSCP services as arguments.

A family of skeletons for motion planning and geometric reasoning applications

AbstractThe task of planning a path between two spatial configurations of an artifact moving among obstacles is an important problem in practically all geometrically intensive applications. Despite the ubiquity of the problem, the existing approaches make specific limiting assumptions about the geometry and mobility of the obstacles, or those of the environment in which the motion of the artifact takes place. We present a strategy to construct a family of paths, or roadmaps, for two- and three-dimensional solids moving in an evolving environment that can undergo drastic topological changes. Our approach is based on a potent paradigm for constructing geometric skeletons that relies on constructive representations of shapes with R-functions that operate on real-valued half-spaces as logic operations. We describe a family of skeletons that have the same homotopy as that of the environment and contains the medial axis as a special case. Of importance, our skeletons can be designed so that they are “attracted to” or “repulsed by” prescribed spatial sites of the environment. Moreover, the R-function formulation suggests the new concept of a medial zone, which can be thought of as a “thick” skeleton with significant applications for motion planning and other geometric reasoning applications. Our approach can handle problems in which the environment is not fully known a priori, and intrinsically supports local and parallel skeleton computations for domains with rigid or evolving boundaries. Furthermore, our path planning algorithm can be implemented in any commercial geometric kernel, and has attractive computational properties. The capability of the proposed technique are explored through several examples designed to simulate highly dynamic environments.

A Practical Tree Contraction Algorithm for Parallel Skeletons on Trees of Unbounded Degree

Abstract Algorithmic skeletons are ready-made parallel computation patterns. Since each skeleton can be evaluated efficiently on parallel computation environments, we can develop effcient parallel programs only by specifying our computation by a combination of skeletons. Although effectiveness of algorithmic skeletons, especially those manipulating arrays and lists, is now well-recognized, those for trees of unbounded degree have not been firmly established. Most of the existing studies transform them to binary trees through preprocessing. But this approach is not practical. The transformation not only makes developments of parallel programs diffcult but also affects the performance of the parallel programs developed. In this paper, we propose a parallel tree contraction algorithm named Rake-Shunt contraction algorithm. It is generalization of the Shunt contraction algorithm, which has been used as the algorithmic basis of the binary-tree skeletons, and inherits several good properties such as scalability with respect to the number of processors and simplicity of the implementation. Moreover, it can deal with arbitrary trees and requires no modification of their shapes. Our preliminary experiments show that our algorithm leads to an implementation of the tree reduction, one of the most important tree skeletons, that is easier to use and more effcient than the previous method based on the transformation to binary trees.

Performance analysis of Branch-and-Bound skeletons

This article proposes a study of load balancing and communication pattern for Branch-and-Bound algorithms. Concretely, sequential and parallel generic skeletons to implement this algorithmic technique are presented. To accomplish the work the call and paraver tools are used. call allows us to annotate the code by hand at the special points with a complexity function. Also, some computational results are presented.