Heterogeneous Multicore Clusters Research Articles

A New Hybrid Hierarchical Parallel Algorithm to Enhance the Performance of Large-Scale Structural Analysis Based on Heterogeneous Multicore Clusters

Heterogeneous multicore clusters are becoming more popular for high-performance computing due to their great computing power and cost-to-performance effectiveness nowadays. Nevertheless, parallel efficiency degradation is still a problem in large-scale structural analysis based on heterogeneous multicore clusters. To solve it, a hybrid hierarchical parallel algorithm (HHPA) is proposed on the basis of the conventional domain decomposition algorithm (CDDA) and the parallel sparse solver. In this new algorithm, a three-layer parallelization of the computational procedure is introduced to enable the separation of the communication of inter-nodes, heterogeneous-core-groups (HCGs) and inside-heterogeneous-core-groups through mapping computing tasks to various hardware layers. This approach can not only achieve load balancing at different layers efficiently but can also improve the communication rate significantly through hierarchical communication. Additionally, the proposed hybrid parallel approach in this article can reduce the interface equation size and further reduce the solution time, which can make up for the shortcoming of growing communication overheads with the increase of interface equation size when employing CDDA. Moreover, the distributed sparse storage of a large amount of data is introduced to improve memory access. By solving benchmark instances on the Shenwei-Taihuzhiguang supercomputer, the results show that the proposed method can obtain higher speedup and parallel efficiency compared with CDDA and more superior extensibility of parallel partition compared with the two-level parallel computing algorithm (TPCA).

Алгоритм автоматизованого розпаралелювання циклічних операторів для графічних прискорювачів

Parallelization of loop operators is a long standing problem of parallel programming. The widespread use of graphics processing units for computational tasks has resulted in the new statement of the mentioned problem for this class of multicore systems. The purpose of this work is to improve the mechanism of transformation of cyclic operators for loop parallelization for execution on a graphics processing unit. Software tool for computation optimization that allows to parallelize cyclic operators semi‑automatically was developed. Data bufferization synchronized with main loop execution was implemented, and the software tool using the rewriting rules system TermWare was built and integrated with the toolkit for design and synthesis of programs IDS. The developed system was tested using heterogeneous multicore cluster. The advantages of the developed system in comparison with well-known parallelization system Par4All consist in processing speed and the possibility of processing of data amounts exceeding the amount of memory of a graphics processing unit, and also the ability to use several graphics processing units simultaneously. The developed system was applied for parallelization of a serial loop, which is the part of a numerical weather forecasting program. Problems in programming 2017; 4: 028-036

Performance modeling for hierarchical graph partitioning in heterogeneous multi-core environment

A model to estimate the performance of graph partitioning running on heterogeneous multi-core clusters is proposed.We discover pitfalls of conventional methodologies in obtaining model parameters from multi-core systems.The impact of intra-node contention is too significant to be ignored.Modeling accuracy depends on whether overlap is adequately considered.Characteristics of input meshes may affect memory access behavior and hence become a determinant factor. Considering application behavior in graph partitioning is an arduous task because of the chicken-and-egg problem: the application behavior depends on how the graph is decomposed while achieving load balance requires the knowledge of how the application utilizes the underlying resources. Advances in multi-core processors further complicate the endeavor by introducing hardware diversity and intra-node contention. As an attempt to quantify performance for partitioning refinement, we propose a model that predicts execution times of iterative mesh-based applications running on heterogeneous multi-core clusters. Apart from considering resource heterogeneity, the model takes into account hierarchical communication characteristics, overlap between computation and communication, as well as performance penalties due to intra-node contention. We present a detailed methodology on how to obtain key parameters from a real system and highlight potential pitfalls of conventional approaches in obtaining the parameters. Experiments were conducted using a synthetic application benchmark solving a partial differential equation. Evaluation shows a good agreement between actual time measurement and the performance model.

Editorial: special issue on parallel nature-inspired optimization

Nature-inspired optimization techniques have proven to efficiently solve a wide range of problems related to parallel computing and more generally to computer science. In addition, the intrinsic parallelization and distribution capabilities of nature-inspired techniques can been exploited in order to provide powerful optimization solutions. However many research challenges remain to be addressed, such as the design and implementation of efficient nature-inspired optimization algorithms for massively parallel and distributed architectures and their application to real-world problems. This special issue is composed of extended versions of selected best papers presented at the 14th International Workshop on Nature Inspired Distributed Computing (NIDISC 2011). These articles provide a good theoretical and practical overview of nature-inspired optimization techniques and their application to metaheuristics and parallel/distributed computing. In the following, we propose a brief overview of the different contributions included in this special issue. In the first paper by E.A. Macedo et al., “Multiple Biological Sequence Alignment in Heterogeneous Multicore Clusters with User-Selectable Task Allocation Policies” a parallel version of a heuristic iterative algorithm, DIALIGN-TX, is proposed to tackle the Multiple Sequence Alignment (MSA) problem, a well-known NP-Hard bioinformatics problem. The authors empirically demonstrate the execution time reduction obtained with the proposed parallel implementation as well as the impact of the choice of the allocation policy. The second paper by P. Korosec et al., “Multi-core Implementation of the Differential Ant-Stigmergy Algorithm,” also proposes to parallelize a state-of-the-art optimization algorithm, the Differential Ant-Stigmergy algorithm (DASA) which is a

The impact of heterogeneous multi-core clusters on graph partitioning: an empirical study

The advent of multi-core architectures provides an opportunity for accelerating parallelism in mesh-based applications. This multi-core environment, however, imposes challenges not addressed by conventional graph-partitioning techniques that are originally designed for distributed-memory uniprocessors. As the first step to exploit the multi-core platform, this paper presents experimental evaluation to understand partitioning performance on small-scaled heterogeneous multi-core clusters. With results and analyses gathered, we propose a hierarchical framework for resource-aware graph partitioning on heterogeneous multi-core clusters. Preliminary evaluation demonstrates the potential of the framework and motivates directions for incorporating application requirements into graph partitioning.

Multiple biological sequence alignment in heterogeneous multicore clusters with user-selectable task allocation policies

Multiple Sequence Alignment (MSA) is an important problem in Bioinformatics that aims to align more than two sequences in order to emphasize similarity regions. This problem is known to be NP-Hard, so heuristic methods are used to solve it. DIALIGN-TX is an iterative heuristic method for MSA that generates alignments by concatenating ungapped regions with high similarity. Usually, the first phase of MSA algorithms is parallelized by distributing several independent tasks among the nodes. Even though heterogeneous multicore clusters are becoming very common nowadays, very few task allocation policies were proposed for this type of architecture. This paper proposes an MPI/OpenMP master/slave parallel strategy to run DIALIGN-TX in heterogeneous multicore clusters, with several allocation policies. We show that an appropriate choice of the master node has great impact on the overall system performance. Also, the results obtained in a heterogeneous multicore cluster composed of 4 nodes (30 cores), with real sequence sets show that the execution time can be drastically reduced when the appropriate allocation policy is used.

Performance evaluation of enhancement of the layered self-scheduling approach for heterogeneous multicore cluster systems

Previously we have proposed a Layered Self-Scheduling (LSS) approach that is a hybrid MPI and OpenMP based loop self-scheduling approach for dealing with the heterogeneity problem on a cluster system consisting of multi-core compute nodes, where the allocation functions of several well-known schemes have been modified for better performance. Though LSS provides better performance than the conventional self-scheduling schemes, we found the performance can be improved further after our comprehensive experiments and analyses. The newly proposed task scheduling strategy, called Enhanced Layered Self-Scheduling (ELSS), aims at how to utilize the compute powers of multiple processor cores more efficiently in the master compute node and how to schedule tasks to have more stable performance improvements. We have evaluated the new task scheduling strategy by three benchmark applications: Matrix Multiplication, Monte Carlo Integration, and Mandelbrot Set Computation. It is recommended that the global scheduler adopts Guided Self-Scheduling (GSS) for all, and the local scheduler adopts the static scheme for applications with regular workload distribution but any scheme for applications with irregular workload distribution. Experimental results show the best speedups obtained by ELSS for the three benchmark programs are 1.373, 13.34 and 2.4, respectively, compared with that scheduled by LSS.