Multi-core Clusters Research Articles

New Model-Based Methods and Algorithms for Performance and Energy Optimization of Data Parallel Applications on Homogeneous Multicore Clusters

Modern homogeneous parallel platforms are composed of tightly integrated multicore CPUs. This tight integration has resulted in the cores contending for various shared on-chip resources such as Last Level Cache (LLC) and interconnect, leading to resource contention and non-uniform memory access (NUMA). Due to these newly introduced complexities, the performance and energy profiles of real-life scientific applications on these platforms are not smooth and may deviate significantly from the shapes that allowed traditional and state-of-the-art load balancing algorithms to minimize their computation time. In this paper, we propose new model-based methods and algorithms for minimization of time and energy of computations for the most general shapes of performance and energy profiles of data parallel applications observed on the modern homogeneous multicore clusters. We formulate the performance and energy optimization problems and present efficient algorithms of complexity O(p <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) solving these problems where p is the number of processors. It is important to note that the globally optimal solutions found by these algorithms may not load-balance the application. We experimentally study the efficiency and scalability of our algorithms for two data parallel applications, matrix multiplication and fast Fourier transform, on a modern multicore CPU and clusters of such CPUs. We also demonstrate the optimality of solutions determined by our algorithms.

A hierarchical load balancing parallel computing approach for finite element structural analysis

Multi-core clusters have become primary tools for high performance computing due to their great computing power and cost-to-performance effectiveness in nowadays. However, it introduces new challenges for the design of efficient parallel algorithms because of the different storage mechanisms and non-uniform communication latencies on these machines. The traditional domain decomposition methods use the direct partition method to achieve load balancing, which directly divides the structure into a number of subdomains with equal according to the number of processing cores involved in parallel computing. As the number of processing cores in a single node of multi-core clusters increases exponentially, the number of subdomains will increase dramatically as well. A substantial increase in the number of subdomains leads to the rapid expansion of the size and the condition number of interface equations, thereby reducing the numerical convergence of the system. In addition, it leads to a considerable increase in the number of processes involved in parallel computing, thereby increasing contention for the limited network ports and bandwidth. The decrease of the numerical convergence and the increase of network communication overheads seriously affect the solution efficiency of interface equations, and greatly reduce the overall parallel efficiency of the domain decomposition method. In order to make full use of the computing power of multi-core clusters to improve the parallel efficiency of large-scale finite element structural analysis, a hierarchical load balancing approach is proposed in the paper. The proposed approach is based on the full mining of computational tasks. In order to adapt to the hardware topology architecture of multi-core clusters, the computational tasks of finite element structural analysis are divided into three layers: inter-node parallelism, inter-chip parallelism and inter-core parallelism. The coarse grain parallel computing method is utilized in inter-node parallelism and inter-chip parallelism, and the fine grain parallel computing method is used in inter-core parallelism. Through mapping computing tasks to different hardware layers of multi-core clusters, the proposed method not only efficiently achieves the load balancing at different layers, but also greatly reduces the communication overheads of the system. Furthermore, it considerably reduces the number of subdomains and significantly improves the numerical convergence of the interface equations. In order to verify the effectiveness of the algorithms, two numerical experiments about finite element structural linear static analysis for large-scale parallel computing were conducted on “Tianhe 2” supercomputer. For each model, both the traditional domain decomposition method and the proposed hierarchical load balancing approach were employed for numerical simulation utilizing 50, 100, 150, and 200 nodes, respectively. Test results show that the proposed method could obtain higher speedup and parallel efficiency compared with the conventional domain decomposition method. The proposed approach can be widely used for solving many kinds of structural analysis problems including linear static analysis, nonlinear static analysis and nonlinear dynamic analysis and so on. In this paper, the authors’ current research only focuses on the linear static analysis. For the nonlinear static or dynamic analysis and other kinds of structural analysis, the proposed method can be used as a sub-procedure because the calculations are still dominated by solutions of the same sort of equations.

Near-Threshold RISC-V Core With DSP Extensions for Scalable IoT Endpoint Devices

Endpoint devices for Internet-of-Things not only need to work under extremely tight power envelope of a few milliwatts, but also need to be flexible in their computing capabilities, from a few kOPS to GOPS. Near-threshold (NT) operation can achieve higher energy efficiency, and the performance scalability can be gained through parallelism. In this paper, we describe the design of an open-source RISC-V processor core specifically designed for NT operation in tightly coupled multicore clusters. We introduce instruction extensions and microarchitectural optimizations to increase the computational density and to minimize the pressure toward the shared-memory hierarchy. For typical data-intensive sensor processing workloads, the proposed core is, on average, $3.5\times $ faster and $3.2\times $ more energy efficient, thanks to a smart L0 buffer to reduce cache access contentions and support for compressed instructions. Single Instruction Multiple Data extensions, such as dot products, and a built-in L0 storage further reduce the shared-memory accesses by $8\times $ reducing contentions by $3.2\times $ . With four NT-optimized cores, the cluster is operational from 0.6 to 1.2 V, achieving a peak efficiency of 67 MOPS/mW in a low-cost 65-nm bulk CMOS technology. In a low-power 28-nm FD-SOI process, a peak efficiency of 193 MOPS/mW (40 MHz and 1 mW) can be achieved.

A general and efficient divide-and-conquer algorithm framework for multi-core clusters

Divide-and-conquer is one of the most important patterns of parallelism, being applicable to a large variety of problems. In addition, the most powerful parallel systems available nowadays are computer clusters composed of distributed-memory nodes that contain an increasing number of cores that share a common memory. The optimal exploitation of these systems often requires resorting to a hybrid model that mimics the underlying hardware by combining a distributed and a shared memory parallel programming model. This results in longer development times and increased maintenance costs. In this paper we present a very general skeleton library that allows to parallelize any divide-and-conquer problem in hybrid distributed-shared memory systems with little effort while providing much flexibility and good performance. Our proposal combines a message-passing paradigm at the process level and a threaded model inside each process, hiding the related complexity from the user. The evaluation shows that this skeleton provides performance comparable, and often better than that of manually optimized codes while requiring considerably less effort when parallelizing applications on multi-core clusters.

A scalable parallel algorithm for atmospheric general circulation models on a multi-core cluster

High-performance computing of atmospheric general circulation models (AGCMs) has been receiving increasing attention in earth science research. However, when scaling to large-scale multi-core computing, the parallelization of an AGCM which demands fast parallel computing for long-time integration or climate simulation becomes extremely challenging due to its inner complex numerical calculation. The previous Institute of Atmospheric Physics of the Chinese Academy of Sciences Atmospheric General Circulation Model version 4.0 (IAP AGCM4.0) with one-dimensional domain decomposition can only run on dozens of CPU cores, so the paper proposes a two-dimensional domain decomposition parallel algorithm for it. In the parallel implementation of the IAP AGCM4.0, its dynamical core utilizes a hybrid form of latitude/longitude decomposition and vertical direction/longitude circle direction decomposition. Through experiments on a multi-core cluster, we confirmed that our algorithm is efficient and scalable. The parallel efficiency of the IAP AGCM4.0 can reach up to 50.88% on 512 CPU cores, and the IAP AGCM4.0 can be run long-term simulations for climate change research.

Fast Parallel Construction of Correlation Similarity Matrices for Gene Co-Expression Networks on Multicore Clusters

Abstract Gene co-expression networks are gaining attention in the present days as useful representations of biologically interesting interactions among genes. The most computationally demanding step to generate these networks is the construction of the correlation similarity matrix, as all pairwise combinations must be analyzed and complexity increases quadratically with the number of genes. In this paper we present MPICorMat, a hybrid MPI/OpenMP parallel approach to construct similarity matrices based on Pearson’s correlation. It is based on a previous tool (RMTGeneNet) that has been used on several biological studies and proved accurate. Our tool obtains the same results as RMTGeneNet but significantly reduces runtime on multicore clusters. For instance, MPICorMat generates the correlation matrix of a dataset with 61,170 genes and 160 samples in less than one minute using 16 nodes with two Intel Xeon Sandy-Bridge processors each (256 total cores), while the original tool needed almost 4.5 hours. The tool is also compared to another available approach to construct correlation matrices on multicore clusters, showing better scalability and performance. MPICorMat is an open-source software and it is publicly available at https://sourceforge.net/projects/mpicormat/ .

On the Parallel Programmability of JavaSymphony for Multi-cores and Clusters

This paper explains the programming aspects of a promising Java-based programming and execution framework called JavaSymphony. JavaSymphony provides unified high-level programming constructs for applications related to shared, distributed, hybrid memory parallel computers, and co-processors accelerators. JavaSymphony applications can be executed on multi-/many-core conventional and data-parallel architectures. JavaSymphony is based on the concept of dynamic virtual architectures, which allows programmers to define a hierarchical structure of the underlying computing resources and to control load-balancing and task-locality. In addition to GPU support, JavaSymphony provides a multi-core aware scheduling mechanism capable of mapping parallel applications on large multi-core machines and heterogeneous clusters. Several real applications and benchmarks (on modern multi-core computers, heterogeneous clusters, and machines consisting of a combination of different multi-core CPU and GPU devices) have been used to evaluate the performance. The results demonstrate that the JavaSymphony outperforms the Java implementations, as well as other modern alternative solutions.

Heuristic Experiments of Threading and Equal Load Partitioning For Hierarchical Heterogeneous Cluster

Presently, the issue of processing large data on a timely manner poses as a challenge to many ICT researchers. Most commodity computers are interconnected in a network forming a cluster computing resource simulating a super computer. This paper explores heuristically the performance of homogeneous, heterogeneous and multi-core clusters. This work consists of five experiments: Equal task partitioning according to the number of nodes in homogeneous cluster, number of nodes in heterogeneous cluster, number of nodes in heterogeneous cluster with multithreading, number of cores in heterogeneous cluster and number of cores in heterogeneous cluster with multithreading. The task is Sobel edge detection method tested with an array of images. The images are processed in three different sizes; 1K × 1K, 2K × 2K and 3K × 3K. The performance evaluations are based on processing speed. The results yield promising impact of equal partitioning and threading in parallel processing hierarchical heterogeneous cluster.

Parallel definition of tear film maps on distributed-memory clusters for the support of dry eye diagnosis

Background and objectivesThe analysis of the interference patterns on the tear film lipid layer is a useful clinical test to diagnose dry eye syndrome. This task can be automated with a high degree of accuracy by means of the use of tear film maps. However, the time required by the existing applications to generate them prevents a wider acceptance of this method by medical experts. Multithreading has been previously successfully employed by the authors to accelerate the tear film map definition on multicore single-node machines. In this work, we propose a hybrid message-passing and multithreading parallel approach that further accelerates the generation of tear film maps by exploiting the computational capabilities of distributed-memory systems such as multicore clusters and supercomputers. MethodsThe algorithm for drawing tear film maps is parallelized using Message Passing Interface (MPI) for inter-node communications and the multithreading support available in the C++11 standard for intra-node parallelization. The original algorithm is modified to reduce the communications and increase the scalability. ResultsThe hybrid method has been tested on 32 nodes of an Intel cluster (with two 12-core Haswell 2680v3 processors per node) using 50 representative images. Results show that maximum runtime is reduced from almost two minutes using the previous only-multithreaded approach to less than ten seconds using the hybrid method. ConclusionsThe hybrid MPI/multithreaded implementation can be used by medical experts to obtain tear film maps in only a few seconds, which will significantly accelerate and facilitate the diagnosis of the dry eye syndrome.

On the Implementation of an Or-Parallel Prolog System for Clusters of Multicores

AbstractNowadays, clusters of multicores are becoming the norm and, although, many or-parallel Prolog systems have been developed in the past, to the best of our knowledge, none of them was specially designed to explore the combination of shared and distributed memory architectures. In recent work, we have proposed a novel computational model specially designed for such combination which introduces a layered model with two scheduling levels, one for workers sharing memory resources, which we named a team of workers, and another for teams of workers (not sharing memory resources). In this work, we present a first implementation of such model and for that we revive and extend the YapOr system to exploit or-parallelism between teams of workers. We also propose a new set of built-in predicates that constitute the syntax to interact with an or-parallel engine in our platform. Experimental results show that our implementation is able to increase speedups as we increase the number of workers per team, thus taking advantage of the maximum number of cores in a machine, and to increase speedups as we increase the number of teams, thus taking advantage of adding more computer nodes to a cluster.

A Comment on “Process Placement in Multicore Clusters: Algorithmic Issues and Practical Techniques”

In “Process placement in multicore clusters: Algorithmic issues and practical techniques,” Jeannot, Mercier, and Tessier presented an algorithm called TreeMatch for determining the best placement of a set of communicating processes on a hierarchically structured computing architecture, described by a tree. In order to speed up the algorithm, it was suggested to decompose levels of the tree with high arity into several levels of smaller arity. The authors conjectured what the optimal strategy for decomposition is. In this contribution, we prove that their conjecture was right.

The 1.1 Å resolution structure of a periplasmic phosphate-binding protein fromStenotrophomonas maltophilia: a crystallization contaminant identified by molecular replacement using the entire Protein Data Bank

During efforts to crystallize the enzyme 2,4-dihydroxyacetophenone dioxygenase (DAD) from Alcaligenes sp. 4HAP, a small number of strongly diffracting protein crystals were obtained after two years of crystal growth in one condition. The crystals diffracted synchrotron radiation to almost 1.0 Å resolution and were, until recently, assumed to be formed by the DAD protein. However, when another crystal form of this enzyme was eventually solved at lower resolution, molecular replacement using this new structure as the search model did not give a convincing solution with the original atomic resolution data set. Hence, it was considered that these crystals might have arisen from a protein impurity, although molecular replacement using the structures of common crystallization contaminants as search models again failed. A script to perform molecular replacement using MOLREP in which the first chain of every structure in the PDB was used as a search model was run on a multi-core cluster. This identified a number of prokaryotic phosphate-binding proteins as scoring highly in the MOLREP peak lists. Calculation of an electron-density map at 1.1 Å resolution based on the solution obtained with PDB entry 2q9t allowed most of the amino acids to be identified visually and built into the model. A BLAST search then indicated that the molecule was most probably a phosphate-binding protein from Stenotrophomonas maltophilia (UniProt ID B4SL31; gene ID Smal_2208), and fitting of the corresponding sequence to the atomic resolution map fully corroborated this. Proteins in this family have been linked to the virulence of antibiotic-resistant strains of pathogenic bacteria and with biofilm formation. The structure of the S. maltophilia protein has been refined to an R factor of 10.15% and an Rfree of 12.46% at 1.1 Å resolution. The molecule adopts the type II periplasmic binding protein (PBP) fold with a number of extensively elaborated loop regions. A fully dehydrated phosphate anion is bound tightly between the two domains of the protein and interacts with conserved residues and a number of helix dipoles.

Compiling Affine Loop Nests for a Dynamic Scheduling Runtime on Shared and Distributed Memory

Current de-facto parallel programming models like OpenMP and MPI make it difficult to extract task-level dataflow parallelism as opposed to bulk-synchronous parallelism . Task parallel approaches that use point-to-point synchronization between dependent tasks in conjunction with dynamic scheduling dataflow runtimes are thus becoming attractive. Although good performance can be extracted for both shared and distributed memory using these approaches, there is little compiler support for them. In this article, we describe the design of compiler--runtime interaction to automatically extract coarse-grained dataflow parallelism in affine loop nests for both shared and distributed-memory architectures. We use techniques from the polyhedral compiler framework to extract tasks and generate components of the runtime that are used to dynamically schedule the generated tasks. The runtime includes a distributed decentralized scheduler that dynamically schedules tasks on a node. The schedulers on different nodes cooperate with each other through asynchronous point-to-point communication, and all of this is achieved by code automatically generated by the compiler. On a set of six representative affine loop nest benchmarks, while running on 32 nodes with 8 threads each, our compiler-assisted runtime yields a geometric mean speedup of 143.6× (70.3× to 474.7× ) over the sequential version and a geometric mean speedup of 1.64× (1.04× to 2.42× ) over the state-of-the-art automatic parallelization approach that uses bulk synchronization . We also compare our system with past work that addresses some of these challenges on shared memory, and an emerging runtime (Intel Concurrent Collections) that demands higher programmer input and effort in parallelizing. To the best of our knowledge, ours is also the first automatic scheme that allows for dynamic scheduling of affine loop nests on a cluster of multicores.

Crystal MD: The massively parallel molecular dynamics software for metal with BCC structure

Material irradiation effect is one of the most important keys to use nuclear power. However, the lack of high-throughput irradiation facility and knowledge of evolution process, lead to little understanding of the addressed issues. With the help of high-performance computing, we could make a further understanding of micro-level-material. In this paper, a new data structure is proposed for the massively parallel simulation of the evolution of metal materials under irradiation environment. Based on the proposed data structure, we developed the new molecular dynamics software named Crystal MD. The simulation with Crystal MD achieved over 90% parallel efficiency in test cases, and it takes more than 25% less memory on multi-core clusters than LAMMPS and IMD, which are two popular molecular dynamics simulation software. Using Crystal MD, a two trillion particles simulation has been performed on Tianhe-2 cluster.

An approach for an efficient execution of SPMD applications on Multi-core environments

Executing traditional Message Passing Interface (MPI) applications on multi-core cluster balancing speed and computational efficiency is a difficult task that parallel programmers have to deal with. For this reason, communications on multi-core clusters ought to be handled carefully in order to improve performance metrics such as efficiency, speedup, execution time and scalability. In this paper we focus our attention on SPMD (Single Program Multiple Data) applications with high communication volume and synchronicity and also following characteristics such as: static, local and regular. This work proposes a method for SPMD applications, which is focused on managing the communication heterogeneity (different cache level, RAM memory, network, etc.) on homogeneous multi-core computing platform in order to improve the application efficiency. In this sense, the main objective of this work is to find analytically the ideal number of cores necessary that allows us to obtain the maximum speedup, while the computational efficiency is maintained over a defined threshold (strong scalability). This method also allows us to determine how the problem size must be increased in order to maintain an execution time constant while the number of cores are expanded (weak scalability) considering the tradeoff between speed and efficiency. This methodology has been tested with different benchmarks and applications and we achieved an average improvement around 30.35% of efficiency in applications tested using different problems sizes and multi-core clusters. In addition, results show that maximum speedup with a defined efficiency is located close to the values calculated with our analytical model with an error rate lower than 5% for the applications tested.

Highly efficient spatial data filtering in parallel using the opensource library CPPPO

CPPPO is a compilation of parallel data processing routines developed with the aim to create a library for “scale bridging” (i.e. connecting different scales by mean of closure models) in a multi-scale approach. CPPPO features a number of parallel filtering algorithms designed for use with structured and unstructured Eulerian meshes, as well as Lagrangian data sets. In addition, data can be processed on the fly, allowing the collection of relevant statistics without saving individual snapshots of the simulation state. Our library is provided with an interface to the widely-used CFD solver OpenFOAM®, and can be easily connected to any other software package via interface modules. Also, we introduce a novel, extremely efficient approach to parallel data filtering, and show that our algorithms scale super-linearly on multi-core clusters. Furthermore, we provide a guideline for choosing the optimal Eulerian cell selection algorithm depending on the number of CPU cores used. Finally, we demonstrate the accuracy and the parallel scalability of CPPPO in a showcase focusing on heat and mass transfer from a dense bed of particles. Program summaryProgram title: CPPPOCatalogue identifier: AFAQ_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AFAQ_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: GNU Lesser General Public License, version 3No. of lines in distributed program, including test data, etc.: 1043965No. of bytes in distributed program, including test data, etc.: 11053655Distribution format: tar.gzProgramming language: C++, MPI, octave.Computer: Linux based clusters for HPC or workstations.Operating system: Linux based.Classification: 4.14, 6.5, 12.External routines: Qt5, hdf5-1.8.15, jsonlab, OpenFOAM/CFDEM, Octave/MatlabNature of problem:Development of closure models for momentum, species transport and heat transfer in fluid and fluid–particle systems using purely Eulerian or Euler–Lagrange simulators.Solution method:The CPPPO library contains routines to perform on-line (i.e., runtime) filtering and compute statistics on large parallel data sets.Running time:Performing a Favre averaging on a structured mesh of 1283 cells with a filter size of 643 cells using one Intel Xeon(R) E5-2650, requires approximately 4 h of computation.

Graphine: Programming Graph-Parallel Computation of Large Natural Graphs for Multicore Clusters

Graph-parallel computation has become a crucial component in emerging applications of web search, data analytics and machine learning. In practice, most graphs derived from real-world phenomena are very large and scale-free. Unfortunately, distributed graph-parallel computation of these natural graphs still suffers strong scalability issues on contemporary multicore clusters. To embrace the multicore architecture in distributed graph-parallel computation, we propose the framework Graphine, which features (i) A Scatter-Combine computation abstraction that is evolved from the traditional vertex-centric approach by fusing the paired scatter and gather operations, executed separately on two edge sides, into a one-sided scatter. Further coupled with active message mechanism, it potentially reduces intermediate message cost and enables fine-grained parallelism on multicore architecture. (ii) An Agent-Graph data model, which leverages an idea similar to vertex-cut but conceptually splits the remote replica into two agent types of scatter and combiner, resulting in less communication. We implement the Graphine framework and evaluate it using several representative algorithms on six large real-world graphs and a series of synthetic graphs with power-law degree distributions. We show that Graphine achieves sublinear scalability with the number of cores per node, number of nodes, and graph sizes (up to one billion vertices), and is 2 $\sim$ 15 times faster than the state-of-the-art PowerGraph on a cluster of 16 multicore nodes.

Hybrid swarm intelligent parallel algorithm research based on multi-core clusters

In order to solve poor fine searching capacity of artificial fish swarm algorithm and artificial bee colony swarm algorithm in late state to result in insufficient local optimization, hybrid swarm intelligent parallel algorithm research based on multi-core clusters is proposed; Then, reverse learning mechanism is introduced in early stage of algorithm, initialized swarms are evenly distributed, and swarms are randomly divided into two groups to make interactive learning strategy accelerates rate of convergence, and basic artificial fish swarm algorithm and artificial bee colony swarm algorithm are used to make global searching. In late stage of algorithm, niches artificial fish swarm algorithm and Random Perturbation Artificial Bee Colony are used to make local fine searching to the solution obtained in early stage; On this basis, MPI+OpenMP+STM parallel programming model based on multi-core clusters is established for parallel design and analysis. Finally, stimulation experiment indicates optimizing efficiency of this algorithm is higher than single artificial fish swarm algorithm and artificial bee colony swarm algorithm.

Combining Data and Computation Distribution Directives for Hybrid Parallel Programming : A Transformation System

This paper describes dSTEP, a directive-based programming model for hybrid shared and distributed memory machines. The originality of our work is the definition and an implementation of a unified high-level programming model addressing both data and computation distributions, providing a particularly fine control of the computation. The goal is to improve the programmer productivity while providing good performances in terms of execution time and memory usage. We define a generic compilation scheme for computation mapping and communication generation. We implement the solution in a source-to-source compiler together with a runtime library. We provide a series of optimizations to improve the performance of the generated code, with a special focus on reducing the communications time. We evaluate our solution on several scientific kernels as well as on the more challenging NAS BT benchmark, and compare our results with the hand written Fortran MPI and UPC implementations. The results show first that our solution allows to make explicit the non trivial parallel execution of the NAS BT benchmark using the dSTEP directives. Second, the results show that our generated MPI+OpenMP BT program runs with a 83.35 speedup over the original NAS OpenMP C benchmark on a hybrid cluster composed of 64 quadricores (256 cores). Overall, our solution dramatically reduces the programming effort while providing good time execution and memory usage performances. This programming model is suitable for a large variety of machines as multi-core and accelerator clusters.

Parallel Optimal Pairwise Biological Sequence Comparison

Many bioinformatics applications, such as the optimal pairwise biological sequence comparison, demand a great quantity of computing resource, thus are excellent candidates to run in high-performance computing (HPC) platforms. In the last two decades, a large number of HPC-based solutions were proposed for this problem that run in different platforms, targeting different types of comparisons with slightly different algorithms and making the comparative analysis of these approaches very difficult. This article proposes a classification of parallel optimal pairwise sequence comparison solutions, in order to highlight their main characteristics in a unified way. We then discuss several HPC-based solutions, including clusters of multicores and accelerators such as Cell Broadband Engines (CellBEs), Field-Programmable Gate Arrays (FPGAs), Graphics Processing Units (GPUs) and Intel Xeon Phi, as well as hybrid solutions, which combine two or more platforms, providing the actual landscape of the main proposals in this area. Finally, we present open questions and perspectives in this research field.