Heterogeneous Computer Clusters Research Articles

Scalable and Distributed Mathematical Modeling Algorithm Design and Performance Evaluation in Heterogeneous Computing Clusters

Scalable and Distributed Mathematical Modeling Algorithm Design and Performance Evaluation in Heterogeneous Computing Clusters

Improving Hadoop MapReduce performance on heterogeneous single board computer clusters

Over the past decade, Apache Hadoop has become a leading framework for big data processing. Single board computer (SBC) clusters, predominantly adopting Raspberry Pi (RPi), have been employed to explore the potential of MapReduce processing in terms of low power and cost because, capital costs aside, power consumption has also become a primary concern in many industries. After building SBC clusters, it is prevalent to consider adding more nodes, particularly newer generation SBCs, to the existing clusters or replacing old (or inactive) nodes with new ones to improve performance, inevitably causing heterogeneous SBC clusters. The Hadoop framework on these heterogeneous SBC clusters creates challenging new problems due to computing resource discrepancies in each node. Native Hadoop does not carefully consider the heterogeneity of the cluster nodes. Consequently, heterogeneous SBC Hadoop clusters result in significant performance variation or, more critically, persistent node failures. This paper proposes a new Hadoop Yet Another Resource Negotiator (YARN) architecture design to improve MapReduce performance on heterogeneous SBC Hadoop clusters with tight computing resources. We newly implement two main scheduling policies on Hadoop YARN based on the correct computing resource information that each SBC node provides: (1) two (master-driven vs. slave-driven) MapReduce task scheduling frameworks to determine more effective processing modes and (2) ApplicationMaster (AM) and reduce task distribution mechanisms to provide the best Hadoop performance by minimizing performance variation. Thus, the proposed Hadoop framework makes the best use of the performance-frugal SBC Hadoop cluster by intelligently distributing MapReduce tasks to each node. To our knowledge, the proposed framework is the first redesigned Hadoop YARN architecture to address various challenging problems particularly on heterogeneous SBC Hadoop clusters for big data processing. The extensive experiments with Hadoop benchmarks demonstrate that the redesigned framework performs better performance than the native Hadoop by an average of 2.55× and 1.55× under I/O intensive and CPU-intensive workloads, respectively.

Data distribution and parallel code generation for heterogeneous computational clusters

We present new techniques for compilation of sequential programs for almost affine accesses in loop nests for distributed-memory parallel architectures. Our approach is implemented as a source-to-source automatic parallelizing compiler that expresses parallelism with the DVMH directive-based programming model. Compared to all previous approaches ours addresses all three main sub-problems of the problem of distributed memory parallelization: data and computation distribution and communication optimization. Parallelization of sequential programs with structured grid computations is considered. In this paper, we use the NAS Parallel Benchmarks to evaluate the performance of generated programs and provide experimental results on up to 9 nodes of a computational cluster with two 8-core processors in a node.

DVM-Approach to the Automation of the Development of Parallel Programs for Clusters

The DVM-approach to the development of parallel programs for heterogeneous computer clusters with accelerators. The basic capabilities of DVM and SAPFOR , which automate the parallelization of applications, are discussed.

Решение прикладных задач с использованием DVM-системы

DVM-system was designed to create parallel programs of scientific-technical computations in C-DVMH and Fortran-DVMH languages. These languages use the same model of parallel programming (DVMH-model) and are the extensions of standard C and Fortran languages by parallelism specifications, implemented as compiler directives. DVMH-model allows creating efficient parallel programs for heterogeneous computational clusters, the nodes of which use as computing devices not only universal multi-core processors but also can use attached accelerators (GPUs or Intel Xeon Phi coprocessors). This article describes the experience of parallelizing various application programs using DVM-system. The method of incremental or partial parallelization, the system's capabilities for working with unstructured grids, new tools for mapping MPI-programs to multi-core processors and accelerators are considered. The efficiency of parallel DVMH-programs on heterogeneous computing clusters K-10, K-100, Lomonosov and MVS-10P is investigated. The main advantages of DVM-approach for the development of parallel programs are described. The main features of DVM-system tools for performance analysis and functional debugging of parallel programs are presented. The directions for further development of DVM-system are determined.

The experience of using DVM and SAPFOR systems in semi automatic parallelization of an application for 3D modeling in geophysics

The DVM and SAPFOR systems were primary designed to simplify the development of parallel programs of scientific-technical calculations. DVM includes CDVMH and Fortran-DVMH programming languages which are the extensions of standard C and Fortran languages by parallelism specifications. The DVMH programs can be efficiently executed on heterogeneous computational clusters with accelerators. SAPFOR is a software development suit that can be used to produce a parallel version of a sequential program in a semiautomatic way, according to DVMH model of the parallel programming. The paper considers the application of DVM and SAPFOR systems at all stages of the program parallelization on the example of mapping of 3D elastic waves simulation method on high-performance heterogeneous clusters with GPUs. SAPFOR enables us to investigate original sequential program. Then, Fortran-DVMH language is used to exploit parallelism. Tools for functional and performance debugging, that are part of the DVM system, allow to resolve errors in the parallel program and to improve its performance. The paper also compares the DVMH-based program with a program obtained after manual parallelization using MPI programming technology.

Sequence Similarity Parallelization over Heterogeneous Computer Clusters Using Data Parallel Programming Model

Sequence similarity, as a special case of data intensive applications, is one of the neediest applications for parallelization. Clustered commodity computers as a cost-effective platform for distributed and parallel processing, can be leveraged to parallelize sequence similarity. However, manually designing and developing parallel programs on commodity computers is a time-consuming, complex and error-prone process. In this paper, we present a sequence similarity parallelization technique using the Apache Storm as a stream processing framework with a data parallel programming model. Storm automatically parallelizes computations via a special user-defined topology that is represented as a directed acyclic graph. The proposed technique collects streams of data from a disk and sends them sequence by sequence to clustered computers for parallel processing. We also present a dispatching policy for balancing the cluster workload and managing the cluster heterogeneity to achieve more than 99 percent parallelism. An alignment-free method, known as n-gram modeling, is used to calculate similarities between the sequences. To show the cost-performance superiority of our method on clustered commodity computers over serial processing in powerful computers, we simply use UniProtKB/SwissProt dataset for evaluation of the performance of sequence similarity as an interesting large-scale Bioinformatics application.

Job flow management for virtualized resources of heterogeneous distributed computing environment

We address to the problem of an integration of heterogeneous computational clusters to the united environment applying the OpenStack platform for a cluster resource virtualization. However, this platform does not interact with the traditional resource management systems such as the PBS Torque or SLURM that are used in cluster resources. To this end, we developed an additional hypervisor shell to run virtual machines through queues of the aforementioned systems. Thus, we expand the OpenStack capabilities for an application job management and provide a computation scalability for virtualized resources of clusters. In order to show the benefits of our approach to the scalable application management in integrated cluster environments, we developed a parameter sweep application for simulation modeling of warehouse logistics and solved three optimization tasks for the real refrigerated warehouse. We compared the selected criteria of task solving processes with the hypervisor shell applying and without it by means of simulation modeling. Experiments show that the hypervisor shell can significantly improve computation speedup and task solving time through reallocating virtual machines to queues of the resource management systems.

Simple, efficient allocation of modelling runs on heterogeneous clusters with MPI

In scientific modelling and computation, the choice of an appropriate method for allocating tasks for parallel processing depends on the computational setting and on the nature of the computation. The allocation of independent but similar computational tasks, such as modelling runs or Monte Carlo trials, among the nodes of a heterogeneous computational cluster is a special case that has not been specifically evaluated previously. A simulation study shows that a method of on-demand (that is, worker-initiated) pulling from a bag of tasks in this case leads to reliably short makespans for computational jobs despite heterogeneity both within and between cluster nodes. A simple reference implementation in the C programming language with the Message Passing Interface (MPI) is provided.

Server consolidation for heterogeneous computer clusters using Colored Petri Nets and CPN Tools

In this paper, we present a new approach to server consolidation in heterogeneous computer clusters using Colored Petri Nets (CPNs). Server consolidation aims to reduce energy costs and improve resource utilization by reducing the number of servers necessary to run the existing virtual machines in the cluster. It exploits the emerging technology of live migration which allows migrating virtual machines between servers without stopping their provided services. Server consolidation approaches attempt to find migration plans that aim to minimize the necessary size of the cluster. Our approach finds plans which not only minimize the overall number of used servers, but also minimize the total data migration overhead. The latter objective is not taken into consideration by other approaches and heuristics. We explore the use of CPN Tools in analyzing the state spaces of the CPNs. Since the state space of the CPN model can grow exponentially with the size of the cluster, we examine different techniques to generate and analyze the state space in order to find good plans to server consolidation within acceptable time and computing power.

Parallel CFD simulation of flow in a 3D model of vibrating human vocal folds

Accurate computation of highly unsteady and massively separated airflow in human vocal folds during phonation, which is fundamental for the understanding and modeling of the aeroacoustic processes involved in human voice production, requires CFD simulations on large 3D dynamic meshes. This work presents a 3D model of flow past vibrating vocal folds solved by cell-centered finite volume method, verification of the solver on a benchmark cylinder cross-flow problem, grid dependence study and parallel scalability results obtained on a heterogeneous Linux computational cluster with 12 Intel Xeon dual-processor dual-core and 17 AMD Opteron dual-processor single-core nodes. Transient computations on dynamic meshes with up to 3.2M elements were performed on two geometries modeling convergent and divergent glottis with an angle of 20 degrees. In the results, the jet deflection angle, flow rate, glottal velocity and pressure, drag and lift force and jet contours (including phase-averaged data) are compared for these two configurations.

Autonomous System Management for the ALICE High-Level-Trigger Cluster using the SysMES framework

The ALICE HLT cluster is a heterogeneous computer cluster currently consisting of 200 nodes. This cluster is used for on-line processing of data produced by the ALICE detector during the next 10 or more years of operation. A major management challenge is to reduce the number of manual interventions in case of failures. Classical approaches like monitoring tools lack mechanisms to detect situations with multiple failure conditions and to automatically react to such situations. We have therefore developed SysMES (System Management for networked Embedded Systems and Clusters), a decentralized, fault tolerant, tool-set for autonomous management. It comprises a monitoring facility for detecting the working states of the distributed resources, a central interface for visualizing and managing the cluster environment and a rule system for coupling of the monitoring and management aspects. We have developed a formal language by which an administrator can define complex spatial and temporal conditions for failure states and according reactions. For the HLT we have defined a set of rules for known and recurring problem states such that SysMES takes care of most of day-to-day administrative work.

Power-aware linear programming based scheduling for heterogeneous computer clusters

In the past few years, scheduling for computer clusters has become a hot topic. The main focus has been towards achieving better performance. It is true that this goal has been attained to a certain extent, but on the other hand, it has been at the expense of increased energy consumption and consequent economic and environmental costs. As these clusters are becoming more popular and complex, reducing energy consumption in such systems has become a necessity. Several power-aware scheduling policies have been proposed for homogeneous clusters. In this work, we propose a new policy for heterogeneous clusters. Our simulation experiments show that using our proposed policy results in significant reduction in energy consumption while performing very competitively in heterogeneous clusters.

HeteroPBLAS: A Set of Parallel Basic Linear Algebra Subprograms Optimized for Heterogeneous Computational Clusters

This paper presents a software library, called Heterogeneous PBLAS (HeteroPBLAS), which provides optimized parallel basic linear algebra subprograms for Heterogeneous Computational Clusters. This library is written on the top of HeteroMPI and PBLAS whose building blocks, the de facto standard kernels for matrix and vector operations (BLAS) and message passing communication (BLACS), are optimized for heterogeneous computational clusters. This is the first step towards the development of a parallel linear algebra package for Heterogeneous Computational Clusters. We show that the efficiency of the parallel routines is due to the most important feature of the library, which is the automation of the difficult optimization tasks of parallel programming on heterogeneous computing clusters. They are the determination of the accurate values of the platform parameters such as the speeds of the processors and the latencies and bandwidths of the communication links connecting different pairs of processors, the optimal values of the algorithmic parameters such as the data distribution blocking factor, the total number of processes, the 2D process grid arrangement, and the efficient mapping of the processes executing the parallel algorithm to the executing nodes of the heterogeneous computing cluster. We present the user interface and the software hierarchy of the first research implementation of HeteroPBLAS. We demonstrate the efficiency of the HeteroPBLAS programs on a homogeneous computing cluster and a heterogeneous computing cluster.

Parallel and Distributed Computing Techniques, Selection of papers from ISPDC 2008

Parallel and Distributed Computing Techniques, Selection of papers from ISPDC 2008

Distributed loop‐scheduling schemes for heterogeneous computer systems

AbstractDistributed computing systems are a viable and less expensive alternative to parallel computers. However, a serious difficulty in concurrent programming of a distributed system is how to deal with scheduling and load balancing of such a system which may consist of heterogeneous computers. Some distributed scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been designed in the past. In this work we study self‐scheduling schemes for parallel loops with independent iterations which have been applied to multiprocessor systems in the past. We extend one important scheme of this type to a distributed version suitable for heterogeneous distributed systems. We implement our new scheme on a network of computers and make performance comparisons with other existing schemes. Copyright © 2005 John Wiley & Sons, Ltd.

Scalable loop self-scheduling schemes for heterogeneous clusters

Heterogeneous cluster systems (e.g., a LAN of computers) can be used for concurrent processing for some applications. However, a serious difficulty in concurrent programming of a heterogeneous system is how to deal with scheduling and load balancing of such a system that may consist of heterogeneous computers. Distributed scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been proposed and analysed in the past. Here, we implement the previous schemes in MPI. We present an extension of these schemes implemented in a hierarchical Master Slave architecture and include experimental results and comparisons.

A Parallel Loop Self-Scheduling on Extremely Heterogeneous PC Clusters

Cluster computers are a viable and less expensive alternative to symmetric multiprocessor systems. However, a serious difficulty in concurrent programming of a cluster computer system is how to deal with scheduling and load balancing of such a system which may consist of heterogeneous computers Self-scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been designed in the past These schemes, such as FSS, GSS and TSS, can achieve load balancing in SMP, even in a moderate heterogeneous environment, but are not suitable in extremely heterogeneous environments. In this paper, we propose a heuristic approach to solve parallel loop scheduling problems on an extremely heterogeneous PC cluster environment.

Distributed Parallel Computing for Fluid-Structure Coupled Simulations on a Heterogeneous Parallel Computer Cluster

Distributed parallel computing for a fluid-structure coupled simulation has been performed on a heterogeneous parallel computer cluster. The fluid and the structure dynamics are simulated on different parallel computers connected by a high-speed local network. These dynamics are coupled by a loose coupling method exchanging the boundary data between the fluid and the structure domains through the network. The data communication among parallel computers is realized by using the new communication library, Stampi, which has been developed to enable communication in a heterogeneous environment. The performance evaluation on a heterogeneous parallel computer cluster has shown that the distributed parallel computing for fluid-structure coupled simulations has the advantage of increasing the performance compared with the parallel computing on a single parallel computer.