Hybrid Parallel Programming in High Performance Computing Cluster

Abstract

This article provides a brief overview of approaches to calculating the complexity function for sequential programs. To determine the complexity of parallel programs, an approach based on operation analysis was used. The features of the parallelization technologies for sequential programs OpenMP and MPI are described. The main hardware and software factors affecting the parallel programs execution speed on computing cluster nodes are presented. The research of the impact on performance of the ratio of the computing and exchange operations number in the program is the main focus of the work. To implement the research, test parallel OpenMP and MPI programs were developed, in which the total number of operations and the ratio between computational and exchange operations are set. A high performance computing cluster consisting of several nodes was used as a hardware and software platform. The experimental studies have confirmed the hypothesis about more efficient operation of the OpenMP program in comparison with MPI on one node of the computing cluster for programs of a certain class, characterized by a significant share of exchange operations. The efficiency of a parallel program hybrid model in multi-node systems with heterogeneous memory is shown using OpenMP in shared memory subsystem, and MPI in a distributed memory subsystem.