Message Passing Interface Programs Research Articles

Test data generation for covering mutation-based path using MGA for MPI program

Message Passing Interface (MPI) is a communication protocol used for parallel programming in various languages, valued for its reliability and broad applicability. Mutation testing is a software testing method for systematically simulating software faults. However, the significant number of inserted mutation branches in the program escalates the testing cost. To address this issue, we propose the comprehensive PMTDGM framework. The framework first generates mutation-based paths based on the relevancy of mutant branches and the difficulty of mutant branch coverage, followed by the establishment of a multitask model of path coverage. Finally, we employ a multi-population genetic algorithm (MGA) to generate test data. Our experiments, performed on six MPI programs of varying sizes and structures, demonstrate that the mutation-based paths have the small test sets and are easy to cover. Additionally, the multitask model and MGA can significantly improve the efficiency of generating test data and reduce the cost of mutation testing for MPI programs compared to traditional methods.

Predicting Software Defects in Hybrid MPI and OpenMP Parallel Programs Using Machine Learning

High-performance computing (HPC) and its supercomputers are essential for solving the most difficult issues in many scientific computing domains. The proliferation of computational resources utilized by HPC systems has resulted in an increase in the associated error rates. As such, modern HPC systems promote a hybrid programming style that integrates the message-passing interface (MPI) and open multi-processing (OpenMP). However, this integration often leads to complex defects, such as deadlocks and race conditions, that are challenging to detect and resolve. This paper presents a novel approach: using machine learning algorithms to predict defects in C++-based systems by employing hybrid MPI and OpenMP models. We focus on employing a balanced dataset to enhance prediction accuracy and reliability. Our study highlights the effectiveness of the support vector machine (SVM) classifier, enhanced with term frequency (TF) and recursive feature elimination (RFE) techniques, which demonstrates superior accuracy and performance in defect prediction when compared to other classifiers. This research contributes significantly to the field by providing a robust method for early defect detection in hybrid programming environments, thereby reducing development time, costs and improving the overall reliability of HPC systems.

Program partitioning and deadlock analysis for MPI based on logical clocks

The message passing interface (MPI) has become a standard for programming models in the field of high performance computing. It is of great importance to ensure the reliability of MPI programs by detecting whether there exist errors in them. However, as one of the most common errors in MPI programs, deadlock is difficult to detect due to the non-determinism and the asynchronous communication supported by MPI. Existing approaches mainly focus on detecting deadlocks by traversing all possible execution paths in an MPI program. But in this way the detection efficiency is always limited since the number of execution paths increases exponentially with the number of wildcard receives and processes in the program.In order to alleviate the path explosion problem for single-path MPI programs, we propose a program partitioning approach based on logical clocks to detecting deadlocks. In the approach, the program is first divided into several preliminary partitions based on the matching detection rule. Then to obtain the dependency relationships of partitions, the Binary Lazy Clocks algorithm is raised to mark clocks for communication operations. Based on the clocks, the completion orders of communication operations in each process of the program are tracked. Further, we get the dependency relationships of the preliminary partitions by analyzing these completion orders and merge the preliminary partitions with the dependency relationships for generating the final partitions. Finally, deadlocks are detected by traversing all possible execution paths of each final partition. We have implemented our method in a tool called PDMPI and performed experimental evaluation on 14 programs. The experimental results indicate that PDMPI is more effective for detecting deadlocks in MPI programs than two most related tools ISP and SAMPI, especially in programs with numerous interleavings.

Finding Bottlenecks in Message Passing Interface Programs by Scalable Critical Path Analysis

Bottlenecks and imbalance in parallel programs can significantly affect performance of parallel execution. Finding these bottlenecks is a key issue in performance analysis of MPI programs especially on a large scale. One of the ways to discover bottlenecks is to analyze the critical path of the parallel program: the longest execution path in the program activity graph. There are a number of methods of finding the critical path; however, most of them suffer a performance drop when scaled. In this paper, we analyze several methods of critical path finding based on classical Dijkstra and Delta-stepping algorithms along with the proposed algorithm based on topological sorting. Corresponding algorithms for each approach are presented including additional enhancements for increasing performance. The implementation of the algorithms and resulting performance for several benchmark applications (NAS Parallel Benchmarks, CP2K, OpenFOAM, LAMMPS, and MiniFE) are analyzed and discussed.

A parallel numerical algorithm by combining MPI and OpenMP programming models with applications in gravity field recovery

Satellite gravimetry missions have enabled the calculation of high-accuracy and high-resolution Earth gravity field models from satellite-to-satellite tracking data and gravitational gradients. However, calculating high maximum degree/order (e.g., 240 or even higher) gravity field models using the least squares method is time-consuming due to the vast amount of gravimetry observations. To improve calculation efficiency, a parallel algorithm has been developed by combining Message Passing Interface (MPI) and Open Multi-Processing (OpenMP) programming models to calculate and invert normal equations for the Earth gravity field recovery. The symmetrical feature of normal equations has been implemented to speed up the calculation progress and reduce computation time. For example, the computation time to generate the normal equation of an IGGT_R1 test version of degree/order 240 was reduced from 88 h to 27 h by considering the symmetrical feature. Here, the calculation was based on the high-performance computing cluster with 108 cores in the School of Geodesy and Geomatics, at Wuhan University. Additionally, the MPI parallel Gaussian-Jordan elimination method was modified to invert normal equation matrices and scaled up to 100 processor cores in this study while the traditional method was limited in a certain number of processors. Furthermore, the Cholesky decomposition from the ScaLAPACK library was used to compare with the parallel Gauss-Jordan elimination method. The numerical algorithm has effectively reduced the amount of calculation and sped up the calculation progress, and has been successfully implemented in applications such as building the gravity field models IGGT_R1 and IGGT_R1C.

Integrating DSGEO into test case generation for path coverage of MPI programs

When testing a Message-Passing Interface (MPI) program composed of multiple processes, the testing cost for each process is different, and those expensive processes restrict the testing efficiency of the entire MPI program.

An international survey on MPI users

The Message Passing Interface (MPI) plays a crucial part in the parallel computing ecosystem, a driving force behind many of the high-performance computing (HPC) successes. To maintain its relevance to the user community—and in particular to the growing HPC community at large—the MPI standard needs to identify and understand the MPI users’ concerns and expectations, and adapt accordingly to continue to efficiently bridge the gap between users and hardware. This questionnaire survey was conducted using two online questionnaire frameworks and has gathered more than 850 answers from 42 countries since February 2019. Some of preceding surveys of MPI uses are questionnaire surveys like ours, while others are conducted either by analyzing MPI programs to reveal static behavior or by using profiling tools to analyze the dynamic runtime behavior of MPI jobs. Our survey is different from other questionnaire surveys in terms of its larger number of participants and wide geographic spread. As a result, it is possible to illustrate the current status of MPI users more accurately and with a wider geographical distribution. In this report, we will show some interesting findings, compare the results with preceding studies when possible, and provide some recommendations for MPI Forum based on the findings.

Implementation of the LAMMPS package using T-system with an Open Architecture

Supercomputer applications are usually implemented in the C, C++, and Fortran programming languages using different versions of the Message Passing Interface library. The "T-system" project (OpenTS) studies the issues of automatic dynamic parallelization of programs. In practical terms, the implementation of applications in a mixed (hybrid) style is relevant, when one part of the application is written in the paradigm of automatic dynamic parallelization of programs and does not use any primitives of the MPI library, and the other part of it is written using the Message Passing Interface library. In this case, the library is used, which is a part of the T-system and is called DMPI (Dynamic Message Passing Interface). In this way, it is necessary to evaluate the effectiveness of the MPI implementation available in the T-system. The purpose of this work is to examine the effectiveness of DMPI implementation in the T-system. In a classic MPI application, 0% of the code is implemented using automatic dynamic parallelization of programs and 100% of the code is implemented in the form of a regular Message Passing Interface program. For comparative analysis, at the beginning the code is executed on the standard Message Passing Interface, for which it was originally written, and then it is executed using the DMPI library taken from the developed T-system. Сomparing the effectiveness of the approaches, the performance losses and the prospects for using a hybrid programming style are evaluated. As a result of the conducted experimental studies for different types of computational problems, it was possible to make sure that the efficiency losses are negligible. This allowed to formulate the direction of further work on the T-system and the most promising options for building hybrid applications. Thus, this article presents the results of the comparative tests of LAMMPS application using OpenMPI and using OpenTS DMPI. The test results confirm the effectiveness of the DMPI implementation in the OpenTS parallel programming environment

Test Data Generation for Path Coverage of MPI Programs Using SAEO

Message-passing interface (MPI) programs, a typical kind of parallel programs, have been commonly used in various applications. However, it generally takes exhaustive computation to run these programs when generating test data to test them. In this article, we propose a method of test data generation for path coverage of MPI programs using surrogate-assisted evolutionary optimization, which can efficiently generate test data with high quality. We first divide a sample set of a program into a number of clusters according to the multi-mode characteristic of the coverage problem, with each cluster training a surrogate model. Then, we estimate the fitness of each individual using one or more surrogate models when generating test data through evolving a population. Finally, a small number of representative individuals are selected to execute the program, with the purpose of obtaining their real fitness, to guide the subsequent evolution of the population. We apply the proposed method to seven benchmark MPI programs and compare it with several state-of-the-art approaches. The experimental results show that the proposed method can generate test data with reduced computation, thus improving the testing efficiency.

Integrating an Ensemble Surrogate Model’s Estimation into Test Data Generation

For the path coverage testing of a Message-Passing Interface (MPI) program, test data generation based on an evolutionary optimization algorithm (EOA) has been widely known. However, during the use of the above technique, it is necessary to evaluate the fitness of each evolutionary individual by executing the program, which is generally computationally expensive. In order to reduce the computational cost, this article proposes a method of integrating an ensemble surrogate model’s estimation into the process of generating test data. The proposed method first produces a number of test inputs using an EOA, and forms a training set together with their real fitness. Then, this article trains an ensemble surrogate model (ESM) based on the training set, which is employed to estimate the fitness of each individual. Finally, a small number of individuals with good estimations are selected to further execute the program, so as to have their real fitness for the subsequent evolution. This article applies the proposed method to seven benchmark MPI programs, which is compared with several state-of-the-art approaches. The experimental results show that the proposed method can generate test data with significantly low computational cost.

Fault tolerance of MPI applications in exascale systems: The ULFM solution

The growth in the number of computational resources used by high-performance computing (HPC) systems leads to an increase in failure rates. Fault-tolerant techniques will become essential for long-running applications executing in future exascale systems, not only to ensure the completion of their execution in these systems but also to improve their energy consumption. Although the Message Passing Interface (MPI) is the most popular programming model for distributed-memory HPC systems, as of now, it does not provide any fault-tolerant construct for users to handle failures. Thus, the recovery procedure is postponed until the application is aborted and re-spawned. The proposal of the User Level Failure Mitigation (ULFM) interface in the MPI forum provides new opportunities in this field, enabling the implementation of resilient MPI applications, system runtimes, and programming language constructs able to detect and react to failures without aborting their execution. This paper presents a global overview of the resilience interfaces provided by the ULFM specification, covers archetypal usage patterns and building blocks, and surveys the wide variety of application-driven solutions that have exploited them in recent years. The large and varied number of approaches in the literature proves that ULFM provides the necessary flexibility to implement efficient fault-tolerant MPI applications. All the proposed solutions are based on application-driven recovery mechanisms, which allows reducing the overhead and obtaining the required level of efficiency needed in the future exascale platforms.

Finite Difference Generated Transient Potentials of Open-Layered Media by Parallel Computing Using OpenMP, MPI, OpenACC, and CUDA

The implementation of time-domain Green’s functions (TDGFs) in the graphics processing unit (GPU) and the central processing unit (CPU) using a finite-difference scheme is shown. The TDGFs represent the transient electric scalar and magnetic vector potentials due to a horizontal electric dipole (HED) in open-layered media. The layered media is bounded with a perfectly matched layer (PML), symmetry axis, and perfect electric conductor (PEC). We adopted four different parallel approaches as follows: 1) open multiprocessing (OpenMP) CPU implementation; 2) message passing interface (MPI) CPU implementation; 3) open accelerators (OpenACC) GPU implementation; and 4) compute unified device architecture (CUDA) GPU implementation. The accuracy and efficiency of the utilized programming models are validated by comparing and verifying the obtained results using a sequential CPU implementation. Compared to single threaded CPU implementation, speed-ups obtained by the OpenMP, MPI, OpenACC, and CUDA programming models are $4.8\times $ , $6.12\times $ , $45.97\times $ , and $96.53\times $ higher, respectively. The final result shows that GPU implementation leads to a considerable speed-up while the solution’s accuracy is fixed.

A three-phase workflow for general and expressive representations of nondeterminism in HPC applications

Nondeterminism is an increasingly entrenched property of high-performance computing (HPC) applications and has recently been shown to seriously hamper debugging and reproducibility efforts. Tools for addressing the nondeterministic debugging problem have emerged, but they do not provide methods for systematically cataloging the nondeterminism in a given application. We propose a three-phase workflow for representing executions of nondeterministic message passing interface programs as event graphs, quantifying their structural similarity with graph kernels, and applying machine learning techniques to investigate shared properties across applications. We present an empirical study comparing two graph kernels’ suitability for this task and propose future uses of the methodology.

Scheduling sequence selection for generating test data to cover paths of MPI programs

Abstract Context: As one of key tasks in software testing, test data generation has been receiving widespread attention in recent years. Message-passing Interface (MPI) programs, which are one representative type of parallel programs, have the characteristic of non-determinism, which is reflected by the non-deterministic execution under different scheduling sequences against the same program input. Previous studies have shown that different difficulties are raised in generating test data under different scheduling sequences, suggesting that selecting appropriate scheduling sequences is beneficial to a high efficiency. Objective: We propose a method of selecting a superior and feasible scheduling sequence for generating test data in the criterion of path coverage against each target path of an MPI program. Method: In the proposed method, a number of program inputs are first sampled by Latin hypercube sampling. Then, the program is executed against each sample under each scheduling sequence, and all the scheduling sequences are sorted according to the similarities between the paths traversed by these samples and the target one. Finally, the feasibility of a scheduling sequence with the best quality is investigated based on the symbolic execution. Results: We apply the proposed method to seven typical MPI programs and compare it with the random one. The experimental results show that test data covering the target path can be generated under the selected scheduling sequence with high success rate and low time consumption. Conclusion: The proposed method takes the influence of scheduling sequences on generating test data into consideration, thus providing a competent way to test parallel programs.

Data Flow Analysis of MPI Program Using Dynamic Analysis Technique with Partial Execution

Message Passing Interface (MPI) is a dominant parallel programming paradigm. MPI processes communicate with each other by sending or receiving messages through communication functions. The application's communication latency will be less if processes are scheduled on nearest cores or nodes and communication latency will be more if processes are scheduled on farthest cores or nodes.The communication latency can be reduced by using topology-aware process placement technique. In this technique, MPI processes are placed on the nearest cores if they have more communication between them. To find the communication pattern between processes, analysis of MPI program is required. Various techniques like static, symbolic and dynamic analysis are available for finding communication pattern of MPI program. These techniques are either taking more time for analysis or fail to find correct communication pattern. In this paper, we have proposed DAPE (Dynamic Analysis with Partial Execution) technique for analysis of MPI program, which finds correct communication pattern in less time as compared to existing techniques. The experimental results show that the proposed technique outperforms over the existing techniques.

Precise Predictive Analysis for Discovering Communication Deadlocks in MPI Programs

The Message Passing Interface (MPI) is the standard API for parallelization in high-performance and scientific computing. Communication deadlocks are a frequent problem in MPI programs, and this article addresses the problem of discovering such deadlocks. We begin by showing that if an MPI program is single path, the problem of discovering communication deadlocks is NP-complete. We then present a novel propositional encoding scheme that captures the existence of communication deadlocks. The encoding is based on modeling executions with partial orders and implemented in a tool called MOPPER . The tool executes an MPI program, collects the trace, builds a formula from the trace using the propositional encoding scheme, and checks its satisfiability. Finally, we present experimental results that quantify the benefit of the approach in comparison to other analyzers and demonstrate that it offers a scalable solution for single-path programs.

Inferring Types for Parallel Programs

The Message Passing Interface (MPI) framework is widely used in implementing imperative pro- grams that exhibit a high degree of parallelism. The PARTYPES approach proposes a behavioural type discipline for MPI-like programs in which a type describes the communication protocol followed by the entire program. Well-typed programs are guaranteed to be exempt from deadlocks. In this paper we describe a type inference algorithm for a subset of the original system; the algorithm allows to statically extract a type for an MPI program from its source code.

A virtual memory manager optimized for node-level cooperative multi-tasking in memory constrained systems

There is a growing need to perform large computations on small systems, as access to large systems is not widely available and cannot keep up with the size of the data that needs to be processed. Recently, a runtime system for programs using a library that implements the Message Passing Interface (MPI), called Big Data MPI (BDMPI), that allows MPI programs whose aggregate amount of memory exceeds the physical amount of memory to be executed efficiently by utilizing node-level cooperative multi-tasking. In this paper we present a virtual memory subsystem which we implemented as part of the BDMPI runtime. Our new virtual memory subsystem, which we call SBMA takes advantage of BDMPI’s node-level cooperative multi-tasking in order to intelligently determine the parts of the virtual address space that need to be loaded to and unloaded from the main memory. Benchmarking using a synthetic application shows that for the use cases relevant to BDMPI, the overhead incurred by the memory protection constructs necessary for the BDMPI-SBMA system is amortized such that it performs as fast as explicit data movement by the application developer. Furthermore, testing SBMA with five different classes of applications showed that with no modification to the original MPI program, speedups from 2×–12× over a standard BDMPI implementation can be achieved for the included applications.

Evaluating and extending user-level fault tolerance in MPI applications

The user-level failure mitigation (ULFM) interface has been proposed to provide fault-tolerant semantics in the Message Passing Interface (MPI). Previous work presented performance evaluations of ULFM; yet questions related to its programability and applicability, especially to non-trivial, bulk synchronous applications, remain unanswered. In this article, we present our experiences on using ULFM in a case study with a large, highly scalable, bulk synchronous molecular dynamics application to shed light on the advantages and difficulties of this interface to program fault-tolerant MPI applications. We found that, although ULFM is suitable for master–worker applications, it provides few benefits for more common bulk synchronous MPI applications. To address these limitations, we introduce a new, simpler fault-tolerant interface for complex, bulk synchronous MPI programs with better applicability and support than ULFM for application-level recovery mechanisms, such as global rollback.

Implementation of a three dimensional three-phase fluid flow (“oil–water–gas”) numerical model in LuNA fragmented programming system

To overcome the difficulties of efficient scalable numerical algorithms implementation on multicomputers with a large number of computing nodes, LuNA system is being developed. LuNA automatically generates a code related to communications, resources management and dynamic workload balancing, thus simplifying construction of parallel programs. To examine the efficiency of numerical model implementation, created with LuNA, a real application of a three-phase (oil, water and gas) fluid filtration simulation is studied. The application algorithm is parallelized and implemented using LuNA and conventional approach using MPI (Message Passing Interface). A comparative performance testing of the two implementations is done. The results show that LuNA implementation is easier to develop, but its efficiency is significantly lower than the efficiency of MPI program, but manual tuning of the program execution makes its efficiency comparable to that of MPI program.