Parallel Programming System Research Articles

Towards Automatic Deductive Verification of C Programs with Sisal Loops Using the C-lightVer System

The C-lightVer system is developed in IIS SB RAS for C-program deductive verification. C-kernel is an intermediate verification language in this system. Cloud parallel programming system (CPPS) is also developed in IIS SB RAS. Cloud Sisal is an input language of CPPS. The main feature of CPPS is implicit parallel execution based on automatic parallelization of Cloud Sisal loops. Cloud-Sisal-kernel is an intermediate verification language in the CPPS system. Our goal is automatic parallelization of such a superset of C that allows implementing automatic verification. Our solution is such a superset of C-kernel as C-Sisal-kernel. The first result presented in this paper is an extension of C-kernel by Cloud-Sisal-kernel loops. We have obtained the C-Sisal-kernel language. The second result is an extension of C-kernel axiomatic semantics by inference rule for Cloud-Sisal-kernel loops. The paper also presents our approach to the problem of deductive verification automation in the case of finite iterations over data structures. This kind of loops is referred to as definite iterations. Our solution is a composition of symbolic method of verification of definite iterations, verification condition metageneration and mixed axiomatic semantics method. Symbolic method of verification of definite iterations allows defining inference rules for these loops without invariants. Symbolic replacement of definite iterations by recursive functions is the base of this method. Obtained verification conditions with applications of recursive functions correspond to logical base of ACL2 prover. We use ACL2 system based on computable recursive functions. Verification condition metageneration allows simplifying implementation of new inference rules in a verification system. The use of mixed axiomatic semantics results to simpler verification conditions in some cases.

Support tools for functional programming distance learning and teaching

The cloud parallel programming system CPPS being under development at the A.P. Ershov Institute of Informatics Systems SB RAS is based on the Cloud Sisal language which carries on the traditions of previous versions of the Sisal language while remaining a functional data-flow language focused on writing large scientific programs and expanding their capabilities by supporting cloud computing. In this paper, the online environment of the CPPS system which allows a user on any device with Internet access to develop and execute functional programs in the Cloud Sisal language is considered.

Taskflow: A General-Purpose Parallel and Heterogeneous Task Programming System

Taskflow tackles the long-standing question: How can we make it easier for developers to program parallel and heterogeneous computer-aided design (CAD) applications with high performance and simultaneous high productivity? Taskflow introduces a new powerful task graph programming model to assist developers in the implementation of parallel and heterogeneous algorithms with complex control flow. We develop an efficient system runtime to solve many of the new scheduling challenges arising out of our models and optimize the performance across latency, energy efficiency, and throughput. Taskflow has demonstrated promising performance on both micro-benchmarks and real-world applications. As an example, Taskflow solved a large-scale circuit placement problem up to 17% faster, with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.3\times $ </tex-math></inline-formula> fewer memory, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.1\times $ </tex-math></inline-formula> less power consumption, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.9\times $ </tex-math></inline-formula> higher throughput than two industrial-strength systems, oneTBB and StarPU, on a machine of 40 CPUs and 4 GPUs.

Cpp-Taskflow: A General-Purpose Parallel Task Programming System at Scale

This article introduces Cpp-Taskflow, a high-performance parallel task programming system, to streamline the building of large and complex parallel applications. Cpp-Taskflow leverages the power of modern C++ and task-based approaches to enable efficient implementations of parallel decomposition strategies. Our programming model can quickly handle not only traditional loop-level parallelism but also irregular patterns, such as graph algorithms and dynamic control flows. Compared with existing libraries, Cpp-Taskflow is more cost efficient in performance scaling and software integration. We have evaluated Cpp-Taskflow on both micro-benchmarks and large-scale design automation problems of million-scale tasking. In a particular timing analysis workload, Cpp-Taskflow outperformed OpenMP by 2× faster using 2× fewer lines of code. We have also shown Cpp-Taskflow achieved up to 47.81% speed-up with 28.5% less code over the industrial-strength library, Intel Threading Building Blocks, on a detailed placement problem.

Formal verification of Cloud Sisal programs

A cloud parallel programming system CPPS which is under development at the A.P. Ershov Institute of Informatics Systems is aimed to support the development, verification and debugging of Cloud Sisal programs and their correct conversion into the efficient code of parallel computing systems for its execution in clouds. In this paper, the methods and subsystems of CPPS intended for formal verification of Cloud Sisal programs are described.

QMPI: A next generation MPI profiling interface for modern HPC platforms

As modern HPC applications and systems advance to exascale, their complexity and the need for more efficient resource utilization increases. This fact demands more advanced monitoring, analysis and optimization approaches. Therefore, the Message Passing Interface (MPI), which is the most common parallel programming system for HPC applications, must enable these advanced approaches. Even if the existing MPI Profiling Interface, PMPI, provides comprehensive tool support, it is no longer sufficient to enable these advanced approaches. In particular, PMPI does not support simultaneous or collaborative monitoring solutions from multiple different agents or sources. In this paper we introduce our interface QMPI, which addresses these limitations of PMPI and aims to be a successor to it. This paper presents the use cases and requirements that necessitate the development of QMPI, as well as a design offer for a QMPI prototype followed by its implementation and evaluation.

An OpenMP-Based Algorithm for Multi-Nodes Computational of Super-Resolution

A new common OpenMP based parallel programming method MPMC (multi-node paralleling model base on multiprocessor devices) is proposed and implemented for data separation based to accelerate Super-Resolution (SR) task. PanguOS, a common parallel programming system designed with MPMC, is deployed with Secure Shell (SSH) to control devices and Secure Copy (SCP) to transmit the data stream on Ubuntu 16.04, and it has a good performance for SR task with remote sensing images. Experiments with images from geostationary-orbit earth observing satellite GaoFen(GF)-4, the method proposed can achieve almost 2.95 times acceleration at PanguOS, deployed with 3 Jetson TX2s, than a single Jetson TX2.

Visualization Methods and Algorithms for Graph Representation of Functional Programs

This paper is devoted to the visualization of intermediate representations (the so-called IR graphs) of programs in the Cloud Sisal functional language for the cloud parallel programming system (CPPS). The IR graph represents the data flow in the Cloud Sisal program; it is a hierarchical graph whose vertices correspond to simple and composite expressions and are connected by edges through the vertex ports corresponding to the operands of the expressions (arguments and results). An efficient algorithm for two-dimensional IR graph visualization is proposed. We also describe an efficient implementation of a drawing algorithm for arbitrary attributed hierarchical port graphs in the Visual Graph system. Some methods for IR graph visualization and dynamic modification as applied to Cloud Sisal program debugging are considered.

Foreword to the special issue for the Workshop on Parallel Programming Models and Systems Software for High-End Computing (P2S2 2017)

Foreword to the special issue for the Workshop on Parallel Programming Models and Systems Software for High-End Computing (P2S2 2017)

Angara interconnect makes GPU-based Desmos supercomputer an efficient tool for molecular dynamics calculations

In this article, we describe the Desmos supercomputer that consists of 32 hybrid nodes connected by a low-latency high-bandwidth Angara interconnect with torus topology. This supercomputer is aimed at cost-effective classical molecular dynamics calculations. Desmos serves as a test bed for the Angara interconnect that supports 3-D and 4-D torus network topologies and verifies its ability to unite massively parallel programming systems speeding-up effectively message-passing interface (MPI)-based applications. We describe the Angara interconnect presenting typical MPI benchmarks. Desmos benchmarks results for GROMACS, LAMMPS, VASP and CP2K are compared with the data for other high-performance computing (HPC) systems. Also, we consider the job scheduling statistics for several months of Desmos deployment.

Решение прикладных задач в системе параллельного программирования Аспект

Решение прикладных задач в системе параллельного программирования Аспект

Flexible network management and application service adaptability in software defined wireless sensor networks

The need for highly responsive and adaptable computing systems is essential in today’s network computing age. This is principally due to the drastic evolution in broad computing platforms operating at highly descriptive and abstracted mediums such as; reconfigurable computing systems, smart automation systems, cognitive and parallel programming systems which communicate using very complex resources or modes. Hence, such systems must incorporate the best forms of technologies to cater for the rapidly growing and heterogeneously connected platforms such as with Internet of Things (IoT). However, to effectively manage these network platforms with such high-end computing resources, requires a well-structured and carefully implemented systems. This work implements a Software Defined Wireless Sensor Network (SDWSN) approach coupled with Discrete Event Simulation (DES) and a highly extensible and scalable Software Defined Networking (SDN) controller–OpenDayLight (ODL), to implement a software-oriented network environment to increase network service adaptability and simplify network management. The implemented approach uses the ODL’s Model-Driven Service Abstraction Layer (MD-SAL) to facilitate the forwarding layer by applying state procedures to manage flow rules and introduce software-oriented network services. Experimental results indicate that in this approach, the traffic flow routing is significantly improved, with reduced transmission delays and that the underlying sensor nodes uses less energy since energy demanding tasks are performed on the controller.

MatrixMap: Programming abstraction and implementation of matrix computation for big data analytics

The computation core of many big data applications can be expressed as general matrix computations, including linear algebra operations and irregular matrix operations. However, existing parallel programming systems such as Spark do not have programming abstraction and efficient implementation for general matrix computations. In this paper, we present MatrixMap, a unified and efficient data-parallel programming framework for general matrix computations. MatrixMap provides powerful yet simple abstraction, consisting of a distributed in-memory data structure called bulk key matrix and a programming interface defined by matrix patterns. Users can easily load data into bulk key matrices and program algorithms into parallel matrix patterns. MatrixMap outperforms current state-of-the-art systems by employing three key techniques:matrix patterns with lambda functions for irregular and linear algebra matrix operations, asynchronous computation pipeline with context-aware data shuffling strategies for specific matrix patterns and in-memory data structure reusing data in iterations. Moreover, it can automatically handle the parallelization and distribute execution of programs on a large cluster. The experiment results show that MatrixMap is 12 times faster than Spark.

Метастохастические адаптивные алгоритмы и их реализация в супервычислительной среде T++&amp;MPI

Метастохастические адаптивные алгоритмы и их реализация в супервычислительной среде T++&amp;MPI

A Survey of Performance Analysis Tools for OpenMP and MPI

Objective: Due to the recent development of multiple parallel programming tools with varying features, it is difficult to choose the best tool according to the needs of the user. Methods: This problem is addressed by making a comparative analysis study of different features like license type, source code availability, targeted platforms and languages supportedby these diverse tools. There are different parallel programming languages that support the present multi-core architecture like Message Passing Interface (MPI) and Open Multi-Processing (OpenMP). These are widely used to provide different performance characteristics of parallelism in different test cases. The new architecture and the complexity strengthens the need to monitor and analyze the performance of the various OpenMP kernels and constructs on multi-core processors. Findings: There are many papers that have been published in the past but non of them focuses on a comparative study among the performance analysis tools (PATs) that we mostly opt for. This paper intends to analyze the parallel computing ability of OpenMP and MPI, besides helping the user to understand which tool suites his task the best. Improvement: This study shows that MPI offers the best performance characteristics in the field of shared memory programming whereas OpenMP is a better choice because of the global style of the resulting program. It also provides a roadmap to select the best tool when designing a parallel programming system.

Special Issue on Parallel Programming Models and Systems Software for High-End Computing

Abstract This special issue features a collection of papers that extend the literature in unique ways, improving the state of art of programming models and systems software for high-end computing systems.

Решение задачи Дирихле для уравнения Пуассона методом Гаусса-Зейделя на языке параллельного программирования Т++

Решение задачи Дирихле для уравнения Пуассона методом Гаусса-Зейделя на языке параллельного программирования Т++

Protocol-based verification of message-passing parallel programs

We present ParTypes, a type-based methodology for the verification of Message Passing Interface (MPI) programs written in the C programming language. The aim is to statically verify programs against protocol specifications, enforcing properties such as fidelity and absence of deadlocks. We develop a protocol language based on a dependent type system for message-passing parallel programs, which includes various communication operators, such as point-to-point messages, broadcast, reduce, array scatter and gather. For the verification of a program against a given protocol, the protocol is first translated into a representation read by VCC, a software verifier for C. We successfully verified several MPI programs in a running time that is independent of the number of processes or other input parameters. This contrasts with alternative techniques, notably model checking and runtime verification, that suffer from the state-explosion problem or that otherwise depend on parameters to the program itself. We experimentally evaluated our approach against state-of-the-art tools for MPI to conclude that our approach offers a scalable solution.

A convergence of key‐value storage systems from clouds to supercomputers

SummaryThis paper presents a convergence of distributed key‐value storage systems in clouds and supercomputers. It specifically presents ZHT, a zero‐hop distributed key‐value store system, which has been tuned for the requirements of high‐end computing systems. ZHT aims to be a building block for future distributed systems, such as parallel and distributed file systems, distributed job management systems, and parallel programming systems. ZHT has some important properties, such as being lightweight, dynamically allowing nodes join and leave, fault tolerant through replication, persistent, scalable, and supporting unconventional operations such as append, compare and swap, callback in addition to the traditional insert/lookup/remove. We have evaluated ZHT's performance under a variety of systems, ranging from a Linux cluster with 64 nodes, an Amazon EC2 virtual cluster up to 96 nodes, to an IBM Blue Gene/P supercomputer with 8K nodes. We compared ZHT against other key‐value stores and found it offers superior performance for the features and portability it supports. This paper also presents several real systems that have adopted ZHT, namely, FusionFS (a distributed file system), IStore (a storage system with erasure coding), MATRIX (distributed scheduling), Slurm++ (distributed HPC job launch), Fabriq (distributed message queue management); all of these real systems have been simplified because of key‐value storage systems and have been shown to outperform other leading systems by orders of magnitude in some cases. It is important to highlight that some of these systems are rooted in HPC systems from supercomputers, while others are rooted in clouds and ad hoc distributed systems; through our work, we have shown how versatile key‐value storage systems can be in such a variety of environments. Copyright © 2015 John Wiley &amp; Sons, Ltd.

Generalized Scalable Multiple Copy Algorithms for Biological Molecular Dynamics Simulations in NAMD

Computational methodologies that couple the dynamical evolution of a collection of copies, or replicas, of a system of interest offer powerful and flexible approaches to characterize complex molecular processes. Such multiple copy algorithms (MCAs) can be used to enhance sampling, compute reversible work and free energies, as well as refine transition pathways. Widely used examples of MCAs include temperature and Hamiltonian-tempering replica-exchange molecular dynamics (T-REMD and H-REMD), alchemical free energy perturbation with lambda replica exchange (FEP/-REMD), umbrella sampling with Hamiltonian replica exchange (US/H-REMD), and string method with swarms-of-trajectories to determine conformational transition pathways. Here, we report a robust and general implementation of MCAs for molecular dynamics (MD) simulations in the highly scalable program NAMD built upon the parallel programming system charm++. Multiple concurrent NAMD instances are launched with internal partitions of charm++ and located continuously within a single communication world. Messages between NAMD instances are passed by low-level point-to-point communication functions, which are accessible through NAMD's Tcl scripting interface. The communication-enabled Tcl scripting provides a sustainable application interface for end users to realize generalized MCAs without modifying source code. Illustrative applications of MCAs with fine-grained inter-copy communication structure, including global lambda exchange in FEP/-REMD, window swapping US/H-REMD in multidimensional order parameter space, and string method with swarms-of-trajectories were carried out on IBM Blue Gene/Q to demonstrate the versatility and massive scalability of the present implementation.