High-performance Parallel Programs Research Articles

Development of software for neuromarketing based on artificial intelligence and data science using high-performance computing and parallel programming technologies

This paper deals with the scientific research in a new area of human activity – neuromarketing. The development of new software for neuromarketing research based on modern methods and tools of artificial intelligence and data science (neural network technologies) using technologies of high-performance computing and parallel programming are considered in this paper. The results of experimental measurements of electrical activity of the consumer's brain based on electroencephalographic signals depending on various types of marketing stimuli are presented in this paper. The quality of the developed software for neuromarketing is evaluated on the basis of various quality metrics.

Special Issue on Programming Models and Applications for Multicores and Manycores 2020

This special issue focuses on all aspects of parallel programming on multicore and manycore architectures such as programming models and systems, applications algorithms, performance analysis, and debugging and performance tools. It includes selected articles from the 2020 International Workshop on Programming Models and Applications for Multicores and Manycores (PMAM 2020), which was colocated with the PPoPP 2020 conference, in San Diego, California, February 22–26, 2020. From high-end servers to mobile phones, multicore and manycore systems are steadily entering every aspect of the information technology landscape. While this advanced technology enables new opportunities for scientific discovery, it also opens new challenges. In particular, although program developers have been trained in parallel programming, most existing parallel programs are still prone to errors such as data race and deadlock and suffer from suboptimal performance. In order to fully explore the computing power of multicore and manycore hardware architectures, innovations in parallel programming models as well as the associated development and execution eco-systems are urgently needed to enable delivery of error-free and high-performance parallel programs. The PMAM 2020 workshop provided a discussion forum for people interested in programming environments, models, tools, and applications specifically designed for parallel multicore and manycore hardware environments. The four outstanding contributions for this special issue were selected from eight workshop presentations. The accepted publications cover several emerging research directions, specifically focusing on programming models and runtime techniques on manycore and heterogeneous computing systems for graph and streaming applications, as summarized below. In “A Self-Adjusting Task Granularity Mechanism for the Java Lifeline-Based Global Load Balancer Library,” Finnerty et al. present a tuning mechanism that dynamically adjusts the sensitive task granularity in the load balancing scheme of X10 for Java parallel programs and demonstrate the improved performance of four applications on two manycore supercomputers. In “A Performance Predictor for Implementation Selection of Parallelized Static and Temporal Graph Algorithms,” Rehman et al. analyze the performance of graph workload with different algorithm and hardware conditions. They propose an inter-implementation predictor to choose the best performing parallel implementation for static and temporal graph benchmarks and inputs when executing on a CPU or a GPU architecture by leveraging analytical and neural network models. In “Sharing Non-Cache-Coherent Memory with Bounded Incoherence,” Ren et al. introduce a memory consistency approach, called bounded incoherence, that enables cached access to shared data-structures in non-cache-coherent memory. The proposed model ensures that updates to memory on one node are visible within at most a bounded amount of time on all other nodes. Test results show 30% performance improvement of the PowerGraph graph processing framework over state-of-the-art distributed approaches. In “Code Generation for Energy-Efficient Execution of Dynamic Streaming Task Graphs on Parallel and Heterogeneous Platforms,” Litzinger et al. enhance the programming model of streaming applications by introducing dynamic elements into the task graphs so that the runtime system can remap tasks and adapt the change of streaming-in task structures at runtime. The authors provide a prototype implementation of the toolchain and demonstrate low overhead and low energy consumption of the proposed remapping technique. We hope that the readers of the PMAM 2020 special issue will find the articles insightful in the field of parallel programming for multicore and manycore systems. The guest editors would like to thank all the reviewers and the Concurrency and Computation Practice and Experience Wiley office staff who worked hard to make this high-quality journal issue happen, while facing challenges from the COVID-19 pandemic. Data sharing is not applicable to this article as no new data were created or analyzed in this study.

PySchedCL: Leveraging Concurrency in Heterogeneous Data-Parallel Systems

In the past decade, high performance compute capabilities exhibited by heterogeneous GPGPU platforms have led to the popularity of data parallel programming languages such as CUDA and OpenCL. Developing high performance parallel programming solutions using such languages involve a steep learning curve due to the complexity of the underlying heterogeneous compute devices and their impact on performance. This has led to the emergence of several High Performance Computing frameworks which provide high-level abstractions for easing the development of data-parallel applications on heterogeneous platforms. However, the scheduling decisions undertaken by such frameworks only exploit coarse-grained concurrency in data parallel applications. In this paper, we propose PySchedCL, a framework which explores fine-grained concurrency aware scheduling decisions that harness the power of heterogeneous CPU/GPU architectures efficiently. We showcase the efficacy of such scheduling mechanisms over existing coarse-grained dynamic scheduling schemes by conducting extensive experimental evaluations for a diverse set of popular Deep Learning benchmarks.

Numerical modeling of the convective Kelvin-Helmholtz instabilities of astropauses

In this paper we present the numerical modeling results of the problem of the hypersonic stellar wind interaction with fully ionized interstellar medium that moves with respect to the star with supersonic speed. This is the classical problem that has been firstly studied in 1970 by Baranov et al. [1] under the thin layer approximation.In this paper we present results of numerical solutions obtained with high spatial resolution of the numerical grid. The computations were performed by using GPU processors with high-performance parallel programming technology CUDA from NVIDIA. It is shown that the stationary solution depends only on two dimensionless parameters - Mach number in the interstellar medium, M∞, and adiabatic index, γ. The stationary solutions were obtained for different values of M∞ for low and medium resolution of the numerical grids. For the numerical grids of high spatial resolution the Kelvin-Helmholtz instability appears at the tangential discontinuities (both primary and secondary) in the tail part of interaction region. The instability is convective as it should be according to the linear analyses by Ruderman et al. [6]. We explore the instability evolution with increasing resolution of the numerical grid. Effects of the numerical scheme are also studied.

TIP_finder: An HPC Software to Detect Transposable Element Insertion Polymorphisms in Large Genomic Datasets.

Transposable elements (TEs) are non-static genomic units capable of moving indistinctly from one chromosomal location to another. Their insertion polymorphisms may cause beneficial mutations, such as the creation of new gene function, or deleterious in eukaryotes, e.g., different types of cancer in humans. A particular type of TE called LTR-retrotransposons comprises almost 8% of the human genome. Among LTR retrotransposons, human endogenous retroviruses (HERVs) bear structural and functional similarities to retroviruses. Several tools allow the detection of transposon insertion polymorphisms (TIPs) but fail to efficiently analyze large genomes or large datasets. Here, we developed a computational tool, named TIP_finder, able to detect mobile element insertions in very large genomes, through high-performance computing (HPC) and parallel programming, using the inference of discordant read pair analysis. TIP_finder inputs are (i) short pair reads such as those obtained by Illumina, (ii) a chromosome-level reference genome sequence, and (iii) a database of consensus TE sequences. The HPC strategy we propose adds scalability and provides a useful tool to analyze huge genomic datasets in a decent running time. TIP_finder accelerates the detection of transposon insertion polymorphisms (TIPs) by up to 55 times in breast cancer datasets and 46 times in cancer-free datasets compared to the fastest available algorithms. TIP_finder applies a validated strategy to find TIPs, accelerates the process through HPC, and addresses the issues of runtime for large-scale analyses in the post-genomic era.

Easy PRAM-Based High-Performance Parallel Programming with ICE

Parallel machines have become more widely used. Unfortunately parallel programming technologies have advanced at a much slower pace except for regular programs. For irregular programs, this advancement is inhibited by high synchronization costs, non-loop parallelism, non-array data structures, recursively expressed parallelism and parallelism that is too fine-grained to be exploitable. We present ICE, a new parallel programming language that is easy-to-program, since: (i) ICE is a synchronous, lock-step language so there is no need for programmer-specified synchronization; (ii) for a PRAM algorithm its ICE program amounts to directly transcribing it; and (iii) the PRAM algorithmic theory offers unique wealth of parallel algorithms and techniques. We propose ICE to be a part of an ecosystem consisting of the XMT architecture, the PRAM algorithmic model, and ICE itself, that together deliver on the twin goal of easy programming and efficient parallelization of irregular programs. The XMT architecture, developed at UMD, can exploit fine-grained parallelism in irregular programs. We have built the ICE compiler which translates the ICE language into the multithreaded XMTC language; the significance of this is that multi-threading is a feature shared by practically all current scalable parallel programming languages thus providing a method to compile ICE code. As one indication of ease of programming, we observed a reduction in code size in 11 out of 16 benchmarks as compared to hand-optimized XMTC. For these programs, the average reduction in number of lines of code was 35.5 percent. The remaining 5 benchmarks had almost the same code size for both ICE and hand-optimized XMTC. Our main result is perhaps surprising: The run-time was comparable to XMTC with a 0.53 percent average gain for ICE across all benchmarks.

Dynamic algorithm modeling application for comparing distributed computing frameworks

Scientific computing applies computational methods to solve problems in different scientific fields. It is associated with high performance computing and parallel programming fields as it typically requires large amount of computing resources. Frameworks based on distributed computing models such as MapReduce can simplify the utilization of such resources by providing near automatic parallelization and fault recovery, but they are not suitable for all types of algorithms. Comparing different distributed computing engines and choosing the most suitable one for a given algorithm can be a very time consuming task. We propose a methodology for deciding which distributed computing framework provides optimal parallel efficiency for a given scientific computing algorithm using a dynamic algorithm modeling application (DAMA) which simulates the parallel structure and behavior of algorithms. DAMA is implemented on a number of distributed computing frameworks and can be directly used as a benchmark to estimate the performance of the modeled algorithms by executing it in real computing clusters. The results can then be used to estimate which framework would give the best parallel performance for the given algorithm without having to implement the algorithm on any of the considered frameworks. This approach is applied on a number of scientific computing algorithms for validation and to illustrate its usability.

Hardware transactional memory: A high performance parallel programming model

The transactional memory in multicore processors has been a major research area over past several years. Many transactional memory systems have been proposed to be used to solve the synchronization problem of multicore processors. Hardware transactional memory is one of the critical methods to speedup communications in multicore environment. In this paper, we give a review of the current hardware transactional memory systems for multicore processors. We take a top-down approach to characterizing and classifying various hardware transactional design issues and present a taxonomy of hardware transactional memory systems which is consist of the five fundamental design issues: version management, conflict detection, contention management, virtualization and nesting. Finally, we discussed the active research challenge: the relationship between transactional memory and Input/Output operations and system calls.

Parallel Performance Wizard: A Performance System for the Analysis of Partitioned Global-Address-Space Applications

Given the complexity of high-performance parallel programs, developers often must rely on performance analysis tools to help them improve the performance of their applications. While many tools support analysis of message-passing programs, tool support is limited for applications written in programming models that present a partitioned global address space (PGAS) to the programmer such as UPC and SHMEM. Existing tools that support message-passing models are difficult to extend to support PGAS models due to differences between the two paradigms and the techniques used in their implementations. In this paper, we present our work on Parallel Performance Wizard (PPW), a performance analysis system for PGAS and MPI application analysis. We discuss new concepts, namely the generic-operation-type abstraction and GASP-enabled data collection, developed to facilitate support for multiple programming models and then give an overview of PPW’s automatic analysis and visualization capabilities. Finally, to show the usefulness of our system, we present results on PPW’s overhead, storage requirements and scalability before demonstrating its effectiveness via application case studies.

Costing stepwise refinements of parallel programs

The idea of the stepwise refinement of a problem specification into an efficient target program dates back to the 1970. With the high-level programming languages available today, it is becoming practical to make every node of the refinement tree executable (and not just the leaves of the tree). Consequently, one can also perform run time experiments with the intermediate programs (not just with the leaves). Our problem domain is scientific computing. We use experiments with the intermediate refinements, written in Haskell, to derive cost predictions for the target programs written in C with MPI. These predictions guide our choices of alternative refinements, as we navigate through the refinement tree. This non-standard use of a cost model introduces two new requirements: • The higher up in the refinement tree a programs is, the less determined its target implementation is, i.e., the less accurate the cost prediction will be. However, the cost data we obtain must (and can) suffice to make a correct choice between alternative refinements, based on a relative comparison of the respective performance predictions. • The costing of an intermediate program happens necessarily on an interpreter, not on the installation on which the target program will run—which is not yet fully determined! Still, the cost model must be calibrated with respect to the real installation—even when it is used for intermediate programs—since our choice of refinement will depend on the characteristics of the real installation. We propose a cost model for the stepwise prototyping of high-performance parallel programs, which satisfies these requirements.

Special Issue High Performance Parallel Functional Programming

Engineering high-performance parallel programs is hard: not only must a correct, efficient and inherently-parallel algorithm be developed, but the computations must be effectively and efficiently coordinated across multiple processors. It has long been recognised that ideas and approaches drawn from functional programming may be particularly applicable to parallel and distributed computing (e.g. Wegner 1971). There are several reasons for this suitability. Concurrent stateless computations are much easier to coordinate, high-level coordination abstractions reduce programming effort, and declarative notations are amenable to reasoning, i.e. to optimising transformations, derivation and performance analysis.

Synthesis of High-Performance Parallel Programs for a Class of ab Initio Quantum Chemistry Models

This paper provides an overview of a program synthesis system for a class of quantum chemistry computations. These computations are expressible as a set of tensor contractions and arise in electronic structure modeling. The input to the system is a a high-level specification of the computation, from which the system can synthesize high-performance parallel code tailored to the characteristics of the target architecture. Several components of the synthesis system are described, focusing on performance optimization issues that they address.

Middleware infrastructure for parallel and distributed programming models in heterogeneous systems

We introduce a middleware infrastructure that provides software services for developing and deploying high-performance parallel programming models and distributed applications on clusters and networked heterogeneous systems. This middleware infrastructure utilizes distributed agents residing on the participating machines and communicating with one another to perform the required functions. An intensive study of the parallel programming models in Java has helped identify the common requirements for a runtime support environment, which we used to define the middleware functionality. A Java-based prototype, based on this architecture, has been developed along with a Java object-passing interface (JOPI) class library. Since this system is written completely in Java, it is portable and allows executing programs in parallel across multiple heterogeneous platforms. With the middleware infrastructure, users need not deal with the mechanisms of deploying and loading user classes on the heterogeneous system. Moreover, details of scheduling, controlling, monitoring, and executing user jobs are hidden, while the management of system resources is made transparent to the user. Such uniform services are essential for facilitating the development and deployment of scalable high-performance Java applications on clusters and heterogeneous systems. An initial deployment of a parallel Java programming model over a heterogeneous, distributed system shows good performance results. In addition, a framework for the agents' startup mechanism and organization is introduced to provide scalable deployment and communication among the agents.

Flexible database-driven opacity and spectrum calculations

The integration of computational atomic physics and information technology can provide flexible, convenient tools for high energy density physicists both computational and experimental. In this work, a flexible atomic database is applied to plasma opacity and spectrum calculations. Multiple atomic data models, such as screened hydrogenic, non-relativistic (ATBASE) and jj split relativistic (RSSUTA), are seamlessly incorporated into the software architecture. High performance parallel programming is applied to accelerate the computation of both the atomic data and the opacity calculations. The user interface is graphical and provided as an easy to use and convenient method to access the data and diagnose the calculations. We present opacity and spectrum results for Kr and Nb plasmas as examples of this new capability.

An Introduction of SCore and Its Installation Information

This paper introduces an overview of the SCore cluster system software running on the Linux operating system. SCore utilizes PC cluster hardware efficiently and provides its users a high-performance parallel programming environment. This paper also presents brief installation information for the end-users.

High-performance parallel programming in Java: exploiting native libraries

With most of today's fast scientific software written in Fortran and C, Java has a lot of catching up to do. In this paper we discuss how new Java programs can capitalize on high-performance libraries for other languages. With the help of a tool we have automatically created Java bindings for several standard libraries: MPI, BLAS, BLACS, PBLAS and ScaLAPACK. The purpose of the additional software layer introduced by the bindings is to resolve the interface problems between different programming languages such as data type mapping, pointers, multidimensional arrays, etc. For evaluation, performance results are presented for Java versions of two benchmarks from the NPB and PARKBENCH suites on the IBM SP2 using JDK and IBM's high-performance Java compiler, and on the Fujitsu AP3000 using Toba - a Java-to-C translator. The results confirm that fast parallel computing in Java is indeed possible.

High-performance parallel programming in Java: exploiting native libraries

With most of today's fast scientific software written in Fortran and C, Java has a lot of catching up to do. In this paper we discuss how new Java programs can capitalize on high-performance libraries for other languages. With the help of a tool we have automatically created Java bindings for several standard libraries: MPI, BLAS, BLACS, PBLAS and ScaLAPACK. The purpose of the additional software layer introduced by the bindings is to resolve the interface problems between different programming languages such as data type mapping, pointers, multidimensional arrays, etc. For evaluation, performance results are presented for Java versions of two benchmarks from the NPB and PARKBENCH suites on the IBM SP2 using JDK and IBM's high-performance Java compiler, and on the Fujitsu AP3000 using Toba – a Java-to-C translator. The results confirm that fast parallel computing in Java is indeed possible. © 1998 John Wiley & Sons, Ltd.

AN OVERVIEW OF PROGRAMMING THE IWARP SYSTEM

iWarp is a joint project between Carnegie Mellon University and Intel Corporation to develop a system for high speed signal processing and scientific computing. An iWarp system consists of multiple iWarp cells and employs parallelism on multiple levels. The iWarp cell contains an iWarp component and some local memory; therefore each iWarp cell can execute a different program since programs and data are stored in the local memory of each cell. Furthermore, the iWarp component integrates a communication engine with a powerful computation engine, and these two engines operate independently. The iWarp computation engine is controlled by a wide instruction word, which allows the parallel operation of the major functional units, delivering a peak performance of 20 million integer operations/s plus 20 (10) million single (double) precision floating point operations/s. The iWarp communication engine supports various inter-processor communication styles, ranging from the well-known message-passing style to systolic communication. This paper provides a brief overview of the software tools that deal with the parallelism in an iWarp system. Together they provide a powerful platform for the development of high-performance parallel programs for iWarp based systems. The research was supported in part by Defense Advanced Research Projects Agency (DoD) monitored by the Space and Naval Warfare Systems Command under Contract N00039-87-C-0251. The views and conclusions contained in this paper are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency (DoD), Carnegie Mellon University, or Intel Corporation.