Parallel Programming Environment Research Articles

3D Visualization of Asynchronous Many-Task Scheduling Algorithm

The paper is devoted to the issue of visualization of the algorithm for scheduling parallel tasks. Task scheduling is a key part of the online visualization and parallel programming environment developed by the author. When programming a parallel version of one mathematical application, a suspicion arose that the scheduling algorithm does not optimally distribute the load between the workers. In this connection, it was decided to visualize its work in order to see the overall picture and possible problem areas of the algorithm. The constructed view successfully coped with the problem, and the scheduling algorithm was improved.

Special Issue 19th international workshop on algorithms, models and tools for parallel computing on heterogeneous platforms (HeteroPar'21)

Heterogeneity has emerged as one of the most profound and challenging characteristics of today's parallel environments. From the macro level, where networks of distributed computers, composed of diverse node architectures, are interconnected with potentially heterogeneous networks, to the micro level, where deeper memory hierarchies and various accelerator architectures are increasingly common, the impact of heterogeneity on all computing tasks is increasing rapidly. Traditional parallel algorithms, programming environments, and tools, designed for legacy homogeneous multiprocessors, achieve a small fraction of the efficiency and the potential performance that can be obtained in current and future heterogeneous computing platforms. New ideas, innovative algorithms, and specialized programming environments and tools are needed to efficiently use these modern parallel and heterogeneous architectures. The International workshop on algorithms, models and tools for parallel computing on heterogeneous platforms (HeteroPar) is a forum for researchers working on algorithms, programming languages, tools, and theoretical models for efficiently solving complex problems on heterogeneous parallel platforms. HeteroPar 2021 took place (virtually) in Lisbon, Portugal, organized for the 13th time in conjunction with the Euro-Par annual international conference. The format of the workshop included one keynote and 11 technical presentations. The workshop papers represented an interesting mix of topics, addressing the implementation of algorithms and kernels for heterogeneous computing, programming models, data management, runtime and resource management, energy efficiency, cloud computing, and artificial intelligence-based methods oriented towards heterogeneous platforms, as the basis for the next generation exascale computers. The selected papers cover a good spectrum of research topics in the area of heterogeneous computing showing the challenges present on these modern platforms.

Research on the Main Points of Parallel Programming Technology Based on Heterogeneous System

In recent years, with the continuous improvement of high-performance computing hardware platform, the application of high performance for a wide range of high-performance, traditional parallel programming environment tools cannot meet the growing demand for high-performance applications and the face of parallel software development environment in a relatively backward state. The heterogeneous system of the new coprocessor with a high performance and power ratio can have high performance computing needs to provide the task to accelerate. As a result, heterogeneous systems are becoming a new trend in high performance computing. However, with the introduction of the new coprocessor and the expansion of the system scale, heterogeneous systems are difficult to program the problem is also increasingly prominent. In view of this problem, this paper adopts the abstract technology to enhance the granularity of parallel programming, and designs a structured parallel programming prototype system which can achieve different levels of implicit parallelism.

Research on the Development and Application of Parallel Programming Technology in Heterogeneous Systems

In recent years, with the continuous improvement of high performance computing hardware platform,the application of high performance for a wide range of high-performance, traditional parallel programming environment tools cannot meet the growing demand for high-performance applications and the face of parallel software development environment in a relatively backward state. The heterogeneous system of the new coprocessor with a high performance and power ratio can have high performance computing needs to provide the task to accelerate. As a result, heterogeneous systems are becoming a new trend in high performance computing. However, with the introduction of the new coprocessor and the expansion of the system scale, heterogeneous systems are difficult to program the problem is also increasingly prominent. In view of this problem, this paper adopts the abstract technology to enhance the granularity of parallel programming, and designs a structured parallel programming prototype system which can achieve different levels of implicit parallelism

Implementation of the LAMMPS package using T-system with an Open Architecture

Supercomputer applications are usually implemented in the C, C++, and Fortran programming languages using different versions of the Message Passing Interface library. The "T-system" project (OpenTS) studies the issues of automatic dynamic parallelization of programs. In practical terms, the implementation of applications in a mixed (hybrid) style is relevant, when one part of the application is written in the paradigm of automatic dynamic parallelization of programs and does not use any primitives of the MPI library, and the other part of it is written using the Message Passing Interface library. In this case, the library is used, which is a part of the T-system and is called DMPI (Dynamic Message Passing Interface). In this way, it is necessary to evaluate the effectiveness of the MPI implementation available in the T-system. The purpose of this work is to examine the effectiveness of DMPI implementation in the T-system. In a classic MPI application, 0% of the code is implemented using automatic dynamic parallelization of programs and 100% of the code is implemented in the form of a regular Message Passing Interface program. For comparative analysis, at the beginning the code is executed on the standard Message Passing Interface, for which it was originally written, and then it is executed using the DMPI library taken from the developed T-system. Сomparing the effectiveness of the approaches, the performance losses and the prospects for using a hybrid programming style are evaluated. As a result of the conducted experimental studies for different types of computational problems, it was possible to make sure that the efficiency losses are negligible. This allowed to formulate the direction of further work on the T-system and the most promising options for building hybrid applications. Thus, this article presents the results of the comparative tests of LAMMPS application using OpenMPI and using OpenTS DMPI. The test results confirm the effectiveness of the DMPI implementation in the OpenTS parallel programming environment

Performance evaluation and analysis of distributed multi-agent optimization algorithms with sparsified directed communication

There has been significant interest in distributed optimization algorithms, motivated by applications in Big Data analytics, smart grid, vehicle networks, etc. While there have been extensive theory and theoretical advances, a proportionally small body of scientific literature focuses on numerical evaluation of the proposed methods in actual practical, parallel programming environments. This paper considers a general algorithmic framework of first and second order methods with sparsified communications and computations across worker nodes. The considered framework subsumes several existing methods. In addition, a novel method that utilizes unidirectional sparsified communications is introduced and theoretical convergence analysis is also provided. Namely, we prove R-linear convergence in the expected norm. A thorough empirical evaluation of the methods using Message Passing Interface (MPI) on a High Performance Computing (HPC) cluster is carried out and several useful insights and guidelines on the performance of algorithms and inherent communication-computational trade-offs in a realistic setting are derived.

An automatic self‐initialized clustering method for brain tissue segmentation and pathology detection from magnetic resonance human head scans with graphics processing unit machine

AbstractThe proposed work introduces a fully automatic modified fuzzy c‐means (MFCM) algorithm for segmenting brain tissue into gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) which identifies the pathological conditions of magnetic resonance human head scans. The present work implements histogram smoothing using Gaussian distribution for finding the number of clusters (K) and cluster centers (C) to initialize modified FCM algorithm (MFCM). The modification includes the local impact of each pixel based on the median of local neighborhoods. This needs more computational power to reduce the processing time and requires a parallel programming environment like the Graphics Processing Unit. The parallel MFCM is performed with the help of compute unified device architecture language and reduced the processing time up to 80 speedup folds than the serial implementation in Matlab and 20 speedup folds than C programming implementation. The method is examined with the Internet Brain Segmentation Repository (IBSR20) T1W dataset. The quantitative and qualitative results of the proposed method are compared with state‐of‐the‐art‐methods using the Dice coefficient (DC). Proposed method yields high DC 0.84 ± 0.03 for GM, 0.83 ± 0.04 for WM, and 0.41 ± 0.12 for CSF segmentation. In post‐processing, 3D volumes of segmented regions have been constructed and compared with the gold standard quantitatively and qualitatively.

Unified programming concepts for unobtrusive integration of cloud-based and local parallel computing

The growth in the data and computation need of today’s operations has led to technical solutions that distribute workload over several entities for better performance. To facilitate such a paradigm, research studies have been investigating efficient approaches for combining high performance computing in shared-memory with distributed-memory environments. Meanwhile, the benefits of cloud computing and its modern enhancements have created potentials for applications to leverage the powerful, ubiquitous and cheap resources of cloud infrastructures. Yet, a small portion of the work in this area addresses the programming aspects of cloud-related technologies. Despite the extensive improvements in the fundamental mechanisms of this realm, programming environments offer little support for incorporating the high performance mechanisms of shared-memory computing with cloud computing. This study proposes a solution for an unobtrusive definition and integration of cloud-based and shared-memory parallel computing, in order to further simplify the application of cloud capabilities in local systems. It does so by implementing the proposed concepts in @PT (Annotation Parallel Task), a parallel-programming environment that utilizes native Java annotations as its language constructs. The experimental evaluations discussed here demonstrate that the proposed approach facilitates achieving the potential benefits of cloud computing for performance and energy consumption in local devices.

A Microservice Approach for a Cellular Automata Parallel Programming Environment

This paper presents an architecture for a parallel programming environment designed to assist the process of writing parallel programs. The proposed architecture allows the integration of several tools such as parallel programming skeletons and parallelizing compilers, among others. On the one hand, parallel programming skeletons simplify the design of parallel algorithms. On the other hand, parallelizing compilers rely on compiler capabilities to translate sequential programs into ones capable of using multiple processing units (CPU/GPU) found in personal computers (PCs). The technological solution presented in this work follows a microservices-oriented approach. This approach eases the integration of new parallel programming tools in the existing deployment through well-known protocols and existing services. We show how the aforementioned parallel programming tools were integrated to support the coding and parallelization of a general purpose cellular automata. Since the parallelization rely on the capabilities underlying to each tool, the focus of this work is not to deal with an efficient cellular automata parallelization. Instead, a promising microservice approach to the development of an extensible and scalable parallel programming environment is presented.

VPPE: A Novel Visual Parallel Programming Environment

Parallel programming continues to be a challenging task despite the many advances in parallel architectures and their wide availability in the cloud. The need both to partition the workload among various processing elements and to specify communication between them to share code and data, and to coordinate their tasks, requires from the developer a deep understanding of the problem, the parallel architecture and the programming language used in order to develop efficient parallel applications. This problem can be reduced significantly through the use of visual programming languages to hide most aspects related to the specification of communication and processes management. This paper presents VPPE, a novel Visual Parallel Programming Environment that allows developers to program parallel applications through organising workflows of interconnected icons. VPPE is a cloud environment that supports icons for specifying: I/O operations, workflow organisation, communication, and processing. Processing computing patterns supported so far include Single Program Multiple Data, Multiple Program Multiple Data, Pipeline, and Master–Slave. The paper highlights the design of VPPE based on a context-free graph grammar, its current implementation based on Java-MPI, its use in developing various parallel applications, and its evaluation compared to Java-MPI text-based programming.

The Twenty Sixth International Heterogeneity in Computing Workshop (HCW) and to the Fifteenth International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Platforms (HeteroPar)

Heterogeneity is emerging as one of the most profound and challenging characteristics of today's parallel environments. As most modern computing systems are heterogeneous, from the macro level, where networks of distributed computers, composed by diverse node architectures, are interconnected with potentially heterogeneous networks, to the micro level, where deeper memory hierarchies and various accelerator architectures are increasingly common, the impact of heterogeneity on all computing tasks is increasing rapidly. Traditional parallel algorithms, programming environments and tools, designed for legacy homogeneous multiprocessors, will at best achieve a small fraction of the efficiency and the potential performance that we should expect from parallel computing in tomorrow's highly diversified and mixed environments. New ideas, innovative algorithms, and specialized programming environments and tools are needed to efficiently use these new and multifarious parallel architectures. This special issue has the propose to collect extended version of contributions submitted to the Twenty Sixth International Heterogeneity in Computing Workshop (HCW) and to the Fifteenth International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Platforms (HeteroPar). After a thorough peer-reviewing process, three papers were selected for publication. The topics addressed in this issue are the performance of cloud function providers, a cross-architecture Kalman filter to accelerate the reconstruction computation of particles collision and, a methodology to model applications jointly with a fast and greedy algorithm to obtain realist mapping and scheduling solution on heterogeneous systems. Cloud functions are becoming an increasingly popular method of running distributed applications, as they allow the developer for deploying their code in the form of a function to the cloud, which is then responsible for automatic resource provision and scaling. Kamil Figiela, Adam Gajek, Adam Zima, Beata Obrok and Maciej Malawski [Figiela, 2018] present a performance evaluation study of heterogeneous cloud functions where the major cloud function providers are evaluated, namely, AWS Lambda, Azure Functions, Google Cloud Functions and IBM Cloud Functions. At the LHCb detector in the Large Hadron Collider, the reconstruction of particle collisions in high-energy physics detectors happens at an average rate of 30 million times per second, being the Kalman filter a fundamental element in this process. Due to iterative enhancements in the detector's technology, together with the projected removal of the hardware filter, the rate of particles that will need to

Generating Fine-Grain Multithreaded Applications Using a Multigrain Approach

The recent evolution in hardware landscape, aimed at producing high-performance computing systems capable of reaching extreme-scale performance, has reignited the interest in fine-grain multithreading, particularly at the intranode level. Indeed, popular parallel programming environments, such as OpenMP, which features a simple interface for the parallelization of programs, are now incorporating fine-grain constructs. However, since coarse-grain directives are still heavily used, the OpenMP runtime is forced to support both coarse- and fine-grain models of execution, potentially reducing the advantages obtained when executing an application in a fully fine-grain environment. To evaluate the type of applications that benefit from executing in a unified fine-grain program execution model, this article presents a multigrain parallel programming environment for the generation of fine-grain multithreaded applications from programs featuring OpenMP’s API, allowing OpenMP programs to be run on top of a fine-grain event-driven program execution model. Experimental results with five scientific benchmarks show that fine-grain applications, generated by and run on our environment with two runtimes implementing a fine-grain event-driven program execution model, are competitive and can outperform their OpenMP counterparts, especially for data-intensive workloads with irregular and dynamic parallelism, reaching speedups as high as 2.6× for Graph500 and 51× for NAS Data Cube.

An Optimized and Secure Association Rule Mining Algorithm in Parallel and Distributed Environment

Data Mining is a subfield of Artificial Intelligence which is mainly used for extraction of hidden predictive information from large available databases. To Handle Large Data set is very tedious and complex task. The solution to this problem is to apply distributed or parallel approaches. The field of distributed data mining has therefore gained increasing importance in the last decade. In Distributed data mining Security is the major problem with respect to association rule mining projects. Association rule mining is become one of the core data mining tasks and has attracted tremendous interest among data mining Researchers. The Apriori algorithm by Rakesh Agarwal has emerged as one of the best Association Rule mining algorithms. It also serves as the base algorithm for most parallel algorithms. The performance of data mining algorithm can be accelerated from O(N) to O(N/k) with parallelism, where N = number of data records and k =number of nodes in distributed system. By using various cryptographic techniques and Method interesting associations and patterns between variables of big database can be observed securely. This system addresses the problem of Secure association rule mining over the horizontally distributed database system. In Current technique for association rule mining huge processing time and also high threat from various attacks. The proposed system can provide security at the time of mining data from various data sources and also reduce computation time by using parallel programming environment like OpenMp. The goal of proposed approach is to find all association rule with maintain support s and confidence c and minimize the information disclosed about the private databases held by those player .This system is based on distributed mining algorithm, K&C and AES algorithm. Distributed mining algorithm used here is the distributed version of apriori algorithm. With proposed approach speed up is acquired using Parallel Computing while preserving the privacy of the data .

A Practical Parallel Programming Course based on Problems of the Spanish Parallel Programming Contest

This paper presents an experience of an introductory course on Parallel Programming. The course is dedicated to parallel programming tools and environments, and in particular to the analysis, development and optimization of parallel algorithms. It has a practical orientation and is guided with the use of problems from the Spanish Parallel Programming Contest. The different units are presented in the traditional lecture format, and a practical session accompanies each unit, with problems to work with in the tools or algorithmic paradigms presented in the previous lecture. The students work in the practical sessions on problems and using the system of the contest, which facilitates online and real time validation of their implementations. The practical approach of the course and the continuous evaluation used led to an important increase in the marks.

MADNESS: A Multiresolution, Adaptive Numerical Environment for Scientific Simulation

MADNESS (multiresolution adaptive numerical environment for scientific simulation) is a high-level software environment for solving integral and differential equations in many dimensions that uses adaptive and fast harmonic analysis methods with guaranteed precision based on multiresolution analysis and separated representations. Underpinning the numerical capabilities is a powerful petascale parallel programming environment that aims to increase both programmer productivity and code scalability. This paper describes the features and capabilities of MADNESS and briefly discusses some current applications in chemistry and several areas of physics.

Efficient method of calculating Shannon entropy of non-static transport problem in message passing parallel programming environment

For the Monte Carlo simulation of the non-static transport problem, there must be many calculation steps. Because some particles cannot finish their transport in the last step, they are naturally used as the source particles of the present step. These particles are called undied particles. It is difficult to adjust the history number of each step to obtain higher efficiency because the adjusting rule is hard to find. The most direct method is to set a large enough history number for all steps. But evidently, it is unnecessary for some steps. Among all possible rules, one candidate of adjusting the history number is to check the convergence situation of Shannon entropy (corresponding to the distribution of some undied particle attributes) every some samples in each step to determine whether or not to simulate more particles. So, this method needs to calculate the Shannon entropy frequently. Because the classical method of calculating Shannon entropy in message passing parallel programming environment must reduce massive data, it is unpractical to be used in this situation for the great increasing of computation time with the high frequency of entropy calculation. In this paper, we propose an efficient method of calculating the entropy in the message passing parallel programming environment by letting each process calculate its entropy value based on the local data in each processer and calculating the final entropy by averaging all the entropy values gotten by all processes. The entropy value calculated by this method is not the same as that by the classical method when using finite history number, but the difference goes to zero when the history number goes to infinity. The most remarkable advantage of this method is the small increasing of computation time when calculating the entropy frequently. It is a suitable method of calculating Shannon entropy when adjusting the history number automatically based on the judgment of the convergence situation of Shannon entropy.

Programming models and applications for multicores and manycores

Programming models and applications for multicores and manycores

Calculating Hurst exponent and neutron monitor data in a single parallel algorithm

We implemented an algorithm for simultaneous parallel calculation of the Hurst exponent H and the fractal dimension D for the time series of interest. Parallel programming environment was provided by OpenMPI library installed on three machines networked in the virtual cluster and operated by Debian Wheeze operating system. We applied our program for a comparative analysis of week and a half long, one minute resolution, six channels data from neutron monitor. To ensure a faultless functioning of the written code we applied it to analysis of the random Gaussian noise signal and time series with manually introduced self-affinity features. Both of them have the well-known values of H and D. All results are in good correspondence with each other and supported by the modern theories on signal processing thus confirming the validity of the implemented algorithms. Our code could be used as a standalone tool for the different time series data analysis as well as for the further work on development and optimization of the parallel algorithms for the time series parameters calculations.

JAWS: a JavaScript framework for adaptive CPU-GPU work sharing

This paper introduces jAWS, a JavaScript framework for adaptive work sharing between CPU and GPU for data-parallel workloads. Unlike conventional heterogeneous parallel programming environments for JavaScript, which use only one compute device when executing a single kernel, jAWS accelerates kernel execution by exploiting both devices to realize full performance potential of heterogeneous multicores. jAWS employs an efficient work partitioning algorithm that finds an optimal work distribution between the two devices without requiring offline profiling. The jAWS runtime provides shared arrays for multiple parallel contexts, hence eliminating extra copy overhead for input and output data. Our preliminary evaluation with both CPU-friendly and GPU-friendly benchmarks demonstrates that jAWS provides good load balancing and efficient data communication between parallel contexts, to significantly outperform best single-device execution.

Exploiting GPUs with the Super Instruction Architecture

The Super Instruction Architecture (SIA) is a parallel programming environment designed for problems in computational chemistry involving complicated expressions defined in terms of tensors. Tensors are represented by multidimensional arrays which are typically very large. The SIA consists of a domain specific programming language, Super Instruction Assembly Language (SIAL), and its runtime system, Super Instruction Processor. An important feature of SIAL is that algorithms are expressed in terms of blocks (or tiles) of multidimensional arrays rather than individual floating point numbers. In this paper, we describe how the SIA was enhanced to exploit GPUs, obtaining speedups ranging from two to nearly four for computational chemistry calculations, thus saving hours of elapsed time on large-scale computations. The results provide evidence that the "programming-with-blocks" approach embodied in the SIA will remain successful in modern, heterogeneous computing environments.