Development Of Parallel Applications Research Articles

Boosting HPC data analysis performance with the ParSoDA-Py library

Developing and executing large-scale data analysis applications in parallel and distributed environments can be a complex and time-consuming task. Developers often find themselves diverted from their application logic to handle technical details about the underlying runtime and related issues. To simplify this process, ParSoDA, a Java library, has been proposed to facilitate the development of parallel data mining applications executed on HPC systems. It simplifies the process by providing built-in scalability mechanisms relying on the Hadoop and Spark frameworks. This paper presents ParSoDA-Py, the Python version of the ParSoDA library, which allows for further support of commonly used runtimes and libraries for big data analysis. After a complete library redesign, ParSoDA can be now easily integrated with other Python-based distributed runtimes for HPC systems, such as COMPSs and Apache Spark, and with the large ecosystem of Python-based data processing libraries. The paper discusses the adaptation process, which takes into consideration the new technical requirements, and evaluates both usability and scalability through some case study applications.

Development of Monte Carlo hybrid parallel algorithm and application in high resolution spatial-energy flux distribution calculation

Monte Carlo transport calculations for fusion reactors are very challenging due to factors such as large radiation decay gradients, complex geometric structures, and strong neutron anisotropy. For such deep penetration shielding calculation problems, in order to obtain accurate global flux distribution, variance reduction methods and parallel computing are necessary means. When using the global weight window variance reduction technique, the huge deviation of the particle arrival weight and the weight window can lead to an extreme number of splits, which is the long history problem. If a process encounters the long history problem in parallel computing and takes a lot of time to complete, other processes will stop computing because the histories allocated in the computing cycle have been completed, seriously degrading computing efficiency. In addition, the calculation of the shutdown dose rate, a key issue in fusion neutronics calculations, requires a high “space-energy” resolution, which brings huge challenges to computational memory, especially in parallel computing. In order to overcome the degradation of parallel efficiency caused by the long history problem and the memory limitation in large-scale shutdown dose rate calculation, a hybrid parallel algorithm based on shared memory was researched. Finally, the hybrid parallel architecture was built in the Monte Carlo code cosRMC to realize dynamic load balancing between threads. Through the memory sharing among all threads of a single node, the pressure of memory consumption can be effectively relieved. In order to verify the effectiveness of the hybrid parallel algorithm developed in this paper, it was applied to the calculation of CFETR. The results showed that the hybrid parallel algorithm can effectively alleviate the impact of the long history problem on the parallel efficiency. In terms of saving memory, the hybrid parallel algorithm also showed a significant effect, which provides a guarantee for large-scale shutdown dose rate calculation.

A parallel programming assessment for stream processing applications on multi-core systems

Multi-core systems are any computing device nowadays and stream processing applications are becoming recurrent workloads, demanding parallelism to achieve the desired quality of service. As soon as data, tasks, or requests arrive, they must be computed, analyzed, or processed. Since building such applications is not a trivial task, the software industry must adopt parallel APIs (Application Programming Interfaces) that simplify the exploitation of parallelism in hardware for accelerating time-to-market. In the last years, research efforts in academia and industry provided a set of parallel APIs, increasing productivity to software developers. However, a few studies are seeking to prove the usability of these interfaces. In this work, we aim to present a parallel programming assessment regarding the usability of parallel API for expressing parallelism on the stream processing application domain and multi-core systems. To this end, we conducted an empirical study with beginners in parallel application development. The study covered three parallel APIs, reporting several quantitative and qualitative indicators involving developers. Our contribution also comprises a parallel programming assessment methodology, which can be replicated in future assessments. This study revealed important insights such as recurrent compile-time and programming logic errors performed by beginners in parallel programming, as well as the programming effort, challenges, and learning curve. Moreover, we collected the participants’ opinions about their experience in this study to understand deeply the results achieved.

Stencil Calculations with Algorithmic Skeletons for Heterogeneous Computing Environments

The development of parallel applications is a difficult and error-prone task, especially for inexperienced programmers. Stencil operations are exceptionally complex for parallelization as synchronization and communication between the individual processes and threads are necessary. It gets even more difficult to efficiently distribute the computations and efficiently implement communication when heterogeneous computing environments are used. For using multiple nodes, each having multiple cores and accelerators such as GPUs, skills in combining frameworks such as MPI, OpenMP, and CUDA are required. The complexity of parallelizing the stencil operation increases the need for abstracting from the platform-specific details and simplify parallel programming. One way to abstract from details of parallel programming is to use algorithmic skeletons. This work introduces an implementation of the MapStencil skeleton that is able to generate parallel code for distributed memory environments, using multiple nodes with multicore CPUs and GPUs. Examples of practical applications of the MapStencil skeleton are the Jacobi Solver or the Canny Edge Detector. The main contribution of this paper is a discussion of the difficulties when implementing a universal Skeleton for MapStencil for heterogeneous computing environments and an outline of the identified best practices for communication intense skeletons.

Evaluation of Intel's DPC++ Compatibility Tool in heterogeneous computing

The Intel DPC++ Compatibility Tool is a component of the Intel oneAPI Base Toolkit. This tool automatically transforms CUDA code into Data Parallel C++ (DPC++), thus assisting in the migration process. DPC++ is an implementation of the programming standard for heterogeneous computing known as SYCL, which unifies the development of parallel applications on CPUs, GPUs or even FPGAs.This paper analyzes the DPC++ Compatibility Tool by considering the manual intervention required and the problems encountered while migrating the Rodinia benchmarks. For this suite, this tool achieves an impressive rate of almost 87% for code successfully migrated. Moreover, a comparative study of the performance obtained by the migrated code was carried out, showing a moderate overhead in most of the migrated examples. Finally, a performance comparison on different devices was also performed.

СПОСІБ РОЗРОБКИ ІЗОЕФЕКТИВНОЇ ГЕТЕРОГЕННОЇ СИСТЕМИ НА ОСНОВІ МАШИННОГО НАВЧАННЯ ДЛЯ ЗАДАЧІ ДИСКРЕТНОГО ПЕРЕТВОРЕННЯ ФУР’Є

In this paper, the isoefficiency of MPP systems and heterogeneous CPU-GPU systems on the problem of discrete Fourier transform is considered. The development of parallel applications as its goal can not only reduce execution time, but also provide opportunities to solve problems of a larger dimension. The peculiarity of algorithm parallelization includes the efficient use of hardware while increasing the dimension of the problem is an important characteristic of parallel computing. However, currently heterogeneous systems have not been researched extensively to determine isoefficiency characteristics and build application-specific systems around said method, although there are articles that show potential using isoefficiency to design the system and using heterogeneous approach to accelerate performance of different tasks. Discrete Fourier Transform algorithm lets build systems that discretize analogue and digital signals and it can serve as a benchmark to test different systems. Algorithms suited for MPP systems can use analytical approach to find out issoefficiency function and to determine how scaling the system or changing the size of the task will change its performance metrics. One of the most popular approaches to linking up processing units in MPP systems is using hypercube topology. MPP system that is connected using this topology will be analyzed. CPU-GPU heterogeneous system will be analyzed using an approach based on polynomial regression. Due to the nature of heterogeneous systems, analytic approach used in MPP system is impossible. Predictive model based on polynomial regression will use modelling results from using CPU and GPU separately to estimate how much time it will take for heterogeneous system to finish the task. To ensure accuracy of the experiment, several systems will be used to model the task. Using this approach, resulting issoefficient heterogeneous system will be analyzed using performance metrics s

An MPI-based MPSoC Platform in FPGA

The increase in the level of integration of components on silicon became possible to build systems with multiple processors on a single chip. These systems are named MPSoCs (Multi-Processor Systems-on-Chip), and some of them use Networks-on-Chip (NoC) as the communication infrastructure to interconnect their components. This work presents an MPSoC platform designed for performance evaluation of an NoC in FPGA. The platform relies on the use of 32-bit programmable processors and an MPI (Message Passing Interface) communication library, providing a flexible infrastructure for traffic generation and analysis, and enabling the development of parallel applications. For evaluation, we implemented a proof-of-concept instance of the proposed platform in FPGA, and the experimental results show that the software layer of the communication infrastructure dominates communication latency.

Performance Reduction for Automatic Development of Parallel Applications for Reconfigurable Computer Systems

In the paper, we review a suboptimal methodology of mapping of a task information graph on the architecture of a reconfigurable computer system. Using performance reduction methods, we can solve computational problems which need hardware costs exceeding the available hardware resource. We proved theorems, concerning properties of sequential reductions. In our case, we have the following types of reduction such as the reduction by number of basic subgraphs, by number of computing devices, and by data width. On the base of the proved theorems and corollaries, we developed the methodology of reduction transformations of a task information graph for its automatic adaptation to the architecture of a reconfigurable computer system. We estimated the maximum number of transformations, which, according to the suggested methodology, are needed for balanced reduction of the performance and hardware costs of applications for reconfigurable computer systems.

A Novel, Highly Integrated Simulator for Parallel and Distributed Systems

In an era of complex networked parallel heterogeneous systems, simulating independently only parts, components, or attributes of a system-under-design is a cumbersome, inaccurate, and inefficient approach. Moreover, by considering each part of a system in an isolated manner, and due to the numerous and highly complicated interactions between the different components, the system optimization capabilities are severely limited. The presented fully-distributed simulation framework (called as COSSIM) is the first known open-source, high-performance simulator that can handle holistically system-of-systems including processors, peripherals and networks; such an approach is very appealing to both Cyber Physical Systems (CPS) and Highly Parallel Heterogeneous Systems designers and application developers. Our highly integrated approach is further augmented with accurate power estimation and security sub-tools that can tap on all system components and perform security and robustness analysis of the overall system under design—something that was unfeasible up to now. Additionally, a sophisticated Eclipse-based Graphical User Interface (GUI) has been developed to provide easy simulation setup, execution, and visualization of results. COSSIM has been evaluated when executing the widely used Netperf benchmark suite as well as a number of real-world applications. Final results demonstrate that the presented approach has up to 99% accuracy (when compared with the performance of the real system), while the overall simulation time can be accelerated almost linearly with the number of CPUs utilized by the simulator.

Parallelisation of practical shared sampling alpha matting with OpenMP

In modern filmmaking industry, image matting has been one of the common tasks in video side effects and the necessary intermediate steps in computer vision. It pulls the foreground object from the background of an image by estimating the alpha values. However, the computational speed for matting high resolution images can be significantly slow due to its complexity and computation that is proportional to the size of unknown region. In order to improve the performance, we implement a parallel alpha matting code with OpenMP from existing sequential code for running on the multicore servers. We present and discuss the algorithm and experimentation results from the perspective of the parallel application developer. The development takes less effort, and the results show significant performance improvement of the entire program.

Parallelisation of practical shared sampling alpha matting with OpenMP

In modern filmmaking industry, image matting has been one of the common tasks in video side effects and the necessary intermediate steps in computer vision. It pulls the foreground object from the background of an image by estimating the alpha values. However, the computational speed for matting high resolution images can be significantly slow due to its complexity and computation that is proportional to the size of unknown region. In order to improve the performance, we implement a parallel alpha matting code with OpenMP from existing sequential code for running on the multicore servers. We present and discuss the algorithm and experimentation results from the perspective of the parallel application developer. The development takes less effort, and the results show significant performance improvement of the entire program.

On the code modernization of shared sampling alpha matting with OpenMP

Image matting separates the foreground object from a given image by estimating the alpha values. It has been one crucial step in computer vision, as well as an essential technique in visual effects in the modern filmmaking industry. Due to its time complexity being proportional to the size of the unknown region, the computational speed for processing image matting on high-resolution images has been traditionally slow. In this paper, we present the code modernization of Shared Sampling Alpha Matting (SSAM) algorithm with OpenMP to speed up the computation on multicore server platforms. Significant efforts are required due to the sequential nature of the code, with data dependencies and complex data structures in the original code. The implementation of migrating the SSAM code to multicore platforms and the experiments performed on several server platforms of different categories/specs are presented. Our experimental results demonstrate that the Alpha Matting using OpenMP increases performance significantly on Intel Xeon Phi Knights Landing (KNL) processors when processing high-resolution images. The code improvements we achieved can support a parallel application developer with high-performance goals.

Upgrading a high performance computing environment for massive data processing

High-performance computing (HPC) and massive data processing (Big Data) are two trends that are beginning to converge. In that process, aspects of hardware architectures, systems support and programming paradigms are being revisited from both perspectives. This paper presents our experience on this path of convergence with the proposal of a framework that addresses some of the programming issues derived from such integration. Our contribution is the development of an integrated environment that integretes (i) COMPSs, a programming framework for the development and execution of parallel applications for distributed infrastructures; (ii) Lemonade, a data mining and analysis tool; and (iii) HDFS, the most widely used distributed file system for Big Data systems. To validate our framework, we used Lemonade to create COMPSs applications that access data through HDFS, and compared them with equivalent applications built with Spark, a popular Big Data framework. The results show that the HDFS integration benefits COMPSs by simplifying data access and by rearranging data transfer, reducing execution time. The integration with Lemonade facilitates COMPSs’s use and may help its popularization in the Data Science community, by providing efficient algorithm implementations for experts from the data domain that want to develop applications with a higher level abstraction.

DtCraft: A High-Performance Distributed Execution Engine at Scale

Recent years have seen rapid growth in data-driven distributed systems, such as Hadoop MapReduce, Spark, and Dryad. However, the counterparts for high-performance or compute-intensive applications including large-scale optimizations, modeling, and simulations are still nascent. In this paper, we introduce DtCraft, a modern C++ based distributed execution engine to streamline the development of high-performance parallel applications. Users need no understanding of distributed computing and can focus on high-level developments, leaving difficult details, such as concurrency controls, workload distribution, and fault tolerance handled by our system transparently. We have evaluated DtCraft on both micro-benchmarks and large-scale optimization problems, and shown the promising performance from single multicore machines to clusters of computers. In a particular semiconductor design problem, we achieved ${30\times }$ speedup with 40 nodes and ${15\times }$ less development efforts over hand-crafted implementation.

Impact study of data locality on task-based applications through the Heteroprio scheduler.

The task-based approach has emerged as a viable way to effectively use modern heterogeneous computing nodes. It allows the development of parallel applications with an abstraction of the hardware by delegating task distribution and load balancing to a dynamic scheduler. In this organization, the scheduler is the most critical component that solves the DAG scheduling problem in order to select the right processing unit for the computation of each task. In this work, we extend our Heteroprio scheduler that was originally created to execute the fast multipole method on multi-GPUs nodes. We improve Heteroprio by taking into account data locality during task distribution. The main principle is to use different task-lists for the different memory nodes and to investigate how locality affinity between the tasks and the different memory nodes can be evaluated without looking at the tasks’ dependencies. We evaluate the benefit of our method on two linear algebra applications and a stencil code. We show that simple heuristics can provide significant performance improvement and cut by more than half the total memory transfer of an execution.

Transforming powerlist-based divide-and-conquer programs for an improved execution model

Powerlists are data structures that can be successfully used for defining parallel programs based on divide-and-conquer paradigm. These parallel recursive data structures and their algebraic theories offer both a methodology to design parallel algorithms and parallel programming abstractions to ease the development of parallel applications. The paper presents a technique for speeding up the parallel recursive programs defined based on powerlists. The improvements are achieved by applying transformation rules that introduce tuple functions and prefix operators, for which a more efficient execution model is defined. Together with the execution model, a cost model is also defined in order to allow a proper evaluation. The treated examples emphasise the fact that the transformation leads to important improvements of the programs. The speeding up is achieved by reducing the number of recursive calls, and also by enable the fusion of splitting/combining operations on different data structures. In addition, enhancing the function that has to be computed to other useful functions using a tuple, could improved the cost reduction even more.

RDGC: A Reuse Distance-Based Approach to GPU Cache Performance Analysis

In the present paper, we propose RDGC, a reuse distance-based performance analysis approach for GPU cache hierarchy. RDGC models the thread-level parallelism in GPUs to generate appropriate cache reference sequence. Further, reuse distance analysis is extended to model the multi-partition/multi-port parallel caches and employed by RDGC to analyze GPU cache memories. RDGC can be utilized for architectural space exploration and parallel application development through providing hit ratios and transaction counts. The results of the present study demonstrate that the proposed model has an average error of 3.72 % and 4.5 % (for L1 and L2 hit ratios, respectively). The results also indicate that the slowdown of RDGC is equal to 47 000 times compared to hardware execution, while it is 59 times faster than GPGPU-Sim simulator.

On parallelisation of image dehazing with OpenMP

In this paper, we present our learning experience on the design and implementation of image dehazing parallel code with OpenMP developed from existing fast sequential version. The aim of this work is to present an analysis of a case study showing the development of parallel haze removal with practical and efficient use of shared memory multi-core servers. Implementation technique and result discussions in terms of program improvements that may be needed to support parallel application developers with similar high performance goals are presented. Preliminary studies, results and experiments on haze removal application program are executed on multi-core shared memory platforms, and results show that the performance of the proposed parallel code is promising.

A Very Fast Trace-Driven Simulation Platform for Chip-Multiprocessors Architectural Explorations

Simulation is the main tool for computer architects and parallel application developers for developing new architectures and parallel algorithms on many-core machines. Simulating a many-core architecture represent a challenge to software simulators even with parallelization of these SW on multi-cores. Field Programmable Gate Arrays offer an excellent implementation platform due to inherent parallelism. Existing FPGA-based simulators however, are mostly execution-driven which consumes too many FPGA resources. Hence, they still trade-off accuracy with simulation speed as SW simulators do. In this work, an application-level trace-driven FPGA-based many-core simulator is presented. A parameterized Verilog template was developed that can generate any number of simulator tiles. The input trace has an architecturally agnostic format that is directly interpreted by the FPGA-based timing model to re-construct the execution events of the original application with accurate timing. This allows fitting a large number of simulation tiles on a single FPGA without sacrificing simulation speed or accuracy. Experimental results show that the simulator's average accuracy is ∼14 percent with simulation speeds ranging from 100’s of MIPs to over 2,200 MIPS for a 16-core target architecture. Hence, with accuracy similar to SW simulators, its speed is higher than all other FPGA-based simulators.

Algorithms for Balanced Graph Colorings with Applications in Parallel Computing

Graph coloring-in a generic sense-is used to identify subsets of independent tasks in parallel scientific computing applications. Traditional coloring heuristics aim to reduce the number of colors used as that number also corresponds to the number of parallel steps in the application. However, if the color classes produced have a skew in their sizes, utilization of hardware resources becomes inefficient, especially for the smaller color classes. Equitable coloring is a theoretical formulation of coloring that guarantees a perfect balance among color classes, and its practical relaxation is referred to here as balanced coloring. In this paper, we consider balanced coloring models in the context of parallel computing. The goal is to achieve a balanced coloring of an input graph without increasing the number of colors that an algorithm oblivious to balance would have used. We propose and study multiple heuristics that aim to achieve such a balanced coloring for two variants of coloring problem, distance-1 coloring (the standard coloring problem) and partial distance-2 coloring (defined on a bipartite graph). We present parallelization approaches for multi-core and manycore architectures and cross-evaluate their effectiveness with respect to the quality of balance achieved and performance. Furthermore, we study the impact of the proposed balanced coloring heuristics on a concrete application-viz. parallel community detection, which is an example of an irregular application. In addition, we propose several extensions to our basic balancing schemes and evaluate their balancing efficacy and performance characteristics. The thorough treatment of balanced coloring presented in this paper from algorithms to application is expected to serve as a valuable resource to parallel application developers who seek to improve parallel performance of their applications using coloring.