Execution Of Parallel Programs Research Articles

Optimizing Three-Dimensional Stencil-Operations on Heterogeneous Computing Environments

Complex algorithms and enormous data sets require parallel execution of programs to attain results in a reasonable amount of time. Both aspects are combined in the domain of three-dimensional stencil operations, for example, computational fluid dynamics. This work contributes to the research on high-level parallel programming by discussing the generalizable implementation of a three-dimensional stencil skeleton that works in heterogeneous computing environments. Two exemplary programs, a gas simulation with the Lattice Boltzmann method, and a mean blur, are executed in a multi-node multi-graphics processing units environment, proving the runtime improvements in heterogeneous computing environments compared to a sequential program.

SWOT Analysis of Parallel Processing APIs - CUDA, OpenCL, OpenMP and MPI and their Usage in Various Companies

Purpose: Parallel Processing is the technique of achieving High Performance Computing (HPC) with parallel execution of programs that are synchronized during the execution time. This research paper studied the companies which use Parallel Processing techniques in their projects and products along with the identification of major Application Program Interfaces (APIs) that are used to achieve parallelism. The major aim of this research work is to perform the SWOT analysis of Parallel Processing APIs to identify the importance of each one from the company perspective. Design/Methodology/Approach: The research method adopted to accomplish the SWOT Analysis of Parallel Processing APIs - CUDA, OpenCL, OpenMP and MPI and their Usage in Various Companies is qualitative and exploratory in nature. Systematic literature review of different companies that use Parallel Processing techniques to build and develop parallel programs is done during this research work. Findings/Results: Parallel Processing constructs can be used to solve various problems in the six major application domains as: - Computational Finance & Business Economics, Artificial Intelligence, Machine Learning (ML), Data Science, Numerical Analysis and Design of Algorithms. Major Parallel Processing APIs used in companies are: - CUDA, OpenCL, OpenMP and MPI for implementing the problems with parallel execution. Foremost companies that use Parallel Processing APIs are studied and various applications, systems, models, and projects that are developed using Parallel Processing techniques are listed. SWOT Analysis is performed on all four Parallel Processing APIs and its SWOT(Strengths-Weaknesses-Opportunities-Threats) are identified. Originality/Value: Listing of SWOT Analysis (Strengths-Weaknesses-Opportunities-Threats) of Parallel Processing APIs - CUDA, OpenCL, OpenMP and MPI. Paper Type: Company Analysis research paper

Computational complexity when constructing rational plans for program execution in a given field of parallel computers

Objectives. The construction of rational plans (schedules) for parallel program execution (PPE) represents a challenging problem due to its ambiguity. The aim of this work is to create methods for developing such plans and specialized software for implementing these methods, which are based on the internal properties of algorithms, primarily on the property of internal (hidden) parallelism.Methods. The main method for developing PPE plans was the construction, analysis, and purposeful transformation of the stacked-parallel form (SPF) of information graphs of algorithms (IGA). The SPF was transformed by transferring operators from tier to tier of the SPF (this event was taken as an elementary step in determining the computational complexity of scenario execution). As a transformation tool, a method for developing transformation scenarios in the scripting programming language Lua was used. Scenarios were created by a heuristic approach using a set of Application Programming Interface (API) functions of the developed software system. These functions formed the basis for a comprehensive study of the parameters of the IGA and its SPF representation for the subsequent construction of a PPE plan applying to a given field of parallel computers.Results. Features of the internal properties of the algorithms that affect the efficiency of SPF transformations were identified during the course of computational experiments. Comparative indices of the computational complexity of obtaining PPE plans and other parameters (including code density, etc.) were obtained for various SPF transformation scenarios. An iterative approach to improving heuristic methods favors developing optimal schemes for solving the objective problem.Conclusions. The developed software system confirmed its efficiency for studying the parameters of hidden parallelism in arbitrary algorithms and rational use in data processing. The approach of using a scripting language to develop heuristic methods (scenarios) for the purposeful transformation of IGA forms showed great flexibility and transparency for the researcher. The target consumers of the developed methods for generating schedules for parallel execution of programs are, first of all, developers of translators and virtual machines, and researchers of the properties of algorithms (for identifying and exploiting the potential of their hidden parallelism). The developed software and methods have been successfully used for a number of years for increasing student competence in data processing parallelization at Russian universities.

Autotuning Parallel Programs by Model Checking

The paper presents a new approach to autotuning data-parallel programs. Autotuning is a search for optimal program settings which maximize its performance. The novelty of the approach lies in the use of the model checking method to find the optimal tuning parameters by the method of counterexamples. In our work, we abstract from specific programs and specific processors by defining their representative abstract patterns. Our method of counterexamples implements the following four steps. At the first step, an execution model of an abstract program on an abstract processor is described in the language of a model checking tool. At the second step, in the language of the model checking tool, we formulate the optimality property that depends on the constructed model. At the third step, we find the optimal values of the tuning parameters by using a counterexample constructed during the verification of the optimality property. In the fourth step, we extract the information about the tuning parameters from the counter-example for the optimal parameters. We apply this approach to autotuning parallel programs written in OpenCL, a popular modern language that extends the C language for programming both standard multi-core processors (CPUs) and massively parallel graphics processing units (GPUs). As a verification tool, we use the SPIN verifier and its model representation language Promela, whose formal semantics is good for modelling the execution of parallel programs on processors with different architectures.

Reconfigurable Logic Controller—Direct FPGA Synthesis Approach

Programmable logic controllers are commonly used in automation systems. Continuously growing demands result in the growth of control program complexity. The classic approach, based on programmatic serial-cyclic execution, results in an unacceptable extension of response time. To overcome long response time massive parallel program execution is proposed. It utilize direct in hardware program implementation in field programmable devices. The paper brings a formal method of representing control programs using flow graphs and enabling single cycle computations. The developed method accepts ladder diagrams (LD) and sequential function charts (SFC), according to IEC61131-3 standard requirements. It is capable of handling logic and arithmetic computations, enabling its hardware mapping. The intermediate form is optimized using flow graph representation and BDDs for analyzing logic dependencies. The BDD representation of logic dependencies enables direct mapping to lookup tables of a selected FPGA family. All the above steps deliver high-performance and direct hardware implementation of the control program given by standard languages. The controller response time is short, predictable, and independent from logic conditions during program execution.

Algorithms for Optimizing the Execution of Parallel Programs on High-Performance Systems When Solving Problems of Modeling Physical Processes

Algorithms for Optimizing the Execution of Parallel Programs on High-Performance Systems When Solving Problems of Modeling Physical Processes

Research on Dynamic Optimization Method of Embedded Multi-core Performance based on Runtime

: Embedded multi-core systems often show special limitations in sharing resources and storage capacity, which will lead to the reduction of parallel efficiency due to bandwidth, data competition and other factors. In this paper, a run-time optimization framework is proposed to dynamically and adaptively adjust parallel strategies according to the structure of different parallel programs. On the one hand, the performance analysis model of a parallel program structure is established to get the number of threads during the best performance. On the other hand, in view of the load imbalance in parallel thread execution, an adaptive dynamic scheduling algorithm is proposed to dynamically select the size of the scheduling block. Experimental results show that the proposed dynamic scheduling strategy can improve program performance and achieve improvement of the performance of the embedded multi-core system. Background: Embedded multi-core systems often have special limitations in sharing resources and storage capacity. These limitations often lead to parallel programs running with lower parallel efficiency due to bandwidth, data competition and other factors. Objective: In order to improve the performance of embedded multi-core systems, parallel strategies can be adjusted dynamically and adaptively for different parallel program structures. Methods: A control mechanism of the thread count based on runtime information feedback is proposed, which enables the system to dynamically select the number of threads when the program runs best according to the structure characteristics of parallel programs. Then, an adaptive dynamic scheduling algorithm is proposed to solve the load imbalance in parallel program execution. Results: An optimization framework based on run-time architecture is presented, which consists of two parts: performance monitoring and control interface. It can take corresponding optimization strategies according to the running state of parallel programs. Conclusion: The performance of the embedded multi-core system is improved.

Predicting the Soft Error Vulnerability of Parallel Applications Using Machine Learning

With the widespread use of the multicore systems having smaller transistor sizes, soft errors become an important issue for parallel program execution. Fault injection is a prevalent method to quantify the soft error rates of the applications. However, it is very time consuming to perform detailed fault injection experiments. Therefore, prediction-based techniques have been proposed to evaluate the soft error vulnerability in a faster way. In this work, we present a soft error vulnerability prediction approach for parallel applications using machine learning algorithms. We define a set of features including thread communication, data sharing, parallel programming, and performance characteristics; and train our models based on three ML algorithms. This study uses the parallel programming features, as well as the combination of all features for the first time in vulnerability prediction of parallel programs. We propose two models for the soft error vulnerability prediction: (1) A regression model with rigorous feature selection analysis that estimates correct execution rates, (2) A novel classification model that predicts the vulnerability level of the target programs. We get maximum prediction accuracy rate of 73.2% for the regression-based model, and achieve 89% F-score for our classification model.

Parallel Execution of Programs as a Support for Mutation Testing: A Replication Study

Mutation testing is well known as one of the most effective approaches to create test cases, which can detect software faults. However, its drawback is the low scalability — if no special attention is given to improve efficiency — that directly affects its application in practice. This paper shows a replication study focused on emphasizing evidence in which the use of distributed processing structures can improve mutation testing. For this purpose, an architecture that enables mutation testing concurrent execution was designed. Five load balancing algorithms responsible for controlling the distribution and execution of data while carrying out mutation testing were evaluated. Experiments were conducted in order to evaluate the scalability and performance of the architecture considering homogeneous and heterogeneous setups. A time reduction of 50% was observed when executing mutants in parallel in relation to the conventional sequential application of mutation testing. The performance gain was above 95% when there was a higher number of nodes in the distributed architecture.

Multilevel procedure for decomposition and mapping graphs

The paper analyzes the actual problem of architecture-dependent decomposition, allowing efficient planning of running a parallel task in a multiprocessor computer network. The procedure of planning means decomposition of a parallel task in a required number of processors accounting for balance restrictions and mapping of obtained portions to processors in order to minimize interprocessor communication costs. This procedure allows to reduce parallel program execution time and consequently expensive hardware usage time. The paper provides general mathematical formulation of this problem, and special cases are considered. Multilevel algorithms are suggested for a special and a general case of the problem. Results of experiments in well-known benchmarks as compared to renowned software METIS and SCOTCH.

Алгоритмические параллельные процессы и их сложность

A method for estimating the complexity of parallel processes according to the criteria of the average time of their absolutely parallel execution and the resources required for executing them (the number of nodes of the system performing the process) is proposed. These criteria are considered for a fairly general language of parallel processes, and their values can be used to judge how their real values obtained during the process on a particular system differ from their ultimately possible levels. It is important to note that the considered language of parallel processes has been implemented on multi-core computers, and the operating means for effectively managing the execution of functional parallel programs have been developed on its basis. The functional parallel programming language that has been developed and successfully applied in practice is used as a language for describing parallel programs. The described methods for estimating the complexity of parallel processes are necessary at the stage of designing and optimizing functional parallel programs.

Visualizing Hierarchical Performance Profiles of Parallel Codes Using CallFlow.

Calling context trees (CCTs) couple performance metrics with call paths, helping understand the execution and performance of parallel programs. To identify performance bottlenecks, programmers and performance analysts visually explore CCTs to form and validate hypotheses regarding degraded performance. However, due to the complexity of parallel programs, existing visual representations do not scale to applications running on a large number of processors. We present CallFlow, an interactive visual analysis tool that provides a high-level overview of CCTs together with semantic refinement operations to progressively explore CCTs. Using a flow-based metaphor, we visualize a CCT by treating execution time as a resource spent during the call chain, and demonstrate the effectiveness of our design with case studies on large-scale, production simulation codes.

Enabling On-the-Fly Hardware Tracing of Data Reads in Multicores

Software debugging is one of the most challenging aspects of embedded system development due to growing hardware and software complexity, limited visibility of system components, and tightening time-to-market. To find software bugs faster, developers often rely on on-chip trace modules with large buffers to capture program execution traces with minimum interference with program execution. However, the high volumes of trace data and the high cost of trace modules limit the visibility into the system operation to short program segments. This article introduces a new hardware/software technique for capturing and filtering read data value traces in multicores that enables a complete reconstruction of parallel program execution. The proposed technique exploits tracking of data reads in data caches and cache coherence protocol states to minimize the number of trace messages streamed out of the target platform to the software debugger. The effectiveness of the proposed technique is determined by analyzing the required trace port bandwidth and trace buffer sizes as a function of the data cache size and the number of processor cores. The results show that the proposed technique significantly reduces the required trace port bandwidth, from 12.2 to 73.9 times, when compared to the Nexus-like read data value tracing, thus enabling continuous on-the-fly data tracing at modest hardware cost.

A DSL for graph parallel programming with vertex subsets

The vertex-centric (VC) computation model has emerged as a promising approach for easy parallel programming and massive parallel execution for big graph processing. A simple graph computation can easily be implemented in the VC model, but some algorithms cannot easily be implemented in the VC model. Examples of the latter algorithms are those that use vertex subsets or subgraphs as a manipulation unit. Such a “global view style” is natural for programmers to design graph algorithms, but the VC model requires us to redesign algorithms in the “local view style” focusing on a vertex. The gap between these styles prevents us from writing parallel programs for useful graph computations. In this paper, we propose a novel DSL for graph processing that can be used to write parallel programs for big graph processing in the “global view style”. It is compiled into the VC model so that it can enjoy massive parallelism in various parallel computing environments including commercial cloud services like Amazon EC2. We show non-trivial examples in our DSL, and our experimental results show that the compiled program achieves good scalability.

The benefits of concurrent computing in tribology system design

Abstrect Many optimum designs of tribological components are highly time-constrained before final productions. It is well-known that the process of a complex simulated design can be considerably accelerated by using some form of parallel computing. Also, for many tribological models additional assumptions can be relaxed with stricter design constrains, if the execution can be speeded up. In this study, the concurrent computing for tribological design is proposed, which is to perform parallel computing using the multitasking capability of today's operating system. In the concurrent computing a master program, which manages the process of the optimization, is used to launch a number of standalone slave programs (air bearing models) in a quick succession. And the operating system (MS-Windows) of the computer manages the parallel execution of the slave programs. Other than the standard programming language (Fortran 95) this approach uses none of the general parallel programming paradigms or directives, such as message passing interface, OpenMP, or coding using graphics processing units. In this study, the algorithm for the multiobjective optimization is group inching fortification method and the concurrent computing is executed in the algorithm-level. High parallel computing speedups are obtained in the simulated bearing designs. The approach can also be applied in using commercial general-purpose packages for modelling and self-coded methods for optimum design of tribological components or systems.

A scheduling selection process for energy‐efficient task execution on DVFS processors

SummaryThe efficient execution of parallel programs with respect to execution time and energy consumption is a major concern, and often, scheduling methods are used to achieve a good performance. In this article, we consider the problem of scheduling a set of independent tasks on a parallel system with homogeneous execution units providing frequency scaling. The set of tasks has the property that the tasks exhibit a task‐specific inhomogeneous and non‐linear behavior of their specific time‐energy relation. In addition, it is assumed that execution time and energy consumption behave in a non‐linear manner with respect to frequency scaling. For the assignment of these tasks to execution units, we propose a scheduling selection process combining scheduling algorithms, which determine a task assignment, with a subsequent selection of frequency scaling, which we call schedule execution modes. This process builds a rich set of alternative schedule execution modes from which efficient (or Pareto‐optimal) schedule execution modes can be selected. Experiments are done for the SPEC CPU benchmarks. Experimental results illustrate that the enriched scheduling process leads to task assignments resulting in an efficient execution on DVFS processors.

Hardware Evaluation and Software Framework Construction for Performance Measurement of Embedded Processor

A frame work for analysing the capabilities and area of improvements for working of an embedded processor is constructed, and also a methodology for comparative study of simulation of processor on load and hardware results are explained in this paper. The processor can be modelled as a standalone processor or as a group of processors working together to take parallel program execution mode. The proposed frame work and simulation method uses the processor representation of current embedded processor model which is relevant in product design and devices modelling. This system utilizes the ARM processor model which consists of programs used for computing and standardizing the operational quality of a processor for any particular domain of applications. This paper presents steps for creating experimental framework which uses a simulation system running on a Linux based operating system along with kernel for running test applications on the target which is run in simulation mode. To evaluate the simulation with respect to the real hardware, this paper has made an observation of cache related performance of a typical embedded processor. The processor which is taken for the study is ARM926EJ-S processor. The future architectural components will be designed and simulated in the framework as continuation of this experiment. Their ARM9 technology equivalent design parameters for configurable modules of CPU in 1.8 micro-meter and experimental nano-meter level implementation would be calculated with architecture component design tools.

Machine Learning-Based Energy Optimization for Parallel Program Execution on Multicore Chips

Energy is increasingly becoming the major constraint in designing multicore chips. Power and performance are the main components of energy and are inversely correlated. In this paper, we study the energy optimization of multicore chips that process parallel workloads using either power or performance optimization. To do so, we propose novel machine learning-based global and dynamic power management controller. The controller is used either to maximize performance within a fixed power budget or to minimize the consumed power to achieve the same baseline performance. The controller is also scalable, as it does not incur significant overhead as the number of cores or demands increases. The technique was evaluated using the PARSEC benchmark suite on a full-system simulator. The experimental results show that our global power controller outperforms, in terms of the EDP metric, the non-DVFS baseline by 28 and 35.5%, when optimized for performance and power, respectively. This suggests that optimizing power is more related to energy efficiency than optimizing performance.

Еквівалентність двох систем паралельного виконання

The method for properties proof for parallel programs running multiple-instance interleaving with shared memory is investigated. Two systems for parallel execution of programs are considered and the justification of the mutual expressiveness of these two approaches are presented in this paper. The first one is with a fixed yet parametric number of programs executing in parallel. The second one implements a generation model (start) and joining after the stop (join) of parallel programs (also called multithreading). The corresponding two basic functions are provided, and their semantics are given. Also, the semantics of other functions related to parallel execution, resource management and access synchronization are presented in this paper. The theorem on the (functional) equivalence of these two systems and its justification are presented. The program in this case is considered as a function over the data. It is argued that for an arbitrary program in one of the systems of parallelism it is possible to construct the corresponding program in another system, which returns the same result (that is, functionally equivalent). Only productive programs are considered here in the context of mutual expressiveness, because otherwise they "hang" and do not return any result, thus they are out of our scope. The obtained result allows us to move reasoning from the more complex system (by structure) with a dynamic generation of parallel program instances to the simpler system (for proofs) with a parametric number of identical programs executed in parallel. Questions for further research in this direction are also identified. Problems in programming 2018; 2-3: 093-098

QoS Management on Heterogeneous Architecture for Multiprogrammed, Parallel, and Domain-Specific Applications

Quality-of-service (QoS) management is widely employed to provide differentiable performance to programs with distinctive priorities on conventional chip-multiprocessor (CMP) platforms. Recently, heterogeneous architecture integrating diverse processor cores on the same silicon has been proposed to better serve various application domains, and it is expected to be an important design paradigm of future processors. Therefore, the QoS management on emerging heterogeneous systems will be of great significance. Workloads on heterogeneous architectures can be multiprogrammed, heterogeneous, and/or domain specific depending on the form factor and device of interest. Considering the diverse characteristics of these three classes of workloads is important when managing QoS on heterogeneous architectures. For example, for parallel applications, considering the diverse characteristics of thread synchronization, data sharing, and parallelization pattern of representative parallel applications, governing the execution of multiple parallel programs with different performance requirements becomes a complicated yet significant problem. In this paper, we study QoS management for multiprogrammed, parallel, and domain-specific applications running on heterogeneous CMP systems. We comprehensively assess a series of task-to-core mapping policies on a real heterogeneous hardware (QuickIA) by characterizing their impacts on performance of individual applications. Our evaluation results show that the proposed QoS policies are effective to improve the performance of programs with highest priority while striking good tradeoff with system fairness.