Parallel Execution Time Research Articles

Parallel Lattice Boltzmann Method with Blocked Partitioning

This paper presents and discusses a blocked parallel implementation of bi- and three-dimensional versions of the Lattice Boltzmann Method. This method is used to represent and simulate fluid flows following a mesoscopic approach. Most traditional parallel implementations use simple data distribution strategies to parallelize the operations on the regular fluid data set. However, it is well known that block partitioning is usually better. Such a parallel implementation is discussed and its communication cost is established. Fluid flows simulations crossing a cavity have also been used as a real-world case study to evaluate our implementation. The presented results with our blocked implementation achieve a performance up to 31% better than non-blocked versions, for some data distributions. Thus, this work shows that blocked, parallel implementations can be efficiently used to reduce the parallel execution time of the method.

Communication-Aware Supernode Shape

In this paper we revisit the supernode-shape selection problem, that has been widely discussed in bibliography. In general, the selection of the supernode transformation greatly affects the parallel execution time of the transformed algorithm. Since the minimization of the overall parallel execution time via an appropriate supernode transformation is very difficult to accomplish, researchers have focused on scheduling-aware supernode transformations that maximize parallelism during the execution. In this paper we argue that the communication volume of the transformed algorithm is an important criterion, and its minimization should be given high priority. For this reason we define the metric of the per process communication volume and propose a method to minimize this metric by selecting a communication-aware supernode shape. Our approach is equivalent to defining a proper Cartesian process grid with MPI_Cart_Create, which means that it can be incorporated in applications in a straightforward manner. Our experimental results illustrate that by selecting the tile shape with the proposed method, the total parallel execution time is significantly reduced due to the minimization of the communication volume, despite the fact that a few more parallel execution steps are required.

Performance optimization and parallelization of turbo decoding for software-defined radio

This paper describes the optimization, parallelization, and simulated execution performance of a software double-binary turbo decoder implementation supporting the WiMAX standard suitable for software-defined radio (SDR). Turbo codes offer excellent error-correcting performance, but they introduce significant computational demands in a communication system. In order to enhance execution performance for SDR, software for a turbo decoder based on the maximum a posteriori (MAP) algorithm was first adapted from the open-source Coded Modulation Library. Optimization and parallelization of the adapted software were then pursued and assessed with a multiprocessor version of the SimpleScalar simulator. Simulation results show that serial optimizations of the original adapted stand-alone C decoder software improve performance by more than 200%. The use of special instructions to accelerate important functions provides a further benefit of nearly 40% relative to the new baseline for performance. Exploiting the parallelism available in the MAP algorithm then yields a speedup of 10.8 on 12 processors. Simulation also shows that cache effects do not have a significant impact on parallel execution times.

Improved methods for divisible load distribution on d‐dimensional hypercube using multi‐installment

In divisible load distribution, the classic hypercube method distributes divisible load such that each layer transmits one load fraction to the next layer. This classic load distribution method provides only one communication period at each layer, resulting in too much communication idle‐time on each layer. In this paper, we propose two algorithms which use pipelined communication with multi‐installments to solve the communication idle‐time problem and to achieve better performance. The closed form solutions to the function of parallel execution time and speedup for a d‐dimensional hypercube, based on the proposed methods, are also derived.

Performance modeling and analysis of correlated parallel computations

A performance analysis methodology for correlated parallel computations based on statistical theory is proposed. Divide-and-conquer strategy is widely used in solving problems in parallel by partitioning and allocating a number of given tasks to available computing resources. When the tasks exhibit run-time-dependent behaviors during execution and share a universal distribution function in their execution times, analysis of parallel execution time can be performed with the assistance of probabilistic and statistical models. Correlation (dependence) in execution times among tasks has posed a significant factor in influencing the analysis accuracy which is unmanageable by any known analysis methodologies. We establish a relation between a task’s or a processor’s execution time and the parallel execution time, in terms of expected value as well as variance when each task’s execution time can be closely modeled by a normal distribution, for either uncorrelated or correlated tasks. This relation is then applied to the modeling and analysis of various parallel computation paradigms in which different communication and synchronization patterns along the processing are present. The method proposed has a wider application scope and gives more accurate prediction results than previously known approaches. We also show that, as an extended application of the analysis method to a large scope of problems, load balance among processors can be vastly improved with some novel static task allocation technique in manipulating the correlation among tasks. Experimental results in analyzing a parallel tree search algorithm and two parallel sorting algorithms show very accurate analysis and prediction with the proposed method.

Parallel execution time prediction of the multitask parallel programs

A critical problem of predicting the execution time of parallel programs is computing the maximum execution time of tasks involved in the parallel computation. For a parallel composition of n tasks, distribution information of execution time of the constituent task is crucial to accurately predicting mean, variance and even distribution of execution time of the composition of tasks when the execution time of them is stochastic. This paper presents a method of predicting the maximum parallel execution time of n constituent tasks, each of which has independent, identically distributed random execution time. First, execution time of any constituent task as a random variable is transformed into standard normal variable, or approximately so, by using Johnson distributions. Then the distribution and moments of the maximum parallel execution time are obtained after appropriate Johnson transformation is chosen and its corresponding parameters are determined. The experiment on synthetic distributions such as exponential distribution shows that most relative errors of the first four moments are near the 0.1%. And the experiment on some practical applications shows that the relative errors of the first four moments are below 0.1%. The computing complexity of our algorithm, as a function of the number n of the tasks or processors, is O ( n ) , and even O ( 1 ) while using approximate distribution, such as Gumbel distribution.

A heterogeneity-aware approach to load balancing of computational tasks: a theoretical and simulation study

One of the distinct characteristics of computing platforms shared by multiple users such as a cluster and a computational grid is heterogeneity on each computer and/or among computers. Temporal heterogeneity refers to variation, along the time dimension, of computing power available for a task on a computer, and spatial heterogeneity represents the variation among computers. In minimizing the average parallel execution time of a target task on a spatially heterogeneous computing system, it is not optimal to distribute the target task linearly proportional to the average computing powers available on computers. In this paper, effects of the temporal and spatial heterogeneity on performance of a target task have been analyzed in terms of the mean and standard deviation of parallel execution time. Based on the analysis results, an approach to load balancing for minimizing the average parallel execution time of a target task is described. The proposed approach whose validity has been verified through simulation considers temporal and spatial heterogeneities in addition to the average computing power on each computer.

PRECONDITIONED PARALLEL BLOCK–JACOBI SVD ALGORITHM

We show experimentally, that the QR factorization with the complete column pivoting, optionally followed by the LQ factorization of the R-factor, can lead to a substantial decrease of the number of outer parallel iteration steps in the parallel block-Jacobi SVD algorithm, whereby the details depend on the condition number and on the shape of spectrum, including the multiplicity of singular values. Best results were achieved for well-conditioned matrices with a multiple minimal singular value, where the number of parallel iteration steps has been reduced by two orders of magnitude. However, the gain in speed, as measured by the total parallel execution time, depends decisively on how efficient is the implementation of the distributed QR and LQ factorizations on a given parallel architecture. In general, the reduction of the total parallel execution time up to one order of magnitude has been achieved.

Optimal Period of Workload Redistribution for Dynamic Bulk Synchronous Computations in Heterogeneous Computing Systems

In dynamic bulk synchronous computations, processors may change their workloads from phase to phase. Such workload change will possibly increase the duration of a phase and the overall parallel execution time. Therefore, it is necessary to redistribute workload at runtime to reduce the parallel time. However, such workload redistribution at runtime can be expensive and the overhead of frequent runtime workload redistribution may exceed the benefit of workload redistribution and balancing. The problem of finding the optimal period of runtime workload redistribution for dynamic bulk synchronous computations is motivated by the combined consideration of parallel execution time and system overhead for workload redistribution and balancing. We develop an analytical method to solve the problem in heterogeneous computing systems. We also demonstrate numerical data of the analytical method and simulation results that verify the analytical data.

Low-cost static performance prediction of parallel stochastic task compositions

Current analytic solutions to the execution time distribution of a parallel composition of tasks having stochastic execution times are computationally complex, except for a limited number of distributions. In this paper, we present an analytical solution based on approximating execution time distributions in terms of the first four statistical moments. This low-cost approach allows the parallel execution time distribution to be approximated at ultra-low solution complexity for a wide range of execution time distributions. The accuracy of our method is experimentally evaluated for synthetic distributions as well as for task execution time distributions found in real parallel programs and kernels (NAS-EP, SSSP, APSP, Splash2-Barnes, PSRS, and WATOR). Our experiments show that the prediction error of the mean value of the parallel execution time for N-ary parallel composition is in the order of percents, provided the task execution time distributions are sufficiently independent and unimodal.

Efficient pre-processing in the parallel block-Jacobi SVD algorithm

One way, how to speed up the computation of the singular value decomposition of a given matrix A ∈ C m × n , m ⩾ n , by the parallel two-sided block-Jacobi method, consists of applying some pre-processing steps that would concentrate the Frobenius norm near the diagonal. Such a concentration should hopefully lead to fewer outer parallel iteration steps needed for the convergence of the entire algorithm. It is shown experimentally, that the QR factorization with the complete column pivoting, optionally followed by the LQ factorization of the R-factor, can lead to a substantial decrease of the number of outer parallel iteration steps, whereby the details depend on the condition number and on the distribution of singular values including their multiplicity. A subset of ill-conditioned matrices has been identified, for which the dynamic ordering becomes inefficient. Best results in numerical experiments performed on the cluster of personal computers were achieved for well-conditioned matrices with a multiple minimal singular value, where the number of parallel iteration steps was reduced by two orders of magnitude. However, the gain in speed, as measured by the total parallel execution time, depends decisively on the implementation of the distributed QR and LQ factorizations on a given parallel architecture. In general, the reduction of the total parallel execution time up to one order of magnitude has been achieved.

A $\frac32$‐Approximation Algorithm for Scheduling Independent Monotonic Malleable Tasks

A malleable task is a computational unit that may be executed on any arbitrary number of processors, whose execution time depends on the amount of resources allotted to it. This paper presents a new approach for scheduling a set of independent malleable tasks which leads to a worst case guarantee of $\frac{3}{2}+\varepsilon$ for the minimization of the parallel execution time for any fixed $\varepsilon > 0$. The main idea of this approach is to focus on the determination of a good allotment and then to solve the resulting problem with a fixed number of processors by a simple scheduling algorithm. The first phase is based on a dual approximation technique where the allotment problem is expressed as a knapsack problem for partitioning the set of tasks into two shelves of respective heights $1$ and $\frac{1}{2}$.

PC 클러스터에서 스케줄링 기법의 구현

본 논문에서는 버스 기반의 클러스터 구조에 적합한 새로운 태스크 스케줄링 기법을 소개하고, PC 클러스터에 구현하여 스케줄링 기법의 성능을 분석한다. 구현된 스케줄링 기법은 태스크 그래프를 입력으로 받아 PC 클러스터로 스케줄링하며, 휴리스틱을 사용하여 태스크를 선택적으로 중복함으로써 병렬연산시간을 단축한다. PC 클러스터는 리눅스 OS가 설치된 6대의 PC가 Gigabit Ethernet으로 연결되어 있다. 통신을 위해 TCP/IP 프로토콜을 사용하며, 메시지 교환을 위해 표준화된 병렬 프로그래밍 도구로 MPI를 사용한다. 실험을 한 결과 본 논문에서 소개한 스케줄링 기법이 비교 기법보다 병렬연산시간 측면에서 성능이 우수함을 확인하였다. In this-paper, we propose a new task scheduling scheme for bus-based cluster architectures and analyze performance of the scheduling scheme which has been implemented in a PC cluster. The implemented scheme schedules the tasks of a task graph to the processors of a PC cluster, and it reduces parallel execution time by selectively duplicating critical tasks using heuristic. Experimental results show that the proposed scheduling scheme produces better parallel execution time than the other scheduling scheme.

AVERAGE-CASE SCALABILITY ANALYSIS OF PARALLEL COMPUTATIONS ON k-ARY d-CUBES

We investigate the average-case scalability of parallel algorithms executing on multicomputer systems whose static networks are k-ary d-cubes. Our performance metrics are isoefficiency function and isospeed scalability. For the purpose of average-case performance analysis, we formally define the concepts of average-case isoefficiency function and average-case isospeed scalability. By modeling parallel algorithms on multicomputers using task interaction graphs, we are mainly interested in the effects of communication overhead and load imbalance on the performance of parallel computations. We focus on the topology of static networks whose limited connectivities are constraints to high performance. In our probabilistic model, task computation and communication times are treated as random variables, so that we can analyze the average-case performance of parallel computations. We derive the expected parallel execution time on symmetric static networks and apply the result to k-ary d-cubes. We characterize the maximum tolerable communication overhead such that constant average-case efficiency and average-case average-speed could be maintained and that the number of tasks has a growth rate Θ(P log P), where P is the number of processors. It is found that the scalability of a parallel computation is essentially determined by the topology of a static network, i.e., the architecture of a parallel computer system. We also argue that under our probabilistic model, the number of tasks should grow at least in the rate of Θ(P log P), so that constant average-case efficiency and average-speed can be maintained.

On variable blocking factor in a parallel dynamic block––Jacobi SVD algorithm

The parallel two-sided block-Jacobi singular value decomposition (SVD) algorithm with dynamic ordering, originally proposed in [Parallel Comput. 28 (2002) 243–262], has been extended with respect to the blocking factor ℓ. Unlike the unique blocking factor ℓ=2 p in the original algorithm running on p processors, the current blocking factor is a variable parameter that covers the values in two different regions––namely, ℓ= p/ k and ℓ=2 kp for some integer k. Two new parallel two-sided block-Jacobi SVD algorithms with dynamic ordering are described in detail. They arise in those two regions and differ in the logical data arrangement and communication complexity of the reordering step. For the case of ℓ=2 kp, it is proved that a designed point-to-point communication algorithm is optimal with respect to the amount of communication required per processor as well as to the amount of overall communication. Using the message passing programming model for distributed memory machines, new parallel block-Jacobi SVD algorithms were implemented on an SGI-Cray Origin 2000 parallel computer. Numerical experiments were performed on p=12 and 24 processors using a set of six matrices of order 4000 and blocking factors ℓ, 2⩽ℓ⩽192. To achieve the minimal total parallel execution time, the use of a blocking factor ℓ∈{2, p,2 p} can be recommended for matrices with distinct singular values. However, for matrices with a multiple minimal singular value, the total parallel execution time may monotonically increase with ℓ. In this case, the recommended Jacobi method with ℓ=2 is just the ScaLAPACK routine with some additional matrix multiplications, and it computes the SVD in one parallel iteration step.

On the parallel execution time of tiled loops

Many computationally-intensive programs, such as those for differential equations, spatial interpolation, and dynamic programming, spend a large portion of their execution time in multiply-nested loops that have a regular stencil of data dependences. Tiling is a well-known compiler optimization that improves performance on such loops, particularly for computers with a multilevel hierarchy of parallelism and memory. Most previous work on tiling is limited in at least one of the following ways: they only handle nested loops of depth two, orthogonal tiling, or rectangular tiles. In our work, we tile loop nests of arbitrary depth using polyhedral tiles. We derive a prediction formula for the execution time of such tiled loops, which can be used by a compiler to automatically determine the tiling parameters that minimizes the execution time. We also explain the notion of rise, a measure of the relationship between the shape of the tiles and the shape of the iteration space generated by the loop nest. The rise is a powerful tool in predicting the execution time of a tiled loop. It allows us to reason about how the tiling affects the length of the longest path of dependent tiles, which is a measure of the execution time of a tiling. We use a model of the tiled iteration space that allows us to determine the length of the longest path of dependent tiles using linear programming. Using the rise, we derive a simple formula for the length of the longest path of dependent tiles in rectilinear iteration spaces, a subclass of the convex iteration spaces, and show how to choose the optimal tile shape.

A parallel 3D semiconductor device simulator for gradual heterojunction bipolar transistors

AbstractIn this paper, we present a parallel three‐dimensional semiconductor device simulator for gradual heterojunction bipolar transistor. This simulator uses the drift‐diffusion transport model. The Poisson equation and continuity equations were discretized using a finite element method (FEM) on an unstructured tetrahedral mesh. Fermi–Dirac statistics is considered in our model and a compact formulation is used that makes it easy to take into account other effects such as the non‐parabolic nature of the bands or the presence of various subbands in the conduction process. Domain decomposition methods were tested to solve the linear systems. We have applied this simulator to a gradual heterojunction bipolar transistor (HBT), and we present some measures of the parallel execution time for several solvers and some electrical results. This code has been implemented for distributed memory multicomputers, making use of the MPI message passing standard library and a parallel solver library. Copyright © 2002 John Wiley & Sons, Ltd.

A novel dynamic load balancing scheme for parallel systems

Adaptive mesh refinement (AMR) is a type of multiscale algorithm that achieves high resolution in localized regions of dynamic, multidimensional numerical simulations. One of the key issues related to AMR is dynamic load balancing (DLB), which allows large-scale adaptive applications to run efficiently on parallel systems. In this paper, we present an efficient DLB scheme for structured AMR (SAMR) applications. This scheme interleaves a grid-splitting technique with direct grid movements (e.g., direct movement from an overloaded processor to an underloaded processor), for which the objective is to efficiently redistribute workload among all the processors so as to reduce the parallel execution time. The potential benefits of our DLB scheme are examined by incorporating our techniques into a SAMR cosmology application, the ENZO code. Experiments show that by using our scheme, the parallel execution time can be reduced by up to 57% and the quality of load balancing can be improved by a factor of six, as compared to the original DLB scheme used in ENZO.

A Simple Method for Dynamic Scheduling in a Heterogeneous Computing System

A simple method for the dynamic scheduling on a heterogeneous computing system is proposed in this paper. It was implemented to minimize the parallel program execution time. The proposed method decomposes the program workload into computationally homogeneous subtasks, which may be of the different size, depending on the current load of each machine in a heterogeneous computing system.

Parallel FD-TD Simulation of Radiobase Antennae

The rigorous characterisation of the behaviour of a radiobase antenna for wireless communication systems is a hot topic both for antenna or communication system design and for radioprotection-hazard reasons. Such a characterisation deserves a numerical solution, and the use of a finite-difference time-domain (FD-TD) approach is an attractive candidate. Unfortunately, it has strong memory and CPU-time requirements. Numerical complexity can be successfully afforded by using parallel computing. The parallel implementation of the FD-TD code, individuating the theoretical lower bound for its parallel execution time are discussed and the findings achieved on the APE/Quadrics massively parallel systems are presented. Results obtained from the simulation of actual radiobase antennae, clearly demonstrate that massively parallel processing is a viable approach to solving electromagnetic problems, allowing the simulation of radiating devices which could not be modelled through conventional computing systems. The tests showed a sustained computational speed equal to 17% of the theoretical maximum.