Number Of Available Processors Research Articles

Parallel job scheduling for power constrained HPC systems

Power has become the primary constraint in high performance computing. Traditionally, parallel job scheduling policies have been designed to improve certain job performance metrics when scheduling parallel workloads on a system with a given number of processors. The available number of processors is not anymore the only limitation in parallel job scheduling. The recent increase in processor power consumption has resulted in a new limitation: the available power. Given constraints naturally lead to an optimization problem. We proposed MaxJobPerf, a new parallel job scheduling policy based on integer linear programming. Dynamic Voltage Frequency Scaling (DVFS) is a widely used technique that running applications at reduced CPU frequency/voltage trades increased execution time for power reduction. The optimization problem determines which jobs should run and at which frequency. In this paper, we compare the MaxJobPerf policy against other power budgeting policies for different power budgets. It clearly outperforms the other power-budgeting approaches at the parallel job scheduling level. Furthermore, we give a detailed analysis of the policy parameters including a discussion on how to manage job reservations to avoid job starvation.

Benchmarking the clustering algorithms for multiprocessor environments using dynamic priority of modules

In this paper we give some extensive benchmark results for some dynamic priority clustering algorithms for homogeneous multiprocessor environments. By dynamic priority we mean a priority function that can change with every step of the algorithm. Using dynamic priority can give us more flexibility as compared to static priority algorithms. Our objective in this paper is to compare the dynamic priority algorithms with some well known algorithms from the literature and discuss their strengths and weaknesses. For our study we have selected two recently proposed dynamic priority algorithms: CPPS (Cluster Pair Priority Scheduling algorithm) having complexity O(|V||E|(|V|+|E|)) and DCCL (Dynamic Computation Communication Load scheduling algorithm) having complexity O(|V|2(|V|+|E|)log(|V|+|E|)) where |V| is the number of nodes in the task graph, and |E| is the number of edges in the task graph. We have selected a recently proposed randomized algorithm with static priority (RCCL: Randomized Computation Communication Load scheduling algorithm) and converted it into a dynamic priority algorithm: RDCC (Randomized Dynamic Computation Communication load scheduling algorithm) having complexity O(ab|V|(|V|+|E|)log(|V|+|E|)) where a is the number of randomization steps, and b is a limit on the number of clusters formed. We have also selected three well known algorithms from literature: DSC (Dominant Sequence Clustering algorithm) having complexity O((|V|+|E|)log(|V|)), EZ (Edge Zeroing algorithm) having complexity O(|E|(|V|+|E|)), and LC (Linear Clustering algorithm) having complexity O(|V|(|V|+|E|)). We have compared these algorithms using various comparison parameters including some statistical parameters, and also using various types of task graphs including some synthetic and real task graphs. Our results show that the dynamic priority algorithms give best results for the case of random task graphs, and for the case when the number of available processors are small.

Energy aware DAG scheduling on heterogeneous systems

We address the problem of scheduling directed a-cyclic task graph (DAG) on a heterogeneous distributed processor system with the twin objectives of minimizing finish time and energy consumption. Previous scheduling heuristics have assigned DAGs to processors to minimize overall run-time of the application. But applications on embedded systems, such as high performance DSP in image processing, multimedia, and wireless security, need schedules which use low energy too. We develop a new scheduling algorithm called Energy Aware DAG Scheduling (EADAGS) on heterogeneous processors that can run on discrete operating voltages. Such processors can scale down their voltages and slow down to reduce energy whenever they idle due to task dependencies. EADAGS combines dynamic voltage scaling (DVS) with Decisive Path Scheduling (DPS) to achieve the twin objectives. Using simulations we show average energy consumption reduction over DPS by 40%. Energy savings increased with increasing number of nodes or increasing Communication to Computation Ratios and decreased with increasing parallelism or increasing number of available processors. These results were based on a software simulation study over a large set of randomly generated graphs as well as graphs for real-world problems with various characteristics.

Enhancing the Schedulability of Real-Time Heterogeneous Networks of Workstations (NOWs)

This paper proposes a realtime duplication based algorithm (RT-DBA) for scheduling precedence-related periodic tasks with hard deadlines on networks of workstations (NOWs). We have utilized selective subtask duplication that enables some tasks to have earlier start times, which enables additional tasks (and, hence, task sets) to finish before their deadlines, thereby increasing the schedulability of a real-time application. We strongly believe that duplication can be used as a tool for obtaining a better quality of service (QoS) from the real-time heterogeneous system, and this is our major contribution. We have taken both the computation and the communication heterogeneities into account while modeling such a system. Both data and control dependencies between the tasks have also been considered. Our algorithm exhibits scalability, fully exploits the underlying parallelism, and is capable of scheduling an application, even if the available number of processors is less than the required number of processors. Based on extensive simulation studies, we observe that RT-DBA offers an enhanced success ratio as compared to other scheduling schemes when communication is a dominant factor.

Porosity and permeability estimation by integration of production and time-lapse near and far offset seismic data

This study presents a method based on the Gauss–Newton optimization technique for continuous reservoir model updating with respect to production history and time-lapse seismic data in the form of zero offset amplitudes and amplitude versus offset (AVO) gradients. The main objective of the study is to test the feasibility of using these integrated data as input to reservoir parameter estimation problems. Using only production data or zero offset time-lapse seismic amplitudes as observation data in the parameter estimation process cannot properly limit the solution space. The emphasis of this work is to use the integrated data combined with empirical knowledge about rock types from laboratory measurements, to further constrain the inversion process. The algorithm written for this study consists of three parts: the reservoir simulator, the rock physics petro-elastic model and the optimization algorithm. The Gauss–Newton inversion is tested at a 2D semi-synthetic model inspired by real field data from offshore Norway. The algorithm reduces the misfit between the observed and simulated data which make it possible to estimate porosity and permeability distributions. The Gauss–Newton optimization technique is an efficient parameter estimation technique. However, the numerical estimation of the gradient is time consuming, and it can be prohibitive for practical applications. This method is suitable for distributed computing which considerably reduces the total optimization time. The amount of reduction depends mainly on the number of available processors.

A Heapify Based Parallel Sorting Algorithm

Quick sort is a sorting algorithm whose worst case running time is θ(n2 ) on an input array of n numbers. It is the best practical for sorting because it has the advantage of sorting in place. Problem statement: Behavior of quick sort is complex, we proposed in-place 2m threads parallel heap sort algorithm which had advantage in sorting in place and had better performance than classical sequential quick sort in running time. Approach: The algorithm consisted of several stages, in first stage; it splits input data into two partitions, next stages it did the same partitioning for prior stage which had been spitted until 2 m partitions was reached equal to the number of available processors, finally it used heap sort to sort respectively ordered of non internally sorted partitions in parallel. Results: Results showed the speed of algorithm about double speed of classical Quick sort for a large input size. The number of comparisons needed was reduced significantly. Conclusion: In this study we had been proposed a sorting algorithm that uses less number of comparisons with respect to original quick sort that in turn requires less running time to sort the same input data.

DIVISIBLE LOAD DISTRIBUTION IN A NETWORK OF PROCESSORS

The paper presents a closed form solution for an optimum scheduling of a divisible job on an optimum number of processor arranged in an optimum sequence in a multilevel tree networks. The solution has been derived for a single divisible job where there is no dependency among subtasks and the root processor can either perform communication and computation at the same time. The solution is carried out through three basic theorems. One of the theorems selects the optimum number of available processors that must participate in executing a divisible job. The other solves the sequencing problem in load distribution by which we are able to find the optimum sequence for load distribution in a generalized form. Having the optimum number of processors and their sequencing for load distribution, we have developed a closed form solution that determines the optimum share of each processor in the sequence such that the finish time is minimized. Any alteration of the number of processors, their sequences, or their shares that are determined by the three theorems will increase the finish time.

The Treeterbi and Parallel Treeterbi algorithms: efficient, optimal decoding for ordinary, generalized and pair HMMs

Hidden Markov models (HMMs) and generalized HMMs been successfully applied to many problems, but the standard Viterbi algorithm for computing the most probable interpretation of an input sequence (known as decoding) requires memory proportional to the length of the sequence, which can be prohibitive. Existing approaches to reducing memory usage either sacrifice optimality or trade increased running time for reduced memory. We developed two novel decoding algorithms, Treeterbi and Parallel Treeterbi, and implemented them in the TWINSCAN/N-SCAN gene-prediction system. The worst case asymptotic space and time are the same as for standard Viterbi, but in practice, Treeterbi optimally decodes arbitrarily long sequences with generalized HMMs in bounded memory without increasing running time. Parallel Treeterbi uses the same ideas to split optimal decoding across processors, dividing latency to completion by approximately the number of available processors with constant average overhead per processor. Using these algorithms, we were able to optimally decode all human chromosomes with N-SCAN, which increased its accuracy relative to heuristic solutions. We also implemented Treeterbi for Pairagon, our pair HMM based cDNA-to-genome aligner. The TWINSCAN/N-SCAN/PAIRAGON open source software package is available from http://genes.cse.wustl.edu.

Discretization and parallel performance of an unstructured finite volume Navier–Stokes solver

AbstractThe discretization, parallelization and performance of an implicit, unstructured, time‐dependent Computational Fluid Dynamics code is described. A detailed description is provided of the improvements made on second‐order accurate tools for spatial interpolation and gradient calculation to discretize the Navier–Stokes equations in an unstructured framework. The main goal in the development of the discretization tools was to ensure a scalable and accurate parallel code. The performance of the discretization tools has been validated using standard bench‐mark problems for non‐uniform, non‐orthogonal grids. Parallelization of the code is done within the PETSc framework using a single‐program‐multiple‐data (SPMD) parallelization model. The resulting parallel code is shown to scale linearly within the limit of the available number of processors. Copyright © 2006 John Wiley & Sons, Ltd.

Adaptive parallel decomposition for multidisciplinary design

The conceptual design of a rotorcraft system involves many different analysis disciplines. The decomposition of such a system into several subsystems can make analysis and design more efficient in terms of the total computation time. Adaptive parallel decomposition makes the structure of the overall design problem suitable to apply the multidisciplinary design optimization methodologies and it can exploit parallel computing. This study proposes a decomposition method which adaptively determines the number and sequence of analyses in each sub-problem corresponding to the available number of processors in parallel. A rotorcraft design problem is solved and as a result, the adaptive parallel decomposition method shows better performance than other previous methods for the selected design problem.

Subdomain cluster generation for domain decomposition methods using graph partitioning optimization

Balancing and dual domain decomposition methods (DDMs) comprise a family of efficient high performance solution approaches for a large number of problems in computational mechanics. Such DDMs are used in practice on parallel computing environments with the number of generated subdomains being generally larger than the number of available processors. This paper presents an effective heuristic technique for organizing the subdomains into subdomain clusters, in order to assign each cluster to a processor. This task is handled by the proposed approach as a graph partitioning optimization problem using the publicly available software METIS. The objective of the optimization process is to minimize the communication requirements of the DDMs under the constraint of producing balanced processor workloads. This constraint optimization procedure for treating the subdomain cluster generation task leads to increased computational efficiencies for balancing and dual DDMs.

Fast parallel algorithm for distance transform

We present an O((log log N)/sup 2/) -time algorithm for computing the distance transform of an N /spl times/ N binary image. Our algorithm is designed for the common concurrent read concurrent write parallel random access machine (CRCW PRAM) and requires O(N/sup 2+/spl epsi///log log N) processors, for any /spl epsi/ such that 0 < /spl epsi/ < 1. Our algorithm is based on a novel deterministic sampling scheme and can be used for computing distance transforms for a very general class of distance functions. We also present a scalable version of our algorithm when the number of processors is available p/sup 2+/spl epsi///log log p for some p < N. In this case, our algorithm runs in O((N/sup 2//p/sup 2/)+(N/p) log log p + (log log p)/sup 2/) time. This scalable algorithm is more practical since usually the number of available processors is much less than the size of the image.

Developing SPMD applications with load balancing

The central contribution of this work is SAMBA ( S ingle A pplication, M ultiple Load Ba lancing), a framework for the development of parallel SPMD (single program, multiple data) applications with load balancing. This framework models the structure and the characteristics common to different SPMD applications and supports their development. SAMBA also contains a library of load balancing algorithms. This environment allows the developer to focus on the specific problem at hand. Special emphasis is given to the identification of appropriate load balancing strategies for each application. Three different case studies were used to validate the functionality of the framework: matrix multiplication, numerical integration, and a genetic algorithm. These applications illustrate its ease of use and the relevance of load balancing. Their choice was oriented by the different load imbalance factors they present and by their different task creation mechanisms. The computational experiments reported for these case studies made possible the validation of SAMBA and the comparison, without additional reprogramming costs, of different load balancing strategies for each of them. The numerical results and the elapsed times measurements show the importance of using an appropriate load balancing algorithm and the associated reductions that can be achieved in the elapsed times. They also illustrate that the most suitable load balancing strategy may vary with the type of application and with the number of available processors. Besides the support to the development of SPMD applications, the facilities offered by SAMBA in terms of load balancing play also an important role in terms of the development of efficient parallel implementations.

Formula omitted]: visualizing the performance prediction of parallel iterative solvers

The selection of the best method and preconditioner for solving a sparse linear system is as determinant as the efficient parallelization of the selected method. We propose a tool for helping to solve both problems on distributed memory multiprocessors using iterative methods. Based on a previously developed library of HPF and message-passing interface (MPI) codes, a performance prediction is developed and a visualization tool (AVISPA) is proposed. The tool combines theoretical features of the methods and preconditioners with practical considerations and predictions about aspects of the execution performance (computational cost, communications overhead, etc.). It offers detailed information about all the topics that can be useful for selecting the most suitable method and preconditioner. Another capability is to offer information on different parallel implementations of the code (HPF and MPI) varying the number of available processors.

Parallel Computation of Fully Coupled Hypersonic Radiating Flowfield Using Multiband Model

Parallel computation of a fully coupled, strongly radiating hypersonic flowfleld is carried out. A detailed multiband model is used in the radiation calculation. The radiative heat flux is calculated using one- (tangent-slab) or two-dimensional approximations in radiative transfer, or by considering three-dimensional radiative transfer directly. To reduce the vast computing time due to a spectrally detailed and multidimensional radiation calculation, a parallel computation is employed. The strategy in the parallel implementation of the code is to divide the wavelength range in the multiband model into groups of the same number of available processors, instead of dividing the computational domain. Calculations are carried out for the flowfleld over hemispheres at a speed of 15.24 km/s and an altitude of 57.9 km. The computed results are compared with those obtained in previous studies. A fair agreement of the shock standoff distance with the existing results is shown for several different radii. However, the present result gives a substantially larger radiative heat flux value at the stagnation point for smaller radius cases

On time optimal supernode shape

With the objective of minimizing the total execution time of a parallel program on a distributed memory parallel computer, this paper discusses the selection of an optimal supernode shape of a supernode transformation (also known as tiling). We identify three parameters of a supernode transformation: supernode size, relative side lengths, and cutting hyperplane directions. For supernode transformations on algorithms with perfectly nested loops and uniform dependencies, we prove the optimality of a constant linear schedule vector and give a necessary and sufficient condition for optimal relative side lengths. We also prove that the total running time is minimized by a cutting hyperplane direction matrix from a particular subset of all valid directions and we discuss the cases where this subset is unique. The results are derived in continuous space and should be considered approximate. Our model does not include cache effects and assumes an unbounded number of available processors, the communication cost approximated by a constant, uniform dependences, and loop bounds known at compile time. A comprehensive example is discussed with an application of the results to the Jacobi algorithm.

A shelf-based Lagrangian relaxation algorithm to schedule parallelizable tasks

This paper generalizes classical scheduling theory by removing the restriction that only a single processor can work on a given task at a particular time. Instead, it is assumed that each task can be allocated any number of identical processors from one to the maximum number available, and that a task's completion time is a nonincreasing function of the number of processors allocated. Once started, a task must run to completion without altering the number of processors given to it. Furthermore, a task can start only when no task is currently executing. The objective considered is minimization of overall completion time (make-span) for the tasks subject to the constraint of a limited number of available processors. To approximately solve this problem, an algorithm based on Lagrangian relaxation is developed, and its performance is analyzed through extensive simulations. Approximate duality gaps range from 0 to about 60% on average and are strongly a function of problem type and size.

Distributed Simulation of Freeway Traffic Flow Using Personal Computers

This article presents a personal computer (PC)-based distributed simulation methodology applicable to real-time estimation of time-variant parameters in freeway traffic flow. The method uses the interprocess communication procedures currently available under Windows NT and adopts 2 types of distributed computing architectures. The proposed method, based on macroscopic traffic models developed and tested in prior studies, first divides a given freeway into a set of subsections depending on the number of available processors or PCs. At every time step, each processor simulates its own subsection and exchanges the internal boundary data with another processor that simulates the adjacent subsection. The performance of the parallel algorithm was compared with that of a single-processor-based sequential simulation using 2 different types of hardware configurations: a network of 2 PCs and a dual-processor PC. Test results with the 2-PC network parallel system showed a speedup of 1.94 over the sequential simulation with a single PC for the 1-hour simulation of a 20-mile freeway section. The speedup for 1-hour simulation of an 80-mile section was increased to 1.96 with 2 processors, which is close to the ideal speedup. Performance tests with the dual-processor PC resulted in consistent, but not significantly faster, execution than those with the 2-PC network, indicating the feasibility of developing a practical on-line decision support system with a network of PCs for freeway operations.

A flexible multicomputer algorithm for elementary matrix operations

In the present study we introduce and test a new flexible multicomputer (FM) algorithm for matrix calculations on a distributed memory computer. The FM-algorithm also performs matrix addition, subtraction, and scalar multiplication on both dense and sparse matrices. The FM-algorithm was designed to meet the need for a high-performance flexible software tool for implementing different parallel optimization algorithms. Special consideration has been taken to ensure the usability and portability of the algorithm. A preliminary flexibility test is conducted on an IBM SP2 (Cactus) machine. On the principal level, we will compare the FM-algorithm with another high-performance algorithm Summa and look at an improvement of Summa by combining it with the Strassen algorithm. On the empirical level, we will compare a chained version of the FM-algorithm with the parallel ScaLAPACK code in a set of huge matrix multiplications on a Cray T3E machine. Our results demonstrate that the FM-algorithm performs as well as the parallel ScaLAPACK code for dense matrices. FM is fully scalable for large, sparse matrices. The FM-algorithm is efficient with respect to sequential matrix multiplication. In contrast to ScaLAPACK, the fully scalable FM-algorithm is independent of mesh structure. Arbitrarily large matrices can be processed with a single processor. Scope and purpose In the paper we introduce a flexible multicomputer (FM) algorithm for elementary matrix calculations on a distributed memory computer. The algorithm is fully scalable in matrix multiplication, as shown in the study. The algorithm works as fast as the ScaLAPACK parallel matrix multiplication algorithm on a Cray T3E machine. A distinctive feature of the FM-algorithm is its independence of the number of available processors: the algorithm can process arbitrarily large matrices already on a single processor, contrary to the ScaLapack code. The reason for this possibility is that the original huge input matrices need not be represented in the CPU of the nodes. The FM-algorithm includes easy-to-use facilities for sparse matrix computations, in which blocks of zeros are eliminated from the calculation and communication tasks. The sparse matrix calculation is based on an elimination technique, where zero blocks in one input matrix and the corresponding block pairs from the other are excluded from data transmission and/or computations. The technique allows easy cell representation and facilitates information passing between nodes significantly. In comparison, the usual technique of representing each matrix cell by three numbers (the position coordinates and the cell value), leads to complicated data structures to be recognized and updated during computation. The possibility to represent specific nonzero patterns exactly, takes place at the expense of considerable communication overhead and, in most cases, little practical gain. The algorithm provides new avenues for high-performance computing, e.g., in the field of mathematical programming, where elementary matrix operations of huge dimensions are involved when solving intricate problems in engineering and economics. The FM-algorithm was designed both for architectures with shared disk memory and for architectures, where the data has to be transmitted between the processors. Naturally, in the latter system, the scalability of any matrix processing code is significantly reduced or eliminated since the communication overhead increases drastically with the problem size. However, even in this situation, the FM-algorithm needs only limited communication between nodes.

A hybrid parallelizable low-order algorithm for dynamics of multi-rigid-body systems: Part I, chain systems

This paper presents a new hybrid parallelizable O( n) algorithm for the modeling of the dynamic behavior of multi-rigid-body chain systems. The method is based on the cutting of certain system interbody joints so that largely independent multibody subchain systems are formed. These subchains in turn interact with one another through associated unknown constraint forces f c at the cut joints. The underlying feature of this new hybrid low-order algorithm is the increased coarse grain parallelism obtained through cutting the joints and explicit determination of associated constraint loads combined with a sequential O( n) procedure. In other words, sequential O( n) procedures are performed to form and solve equations of motion within subchains, and parallel strategies are used to form and solve constraint equations between subchains in parallel. It is shown that the resulting algorithm is coarse grain parallelizable up to the total number of bodies in the system. Also, the algorithm can easily accommodate the available number of processors while maintaining high efficiency. An O[( n + m)/ N p + m (1+ γ) / N p + m γ Iog 2 N p ] (0 < γ < 1) performance will be achieved with N p processors for a chain system with n degrees of freedom and m constraints due to cutting of interbody joints.