Interprocessor Communication Delays Research Articles

An efficient multi-functional duplication-based scheduling framework for multiprocessor systems

Timing performance and energy consumption are the most critical performance indicators of schedules on multiprocessors. The timing performance of the schedule is limited by the inter-processor communication delay incurred by inter-task data dependence, especially for communication-intensive applications. To mitigate the negative impact of the communication delay on timing performance of the schedule, task duplication strategy, which is based on the fact that the local communication is delay-free, is often utilized. The duplication-based scheduling problem is NP-hard, in terms of optimizing both the schedule latency and energy consumption. However, obtaining the optimal solution is still useful, e.g., help to implement performance-critical system or give insights into the problem. While available works focus on limited performance metrics of the problem and have high time complexity, this paper proposes an efficient and multi-functional duplication-based scheduling framework based on the satisfiability modulo theory with boolean and integer theories, which enables solving the duplication-based scheduling problems quickly; besides, strategies for modeling the non-preemptive condition in static schedule are studied to improve the efficiency of the proposed framework. The proposed framework is applied to optimize different objectives, including the schedule length, the energy consumption, etc. The proposed framework is tested on a set of applications and platforms. The experimental results demonstrate that the proposed method is more than 10 times quicker than the available method and can reduce the energy consumption by up to 70%.

On the Convergence Rate of Distributed Gradient Methods for Finite-Sum Optimization under Communication Delays

Motivated by applications in machine learning and statistics, we study distributed optimization problems over a network of processors, where the goal is to optimize a global objective composed of a sum of local functions. In these problems, due to the large scale of the data sets, the data and computation must be distributed over multiple processors resulting in the need for distributed algorithms. In this paper, we consider a popular distributed gradient-based consensus algorithm, which only requires local computation and communication. An important problem in this area is to analyze the convergence rate of such algorithms in the presence of communication delays that are inevitable in distributed systems. We prove the convergence of the gradient-based consensus algorithm in the presence of uniform, but possibly arbitrarily large, communication delays between the processors. Moreover, we obtain an upper bound on the rate of convergence of the algorithm as a function of the network size, topology, and the inter-processor communication delays.

On the Convergence Rate of Distributed Gradient Methods for Finite-Sum Optimization under Communication Delays

Motivated by applications in machine learning and statistics, we study distributed optimization problems over a network of processors, where the goal is to optimize a global objective composed of a sum of local functions. In these problems, due to the large scale of the data sets, the data and computation must be distributed over processors resulting in the need for distributed algorithms. In this paper, we consider a popular distributed gradient-based consensus algorithm, which only requires local computation and communication. An important problem in this area is to analyze the convergence rate of such algorithms in the presence of communication delays that are inevitable in distributed systems. We prove the convergence of the gradient-based consensus algorithm in the presence of uniform, but possibly arbitrarily large, communication delays between the processors. Moreover, we obtain an upper bound on the rate of convergence of the algorithm as a function of the network size, topology, and the inter-processor communication delays.

Multiprocessor Scheduling by Simulated Evolution

This paper presents a variant of simulated evolution technique for the static non-preemptive scheduling of parallel programs represented by directed acyclic graphs including inter-processor communication delays and contention onto a multiprocessor system with the dual objectives of reducing the total execution time and scaling with the number of processors. The premise of our algorithm is Simulated Evolution, an effective optimization method based on the analogy with the natural selection process of biological evolution. The proposed technique, named Scheduling with Simulated Evolution (SES), combines simulated evolution with list scheduling, wherein simulated evolution efficiently determines suitable priorities which lead to a good solution by applying list scheduling as a decoding heuristic. SES is an effective method that yields reduced length schedules while scaling well and incurring reasonably low complexity. The SES technique does not require problem-specific parameter tuning on test problems of different sizes and structures. Moreover, it strikes a balance between exploration of the search space and exploitation of good solutions found in an acceptable CPU time. We demonstrate the effectiveness of SES by comparing it against two existing static scheduling techniques for the test examples reported in literature and on a suite of randomly generated graphs. The proposed technique produced good quality solutions with a slight increase in the CPU time as compared with the competing techniques.

Provable algorithms for parallel generalized sweep scheduling

We present provably efficient parallel algorithms for sweep scheduling, which is a commonly used technique in radiation transport problems, and involves inverting an operator by iteratively sweeping across a mesh from multiple directions. Each sweep involves solving the operator locally at each cell. However, each direction induces a partial order in which this computation can proceed. On a distributed computing system, the goal is to schedule the computation, so that the length of the schedule is minimized. Due to efficiency and coupling considerations, we have an additional constraint, namely, a mesh cell must be processed on the same processor along each direction. Problems similar in nature to sweep scheduling arise in several other applications, and here we formulate a combinatorial generalization of this problem that captures the sweep scheduling constraints,and call it the generalized sweep scheduling problem. Several heuristics have been proposed for this problem; see [S. Pautz, An algorithm for parallel S n sweeps on unstructured meshes, Nucl. Sci. Eng. 140 (2002) 111–136; S. Plimpton, B. Hendrickson, S. Burns, W. McLendon, Parallel algorithms for radiation transport on unstructured grids, Super Comput. (2001)] and the references therein; but none of these have provable worst case performance guarantees. Here we present a simple, almost linear time randomized algorithm for the generalized sweep scheduling problem that (provably) gives a schedule of length at most O ( log 2 n ) times the optimal schedule for instances with n cells, when the communication cost is not considered, and a slight variant, which coupled with a much more careful analysis, gives a schedule of (expected) length O ( log m log log log m ) times the optimal schedule for m processors. These are the first such provable guarantees for this problem. The algorithm can be extended with an additional multiplicative factor in the case when we have inter-processor communication latency, in the models of Rayward-Smith [UET scheduling with inter-processor communication delays, Discrete Appl. Math. 18 (1) (1987) 55–71] and Hwang et al. [Scheduling precedence graphs in systems with inter-processor communication times, SIAM J. Comput. 18(2) (1989) 244–257]. Our algorithms are extremely simple, and use no geometric information about the mesh; therefore, these techniques are likely to be applicable in more general settings. We also design a priority based list schedule using these ideas, with the same theoretical guarantee, but much better performance in practice; combining this algorithm with a simple block decomposition also lowers the overall communication cost significantly. Finally, we perform a detailed experimental analysis of our algorithm. Our results indicate that the algorithm compares favorably with the length of the schedule produced by other natural and efficient parallel algorithms proposed in the literature [S. Pautz, An Algorithm for parallel S n sweeps on unstructured meshes, Nucl. Sci. Eng. 140 (2002) 111–136; S. Plimpton, B. Hendrickson, S. Burns, W. McLendon, Parallel algorithms for radiation transport on unstructured grids, Super Comput. (2001)].

Comparing the Optimal Performance of Parallel Architectures

Consider a parallel program with n processes and a synchronization granularity z. Consider also two parallel architectures: an SMP with q processors and run-time reallocation of processes to processors, and a distributed system (or cluster) with k processors and no run-time reallocation. There is an inter-processor communication delay of t time units for the system with no run-time reallocation. In this paper we define a function H(n,k,q,t,z) such that the minimum completion time for all programs with n processes and a granularity z is at most H(n,k,q,t,z) times longer using the system with no reallocation and k processors compared to using the system with q processors and run-time reallocation. We assume optimal allocation and scheduling of processes to processors. The function H(n,k,q,t,z)is optimal in the sense that there is at least one program, with n processes and a granularity z, such that the ratio is exactly H(n,k,q,t,z). We also validate our results using measurements on distributed and multiprocessor Sun/Solaris environments. The function H(n,k,q,t,z) provides important insights regarding the performance implications of the fundamental design decision of whether to allow run-time reallocation of processes or not. These insights can be used when doing the proper cost/benefit trade-offs when designing parallel execution platforms.

A linear time algorithm for scheduling outforests with communication delays on three processors

We consider the problem of scheduling n unit-length tasks on identical m parallel processors, when outforest precedence relations and unit interprocessor communication delays exist. Two algorithms have been proposed in the literature for the exact solution of this problem: a linear time algorithm for the special case m=2, and a dynamic programming algorithm which runs in O( n 2 m−2 ). In this paper we give a new linear time algorithm for instances with m=3.

Upper bound on the number of processors for scheduling with interprocessor communication delays

The problem of scheduling a task system with communication delays on multiprocessor systems is known to be NP-hard in its general form as well as many restricted cases even on an unlimited number of processors. In this paper, we study the problem of determining an upper bound on the minimum number of processors achieved by a schedule that minimizes the makespan for scheduling problems with communication delays. We prove that the minimum number of partitioning paths of the precedence graph is an upper bound on the minimum number of processors for UET-UCT (Unit Execution Time-Unit Communication Time) task systems. Then we propose an algorithm of O(n) (n designates the number of tasks) to compute an upper bound, which is valid independently of task processing times and communication delays, in the special case when the precedence graph is an out-tree or an in-tree.

Structure theorems for partially asynchronous iterations of a nonnegative matrix with random delays

We consider partially asynchronous parallel iteration of a fixed nonnegative matrix with stationary ergodic interprocessor communication delays. We study the iteration via a random graph describing the interprocessor influences. Our major result is an invariant description of the rates of convergence of arbitrary sequences of individual processor-time values. In the course of proving this result a number of other invariant properties of the convergence of the iteration are described. The convergence rates that appear in our results are Lyapunov exponents of certain random matrix products derived from the original matrix and the statistics of the delays.

On the minimum number of processors for scheduling problems with communicationdelays

Scheduling problems with interprocessor communication delays are recent problemsarising with the development of new message‐passing architectures whose number of processorsis increasing more and more. The scheduling problem with communication delays isNP‐complete even on an unlimited number of processors, and most of the restrictions whichare known to make the problem easy suppose that the number of processors is unlimited.The aim of this paper is to estimate the minimum number of processors required to realizea schedule that minimizes the makespan for problems with communication delays. Contraryto problems without communication costs, we show that the minimum number of partitioningpaths of tasks is not an upper bound. Then we propose two upper bounds b and b whichare valid independent of task processing times and communication costs. We show thatb can be calculated in O(n 2 ) if n designates the number of tasks. Then we give an algorithmfor determining b in the special case when the precedence graph is an in‐tree or anout-tree. A specific study of SCT task systems (Small Communication Time) shows that theachromatic number of the cocomparability graph is an upper bound on the minimum numberof processors.

The Multiprocessor Scheduling of Precedence-Constrained Task Systems in the Presence of Interprocessor Communication Delays

The problem of scheduling precedence-constrained task systems characterized by interprocessor communication delays is addressed. It is assumed that task duplication is permitted. The target machine is a homogenous multiprocessor with an unbounded number of processors. The general problem is known to be NP-hard; however, when communication delays are small relative to task execution times, the C.P.M. based approach of Colin and Chrétienne (1991) yields an optimal schedule in polynomial time. Extensions to this method of Colin and Chrétienne are presented here, which allow for polynomial-time optimal schedule generation for certain categories of task systems with arbitrary precedence relations, processing times, and communication delays.

A Heuristic for a Scheduling Problem with Communication Delays

This paper addresses a scheduling problem with interprocessor communication delays: the jobs and the communication delays are of unit length. The number of processors is unbounded. The aim is to minimize the makespan. We develop a new list scheduling heuristic and we prove that its worst-case relative performance is 4/3.

The Complexity of Scheduling Trees with Communication Delays

We consider the problem of finding a minimum-length schedule onmmachines for a set ofnunit-length tasks with a forest of intrees as precedence relation, and with unit interprocessor communication delays. First, we prove that this problem is NP-complete; second, we derive a linear time algorithm for the special case thatmequals 2.

Comparative study of task duplication static scheduling versus clustering and non‐clustering techniques

AbstractOne of the major issues that needs to be addressed in distributed memory multiprocessor (DMM) systems is the program task partitioning and scheduling problems, i.e. mapping of an application program's precedence related task threads among the processing elements of a DMM system. The optimal task partitioning and scheduling problem, with the goal of minimizing the program execution time and interprocessor communication overhead, is known to be an NP‐complete problem. The paper addresses the design, development and performance evaluation of a novel static task partitioning and scheduling method called linear clustering with task duplication (LCTD). LCTD employs the linear (sequential) execution of tasks and task duplication heuristics in achieving minimized computation and interprocessor communication delays in DMMs. The superiority of the proposed LCTD algorithm is demonstrated through simulation studies and comparison against several of the existing static scheduling schemes, such as heavy node first (HNF) and linear clustering. We show that the proposed method can obtain an average of 33% improvement in program execution time and 21% improvement in processor utilization compared to linear clustering and HNF methods.

New complexity results on scheduling with small communication delays

Although most of the scheduling problems with interprocessor communication delays have been shown to be NP-complete, some important special cases were still unsolved. This paper deals with the problem where communication times are smaller than processing times and task duplication is not allowed. We prove that this problem is NP-complete and we give an efficient approximate algorithm with performance guarantee.

Fully static multiprocessor array realizability criteria for real-time recurrent DSP applications

The paper considers real time implementation of recurrent digital signal processing algorithms on an application-specific multiprocessor system. The objective is to devise a periodic, fully static task assignment for a DSP algorithm under the constraint of data sampling period by assuming interprocessor communication delay is negligible. Toward this goal, the authors propose a novel algorithm unfolding technique called the generalized perfect rate graph (GPRG). They prove that a recurrent algorithm will admit a fully static multiprocessor implementation for a given initiation interval if and only if the corresponding iterative computational dependence graph of this algorithm is a GPRG. Compared with previous results, GPRG often leads to a smaller unfolding factor /spl alpha//sub GPRG/.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Tree scheduling with communication delays

This paper considers the problem of scheduling a tree-structured task system in a distributed environment with the goal of minimizing the makespan. Interprocessor communication delays are taken into account and task duplication is allowed. Furthermore, it is assumed that the number of processors is unlimited. It is shown that there is a polynomial-time algorithm for an outtree and that the intree case is NP-hard. In addition, a special case of intree is shown to be polynomially solvable.

Task scheduling with interprocessor communication delays

Distributed memory architectures raise new and complex scheduling problems. n this paper, we first define a basic distributed memory computer scheduling problem issued from an ideal architecture. By proving that the corresponding decision problem is NP-complete, we show that unlike shared memory computer scheduling problems, these new problems do not become easy when the processor limitation constraint is removed. Finally, we improve the knowledge of the borderline between the easy and difficult subproblems of the basic one by giving polynomial special cases.

A note on Graham's bound

In [2], Graham establishes an upper bound for the makespan of a set of tasks whose executions are governed by given precedence constraints. In this paper we provide a similar upper bound for the makespan of such parallel processing systems when the interprocessor communication delays are taken into account. Our result is a slight generalization of that of Graham's.

Scheduling Precedence Graphs in Systems with Interprocessor Communication Times

The problem of nonpreemptively scheduling a set of m partially ordered tasks on n identical processors subject to interprocessor communication delays is studied in an effort to minimize the makespan. A new heuristic, called Earliest Task First (ETF), is designed and analyzed. It is shown that the makespan $\omega _{{\text{ETF}}} $ generated by ETF always satisfies $\omega _{{\text{ETF}}} \leqq ({{2 - 1} / n})\omega _{{\text{opt}}}^{(i)} + C$, where $\omega _{{\text{opt}}}^{(i)} $ is the optimal makespan without considering communication delays and C is the communication requirements over some immediate predecessor-immediate successor pairs along one chain. An algorithm is also provided to calculate C. The time complexity of Algorithm ETF is $O(nm^2 )$.