Interprocessor Communication Time Research Articles

High-performance data mining with intelligent SSD

An intuitive way to process the big data efficiently is to reduce the volume of data transferred over the storage interface to a host system. This is the reason that the notion of intelligent SSD (iSSD) was proposed to give processing power to SSD. There is rich literature on iSSD, however, its real implementation has not been provided to the public yet. Most prior work aims to quantify the benefits of iSSD with analytical modeling. In this paper, we first develop on iSSD simulator and present the potential of iSSD in data mining through the iSSD simulator. Our iSSD simulator performs on top of the gem 5 simulator and fully simulates all the processes of data mining algorithms running in iSSD with cycle-level accuracy. Then, we further addresse how to exploit all the computing resources for efficient processing of data mining algorithms. These days, CPU, GPU, and SSD are recently equipped together in most computing environment. If SSD is replaced with iSSD later on, we have a new computing environment where the three computing resources collaborate one another to process big data quite effectively. For this, scheduling is required to decide which computing resource is going to run for which function at which time. In our heterogeneous scheduling, types of computing resources, memory sizes in computing resources, and inter-processor communication times including IO time in SSD are considered. Our scheduling results show that processing in the collaborative environment outperforms that in the traditional one by up to about 10 times.

Task allocation for maximizing reliability of distributed systems: A simulated annealing approach

This paper addresses the problem of task allocation in heterogeneous distributed systems with the goal of maximizing the system reliability. It first develops an allocation model for reliability based on a cost function representing the unreliability caused by the execution of tasks on the system processors and the unreliability caused by the interprocessor communication time subject to constraints imposed by both the application and the system resources. It then presents a heuristic algorithm derived from the well-known simulated annealing (SA) technique to quickly solve the mentioned problem. The performance of the proposed algorithm is evaluated through experimental studies on a large number of randomly generated instances. Indeed, the quality of solutions are compared with those derived by using the branch-and-bound (BB) technique.

Provable algorithms for parallel generalized sweep scheduling

We present provably efficient parallel algorithms for sweep scheduling, which is a commonly used technique in radiation transport problems, and involves inverting an operator by iteratively sweeping across a mesh from multiple directions. Each sweep involves solving the operator locally at each cell. However, each direction induces a partial order in which this computation can proceed. On a distributed computing system, the goal is to schedule the computation, so that the length of the schedule is minimized. Due to efficiency and coupling considerations, we have an additional constraint, namely, a mesh cell must be processed on the same processor along each direction. Problems similar in nature to sweep scheduling arise in several other applications, and here we formulate a combinatorial generalization of this problem that captures the sweep scheduling constraints,and call it the generalized sweep scheduling problem. Several heuristics have been proposed for this problem; see [S. Pautz, An algorithm for parallel S n sweeps on unstructured meshes, Nucl. Sci. Eng. 140 (2002) 111–136; S. Plimpton, B. Hendrickson, S. Burns, W. McLendon, Parallel algorithms for radiation transport on unstructured grids, Super Comput. (2001)] and the references therein; but none of these have provable worst case performance guarantees. Here we present a simple, almost linear time randomized algorithm for the generalized sweep scheduling problem that (provably) gives a schedule of length at most O ( log 2 n ) times the optimal schedule for instances with n cells, when the communication cost is not considered, and a slight variant, which coupled with a much more careful analysis, gives a schedule of (expected) length O ( log m log log log m ) times the optimal schedule for m processors. These are the first such provable guarantees for this problem. The algorithm can be extended with an additional multiplicative factor in the case when we have inter-processor communication latency, in the models of Rayward-Smith [UET scheduling with inter-processor communication delays, Discrete Appl. Math. 18 (1) (1987) 55–71] and Hwang et al. [Scheduling precedence graphs in systems with inter-processor communication times, SIAM J. Comput. 18(2) (1989) 244–257]. Our algorithms are extremely simple, and use no geometric information about the mesh; therefore, these techniques are likely to be applicable in more general settings. We also design a priority based list schedule using these ideas, with the same theoretical guarantee, but much better performance in practice; combining this algorithm with a simple block decomposition also lowers the overall communication cost significantly. Finally, we perform a detailed experimental analysis of our algorithm. Our results indicate that the algorithm compares favorably with the length of the schedule produced by other natural and efficient parallel algorithms proposed in the literature [S. Pautz, An Algorithm for parallel S n sweeps on unstructured meshes, Nucl. Sci. Eng. 140 (2002) 111–136; S. Plimpton, B. Hendrickson, S. Burns, W. McLendon, Parallel algorithms for radiation transport on unstructured grids, Super Comput. (2001)].

Simulated parallel annealing within a neighborhood for optimization of biomechanical systems

Optimization problems for biomechanical systems have become extremely complex. Simulated annealing (SA) algorithms have performed well in a variety of test problems and biomechanical applications; however, despite advances in computer speed, convergence to optimal solutions for systems of even moderate complexity has remained prohibitive. The objective of this study was to develop a portable parallel version of a SA algorithm for solving optimization problems in biomechanics. The algorithm for simulated parallel annealing within a neighborhood (SPAN) was designed to minimize interprocessor communication time and closely retain the heuristics of the serial SA algorithm. The computational speed of the SPAN algorithm scaled linearly with the number of processors on different computer platforms for a simple quadratic test problem and for a more complex forward dynamic simulation of human pedaling.

Parallel ADI solver based on processor scheduling

Gaussian elimination is used for the direct solution of banded linear systems that typically appear in implicit numerical methods for PDEs. Gaussian elimination for narrow-banded systems (also known as the Thomas algorithm (TA)) includes forward and backward recurrences along lines of a numerical grid. Multi-domain decomposition, essential for parallelization of implicit solvers, spans the recurrences across processors in one or more directions. Processor idle time and inter-processor communication time are two interdependent reasons for the poor parallelization efficiency of TAs. In this research an efficient parallel algorithm for 3D directionally split problems is developed. The proposed solver is based on the static scheduling of processors where local and non-local, data-dependent and data-independent computations are scheduled while processors are idle. The proposed algorithm uses a reformulated version of the pipelined Thomas algorithm that starts the backward step computations immediately after the completion of the forward step computations for the first portion of lines. This algorithm has data available for other computational tasks while processors are idle from the TA. A theoretical model of parallelization efficiency is used to define optimal parameters of the algorithm, to show an asymptotic parallelization penalty and to obtain an optimal cover of a global domain with subdomains. It is shown by computational experiments and by the theoretical model that the proposed algorithm considerably reduces the communication cost and processor idle time over the basic algorithm for the range of the number of processors (subdomains) considered and the number of grid nodes per subdomain.

A fast task-to-processor assignment heuristic for real-time multiprocessor DSP applications

The optimal assignment of the tasks to the processors to minimize total delay in a multiprocessor digital signal processing (DSP) architecture is extremely difficult, particularly for systems of many (e.g. 100) tasks. Two factors especially complicate the problem: (1) the multiprocessor architecture affects the inter-processor communication times, and (2) the specific assignment of tasks to processors affects the inter-task communication times. We develop a fast heuristic for assigning tasks to processors. There are two main ingredients in our method: (i) the choice of a useful general-purpose multiprocessor architecture for DSP applications, and (ii) an adaptive list-ordering heuristic which takes advantage of knowledge of the inter-processor communication characteristics of the chosen architecture. Examples are given, including comparisons to exact branch-and-bound methods, and a large sonar example. Scope and purpose General digital signal processing system architectures consist of an arrangement of signal processing units connected by communication links. The tasks to be carried out by the signal processing units are specified via a task graph similar to a PERT diagram. The challenge is to assign the tasks in the task graph to the signal processing units in the architecture in an optimal manner. In this paper, we are interested in task-to-processor assignments that minimize the total delay, which is the lag between the first appearance of a signal at the input port and the production of a processed signal at the output port. The assignment process is complicated by the need to consider the inter-task communication delays, which are themselves greatly affected by the assignment. The task-to-processor assignment problem is NP-complete, hence heuristics must be used. We develop a fast and effective heuristic for a specific general-purpose digital signal processing architecture.

UNIFIED PARALLEL ALGORITHMS FOR GAUSSIAN ELIMINATION WITH BACKWARD SUBSTITUTION ON PRODUCT NETWORKS

The increasing interest in product networks (PNs) as a method of combining desirable properties of component networks, has prompted a need for the general study of the algorithmic issues related to this important class of interconnection networks. In this paper we present unified parallel algorithms for Gaussian elimination, with partial and complete pivoting, on product networks. A parallel algorithm for backward substitution is also presented. The proposed algorithms are network independent and are also independent of the matrix distribution methods employed. These algorithms can be used on a wide range of PNs including hypercube, mesh, and k-ary n-cube. Unified models for estimating computation time and interprocessor communication time are also presented. These models are then used to measure the performance of the proposed algorithms on several product networks

Optimal scheduling algorithm for distributed-memory machines

Task Scheduling is one of the key elements in any distributed-memory machine (DMM), and an efficient algorithm can help reduce the interprocessor communication time. As optimal scheduling of tasks to DMMs is a strong NP-hard problem, many heuristic algorithms have been introduced in the literature. This paper presents a Task Duplication based Scheduling (TDS) algorithm which can schedule directed acyclic graphs (DAGs) with a complexity of O(|V|/sup 2/), where |V| is the number of tasks in the DAG. This algorithm generates an optimal schedule for a class of DAGs which satisfy a simple cost relationship. The performance of the algorithm has been observed by its application to some practical DAGs, and by comparing it with other existing scheduling schemes in terms of the schedule length and algorithm complexity.

Complexity of Task Graph Scheduling with Fixed Communication Capacity

Consider a scheduling problem of parallel computations in multiprocessor systems. Let a parallel program be modeled by a task graph, where vertices represent tasks and arcs the communications between tasks. An interprocessor communication time incurs when two tasks assigned to two different processors have to communicate. Such a scheduling problem has recently been studied in the literature, mostly for the case where interprocessor communication times are fully determined. In this paper, we consider the scheduling problem with communication resource constraints. More specifically, we consider the case where all interprocessor communications take place on a network of bounded capacity. We consider two variants of the problem: communications with independent-data semantics and common-data semantics. We show that even for very specific subproblems, viz. scheduling of general graphs on two processors and scheduling of binary trees on an infinite number of processors, the minimization of the makespan of parallel programs in such a multiprocessor system is strongly [Formula: see text]-hard. We first establish the results for the case of capacity 1, referred to as the single-bus system. We then extend the results to the more general case of fixed communication capacities. As a consequence, the general scheduling problem of parallel programs with communication resource constraints is strongly [Formula: see text]-hard. These results are to be contrasted with the corresponding scheduling problems without contraint on the communication capacity, where the two-processor case has unknown time complexity and the infinite-processor case is polynomial. Our results are also extended to the case of broadcasting communications, and can be applied to multiprocessor systems with shared memory.

A methodology for digital real time simulation of dynamic systems using modern DSPs

The speedup factor in real time simulation of dynamic systems using multiprocessor resources depends on: the architecture of the multiprocessor system, type of interconnection between parallel processors, numerical methods and techniques used for discretization and task assignment and scheduling policy. The minimization of the number of processors needed for real time simulation requires the minimization of processors times for interprocessor communications and efficient scheduling policy. Therefore, this article presents a methodology for the real time simulation of dynamic systems including a new pre-emptive static assignment and scheduling policy. The advantages of applying digital signal processor with parallel architecture, for example TMS320C40, in real time simulation have been described. Some important issues in real time architectures necessary for efficient multiprocessor real time simulations, such as multiple I/O channels, concurrent I/O and CPU processing, direct high speed interprocessor communications, fast context switching, multiple busses, multiple memories, and powerful arithmetic units are inherent to this processor. These features minimize interprocessor communication time and maximize sustained CPU performance.

SCALABLE ALGORITHMS FOR BICHROMATIC LINE SEGMENT INTERSECTION PROBLEMS ON COARSE GRAINED MULTICOMPUTERS

We present output-sensitive scalable parallel algorithms for bichromatic line segment intersection problems for the coarse grained multicomputer model. Under the assumption that n≥p2, where n is the number of line segments and p the number of processors, we obtain an intersection counting algorithm with a time complexity of [Formula: see text], where Ts(m, p) is the time used to sort m items on a p processor machine. The first term captures the time spent in sequential computation performed locally by each processor. The second term captures the interprocessor communication time. An additional [Formula: see text] time in sequential computation is spent on the reporting of the k intersections. As the sequential time complexity is O(n log n) for counting and an additional time O(k) for reporting, we obtain a speedup of [Formula: see text] in the sequential part of the algorithm. The speedup in the communication part obviously depends on the underlying architecture. For example for a hypercube it ranges between [Formula: see text] and [Formula: see text] depending on the ratio of n and p. As the reporting does not involve more interprocessor communication than the counting, the algorithm achieves a full speedup of p for k≥ O( max (n log n log p, n log 3 p)) even on a hypercube.

Task assignment using a problem‐space genetic algorithm

AbstractThe task assignment problem is one of assigning tasks of a parallel program among the processors of a distributed computing system in order to reduce the job turnaround time and to increase the throughput of the system. Since the task assignment problem is known to be NP‐complete except in a few special situations, satisfactory suboptimal solutions obtainable in a reasonable amount of computation time are generally sought. In the paper we introduce a technique based on the problem‐space genetic algorithm (PSGA) for the static task assignment problem in both homogeneous and heterogeneous distributed computing systems to reduce the task turnaround time and to increase the throughput of the system by properly balancing the load and reducing the interprocessor communication time among processors. The PSGA based approach combines the power of genetic algorithms, a global search method, with a simple and fast problem‐specific heuristic to search a large solution space efficiently and effectively to find the best possible solution in an acceptable CPU time. Experimental results on test examples from the literature show considerable improvements in both the assignment cost and the CPU times over the previous work. The proposed scheme is also applied to a digital signal processing (DSP) system consisting of 119 tasks to illustrate its balancing properties and computational advantage on a large system. The proposed scheme offers 12–30% improvement in the assignment cost as compared to the previous best known results for the DSP example.

Partitioning of unstructured meshes for load balancing

AbstractMany large‐scale engineering and scientific calculations involve repeated updating of variables on an unstructured mesh. To do these types of computations on distributed memory parallel computers, it is necessary to partition the mesh among the processors so that the load balance is maximized and interprocessor communication time is minimized. This can be approximated by the problem of partitioning a graph so as to obtain a minimum cut, a well‐studied combinatorial optimization problem. Graph partitioning is NP complete, so for real world applications one resorts to heuristics, i.e. algorithms that give good but not necessarily optimum solutions. These algorithms include recursive spectral bisection, local search methods such as Kernighan‐Lin, and more general purpose methods such as simulated annealing. We show that a general procedure enables us to combine simulating annealing with Kernighan‐Lin. The resulting algorithm is both very fast and extremely effective.

Reasoning assertionally about real-time systems

We consider a real-time system defined by a set of programs that execute concurrently on one or more (hardware) processors and interact between themselves and with the environment. We show how to reason assertionally about such real-time systems, using standard (non-real-time) assertional techniques (i.e. Hoare-logic and linear-history temporal logic). The key step is to transform the set of programs to a state transition system, using epoch variables to model assumptions on statement execution times, interprocessor communication times, scheduling policy, and allocation of programs to processors. Our method is illustrated on a nontrivial example consisting of three concurrent programs: a sensor that buffers inputs from the environment, an integrator that periodically integrates the buffered inputs into a database, and a query-handler that responds to queries from the environment by accessing the database. We model the system for two different program-to-processor allocations (dedicated processors and shared processors with priority scheduling). We prove that the system satisfies desired properties (e.g. bounded query response time and absence of buffer overflow). >

Near embeddings of hypercubes into Cayley graphs on the symmetric group

Simulations of hypercube networks by certain Cayley graphs on the symmetric group are investigated. Let Q(k) be the familiar k-dimensional hypercube, and let S(n) be the star network of dimension n defined as follows. The vertices of S(n) are the elements of the symmetric group of degree n, two vertices x and y being adjacent if xo(1,i)=y for some i. That is, xy is an edge if x and y are related by a transposition involving some fixed symbol (which we take to be /spl I.bold/1). This network has nice symmetry properties, and its degree and diameter are sublogarithmic as functions of the number of vertices, making it compare favorably with the hypercube network. These advantages of S(n) motivate the study of how well it can simulate other parallel computation networks, in particular, the hypercube. The first step in such a simulation is the construction of a one-to-one map f:Q(k)/spl rarr/S(n) of dilation d, for d small. That is, one wants a map f such that images of adjacent points are at most distance d apart in S(n). An alternative approach, best applicable when one-to-one maps are difficult or impossible to find, is the construction of a one-to-many map g of dilation d, defined as follows. For each point x/spl isin/Q(k), there is an associated subset g(x)/spl sube/V(S(n)) such that for each edge xy in Q(k), every x'/spl isin/g(x) is at most distance d in S(n) from some y'/spl isin/g(y). Such one-to-many maps allow one to achieve the low interprocessor communication time desired in the usual one-to-one embedding underlying a simulation. This is done by capturing the local structure of Q(k) inside of S(n) (via the one-to-many embedding) when the global structure cannot be so captured. Our results are the following. 1) There exist the following one-to-many embeddings: a) f:Q(k)/spl rarr/S(3k+1) with dilation (f)=1; b) f:Q(11k+2)/spl rarr/S(13k+2) with dilation (f)=2. 2) There exists a one-to-one embedding f:Q(n2/sup n/spl minus/1/)/spl rarr/S(2/sup n/) with dilation (f)=3.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Estimating the speedup in parallel parsing

A method for estimating the speedup for asynchronous bottom-up parallel parsing is presented. Two models for bottom-up parallel parsing are proposed, and the speedup for each of the two models is estimated. The speedup obtained for model A is a very close to the simulation result already available in literature; however, the model is restrictive because it can only communicate with its immediate left and right neighbors. This increases the processor coordination and interprocessor communication times. Model B, while showing a greater speedup time, is expensive to construct when the number of processors is large. >

Multitransputer architecture for the low-level characterization of speech spectrograms

A parallel multitransputer architecture that applies two-dimensional low-level image processing techniques to capture acoustic patterns in the speech spectrogram is presented. The system consists of a two-dimensional quad-connected mesh interconnection network of transputers. It is capable of executing both linear and non-linear image processing functions. Example functions include linear and non-linear convolutional filtering, edge detection and simulated annealing. Near-linear speed-up performances with increasing numbers of processors have been observed for all image processing functions, the interprocessor communication time being the dominating limiting factor.

A New Parallel Sorting Algorithm and its Efficient VLSI Implementation

In this paper we develop a new parallel algorithm for sorting which has a time complexity of O(log n) and requires n2/log n processors. The algorithm can be readily mapped on an SIMD mesh connected array of processors which has all the features of efficient VLSI implementation. The corresponding hardware algorithm maintains the O(Log n) execution time and has a low O(n) interprocessor communication time.

Efficient mapping algorithms for scheduling robot inverse dynamics computation on a multiprocessor system

Two efficient mapping algorithms are presented for scheduling the execution of a robot inverse dynamics computation using a p-processor multiprocessor system where p is the number of identical processors. An objective function is defined in terms of the sum of the processor finishing time and the interprocessor communication time. A minimax optimization is performed on the objective function to obtain the best mapping. This mapping problem is formulated as a combination of a graph partitioning and scheduling problem; both are known to be NP-complete. Computer simulations were performed to evaluate and verify the performance and the validity of the proposed mapping algorithms. Experiments for computing the inverse dynamics of a six-jointed PUMA-like manipulator based on the Newton-Euler dynamic equations were implemented on an NCUBE/ten hypercube computer to verify the proposed mapping algorithms. >

Scheduling Precedence Graphs in Systems with Interprocessor Communication Times

The problem of nonpreemptively scheduling a set of m partially ordered tasks on n identical processors subject to interprocessor communication delays is studied in an effort to minimize the makespan. A new heuristic, called Earliest Task First (ETF), is designed and analyzed. It is shown that the makespan $\omega _{{\text{ETF}}} $ generated by ETF always satisfies $\omega _{{\text{ETF}}} \leqq ({{2 - 1} / n})\omega _{{\text{opt}}}^{(i)} + C$, where $\omega _{{\text{opt}}}^{(i)} $ is the optimal makespan without considering communication delays and C is the communication requirements over some immediate predecessor-immediate successor pairs along one chain. An algorithm is also provided to calculate C. The time complexity of Algorithm ETF is $O(nm^2 )$.