General DAGs Research Articles

Scheduling Coflows With Dependency Graph

Applications in data-parallel computing typically consist of multiple stages. In each stage, a set of intermediate parallel data flows ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Coflow</i> ) is produced and transferred between servers to enable starting of next stage. While there has been much research on scheduling isolated coflows, the dependency between coflows in multi-stage jobs has been largely ignored. In this paper, we consider scheduling coflows of multi-stage jobs represented by general <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DAG</i> s (Directed Acyclic Graphs) in a shared data center network, so as to minimize the total weighted completion time of jobs. This problem is significantly more challenging than the traditional coflow scheduling, as scheduling even a single multi-stage job to minimize its completion time is shown to be NP-hard. In this paper, we propose a polynomial-time algorithm with approximation ratio of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$O(\mu \log (m)/\log (\log (m)))$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu $ </tex-math></inline-formula> is the maximum number of coflows in a job and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula> is the number of servers. For the special case that the jobs’ underlying dependency graphs are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rooted trees</i> , we modify the algorithm and improve its approximation ratio. To verify the performance of our algorithms, we present simulation results using real traffic traces that show up to 53% improvement over the prior approach. We conclude the paper by providing a result concerning an optimality gap for scheduling coflows with general DAGs.

Space complexity of reachability testing in labelled graphs

Fix an algebraic structure (A,⁎). Given a graph G=(V,E) and the labelling function ϕ (ϕ:E→A) for the edges, two nodes s, t∈V, and a subset F⊆A, the A-Reach problem asks if there is a path p (need not be simple) from s to t whose yield (result of the operation in the ordered set of the labels of the edges constituting the path) is in F. On the complexity frontier of this problem, we show the following results.•When A is a group whose size is polynomially bounded in the size of the graph (hence equivalently presented as a multiplication table at the input), and the graph is undirected, the A-Reach problem is in L. Building on this, using a decomposition in [4], we show that, when A is a fixed quasi-group, and the graph is undirected, the A-Reach problem is in L. In a contrast, we show NL-hardness of the problem over bidirected graphs, when A is a matrix group over Q, or an aperiodic monoid.•As our main theorem, we prove a dichotomy for graphs labelled with fixed aperiodic monoids by showing that for every fixed aperiodic monoid A, A-Reach problem is either in L or is NL-complete.•We show that there exists a monoid M, such that the reachability problem in general DAGs can be reduced to A-Reach problem for planar non-bipartite DAGs labelled with M. In contrast, we show that if the planar DAGs that we obtain above are bipartite, the problem can be further reduced to reachability testing in planar DAGs and hence is in UL.

Data broadcasting for dependent information using multiple channels in wireless broadcast environments

Data broadcasting is an effective approach to disseminating information to mobile clients and has attracted much research attention in recent years. In many applications, the access pattern among the data can be represented by a weighted DAG. In this paper, we consider the problem of efficiently generating the broadcast schedules on multiple channels when the data set has a DAG access pattern. We show that it is NP-hard to find an optimal broadcast schedule which not only minimizes the latency but also satisfies the ancestor property that retains the data dependency. We further derive a condition for the input DAGs under which one can generate an optimal broadcast schedule in linear time and propose an algorithm to generate the schedule. Due to the NP-completeness, we provide three heuristics for general DAGs based on the level of a vertex in the input DAGs and each heuristic uses a different policy to place vertices into the broadcast channels. There are two categories for the policies. The first category mainly considers the probability for a process to stop at a considered vertex. The second category takes the vertices which are affected most when assigning a vertex into consideration. We analyze and discuss these heuristics. A short experimental simulation is given for supporting and validating the discussion. In particular, the experimental results indicate that roughly considering the whole posterior vertices of each vertex is not precise and may not lead to good results and considering the vertices affected most when assigning a vertex will help reducing the latency.

Efficient Subgraph Query Processing Algorithms on Graph-Structured XML Documents

Many challenges arise in subgraph query processing of graph-structured XML data.This paper studies the subgraph query processing of graph-structured XML data and proposes.A hash-based structural join algorithm,HGJoin to handle reachability queries on graph-structured XML data.Then,the algorithm is extended to process subgraph queries in form of bipartite graphs.Finally,based on these algorithms and cost model presented in this paper,a method to process subgraph queries in form of general DAGs is proposed.It is notable that all the algorithms above can be slightly modified to process subgraph queries in form of general graphs.Analysis and experiments show that all the algorithms have high performance.

Hash-base subgraph query processing method for graph-structured XML documents

When XML documents are modeled as graphs, many research issues arise. In particular, there are many new challenges in query processing on graph-structured XML documents because traditional query processing techniques for tree-structured XML documents cannot be directly applied. This paper studies the problem of structural queries on graph-structured XML documents. A hash-based structural join algorithm, HGJoin, is first proposed to handle reachability queries on graph-structured XML documents. Then, it is extended to the algorithms to process structural queries in form of bipartite graphs. Finally, based on these algorithms, a strategy to process subgraph queries in form of general DAGs is proposed. Analysis and experiments show that all the algorithms have high performance. It is notable that all the algorithms above can be slightly modified to process structural queries in form of general graphs.

Competitive Implementation of Parallel Programs

We apply the methodology of competitive analysis of algorithms to the implementation of programs on parallel machines. We consider the problem of finding the best on-line distributed scheduling strategy that executes in parallel an unknown directed acyclic graph (dag) which represents the data dependency relation graph of a parallel program and which is revealed as execution proceeds. We study the competitive ratio of some important classes of dags assuming a fixed communication delay ratio τ that captures the average interprocessor communication measured in instruction cycles. We provide competitive algorithms for divide-and-conquer dags, trees, and general dags, when the number of processors depends on the size of the input dag and when the number of processors is fixed. Our major result is a lower bound Ω (τ / log τ ) of the competitive ratio for trees; it shows that it is impossible to design compilers that produce almost optimal execution code for all parallel programs. This fundamental result holds for almost any reasonable distributed memory parallel computation model, including the LogP and BSP model.

VERTEX SPLITTING IN DAGS AND APPLICATIONS TO PARTIAL SCAN DESIGNS AND LOSSY CIRCUITS

Directed acyclic graphs (dags) are often used to model circuits. Path lengths in such dags represent circuit delays. In the vertex splitting problem, the objective is to determine a minimum number of vertices to split so that the resulting dag has no path of length &gt; δ. This problem has application to the placement of flip-flops in partial scan designs, placement of latches in pipelined circuits, placement of signal boosters in lossy circuits and networks, etc. Several simplified versions of this problem are shown to be NP-hard. A linear time algorithm is obtained for the case when the dag is a tree. A backtracking algorithm and heuristics are developed for general dags and experimental results using dags obtained from ISCAS benchmark circuits are obtained.