Directed Acyclic Graph Structure Research Articles

SEMdag: Fast learning of Directed Acyclic Graphs via node or layer ordering.

A Directed Acyclic Graph (DAG) offers an easy approach to define causal structures among gathered nodes: causal linkages are represented by arrows between the variables, leading from cause to effect. Recently, industry and academics have paid close attention to DAG structure learning from observable data, and many techniques have been put out to address the problem. We provide a two-step approach, named SEMdag(), that can be used to quickly learn high-dimensional linear SEMs. It is included in the R package SEMgraph and employs a two-stage order-based search using previous knowledge (Knowledge-based, KB) or data-driven method (Bottom-up, BU), under the premise that a linear SEM with equal variance error terms is assumed. We evaluated our framework's for finding plausible DAGs against six well-known causal discovery techniques (ARGES, GES, PC, LiNGAM, CAM, NOTEARS). We conducted a series of experiments using observed expression (or RNA-seq) data, taking into account a pair of training and testing datasets for four distinct diseases: Amyotrophic Lateral Sclerosis (ALS), Breast cancer (BRCA), Coronavirus disease (COVID-19) and ST-elevation myocardial infarction (STEMI). The results show that the SEMdag() procedure can recover a graph structure with good disease prediction performance evaluated by a conventional supervised learning algorithm (RF): in the scenario where the initial graph is sparse, the BU approach may be a better choice than the KB one; in the case where the graph is denser, both BU an KB report high performance, with highest score for KB approach based on topological layers. Besides its superior disease predictive performance compared to previous research, SEMdag() offers the user the flexibility to define distinct structure learning algorithms and can handle high dimensional issues with less computing load. SEMdag() function is implemented in the R package SEMgraph, easily available at https://CRAN.R-project.org/package=SEMgraph.

Weighted inverse gamma innovation for the structure learning of DAGs

AbstractIn cancer causal analysis, graphical models can help to identify genomic changes using the relations between proteins and phosphoproteins. In particular, directed acyclic graphs (DAGs) are effective methods to analyze intricate structures of dependence and causal connections among variables. From a Bayesian standpoint, DAG structural learning is a crucial goal in genomics studies since it enables the discovery of dependent relationships that contribute to the comprehension of variable behavior. However, we benefit from Bayesian DAG learning when dealing with small sample cases such as genomics and large networks because it promotes sparsity in the graphs, integrates prior knowledge, and crucially considers the uncertainty in the graph structure. On the other hand, modeling a proper prior distribution plays a vital role in correct posterior learning. Our work recommends using a weighted prior distribution for Gaussian DAG structure learning to improve graphical metrics. Simulation studies show that our proposal is superior to the existing one. We give the network configuration using the weighted prior distribution in a cancer data analysis.

The impact of COVID-19 on the oral health self-care practices of Australian adults

Abstract Aim Grounded in Andersen’s Behavioral Model (ABM), this study aims to explore the network of causal relationships between those factors influencing oral health self-management behavior during the COVID-19 pandemic in Australia. Subject and methods Data were obtained from the Australian population. Participants (N = 565) aged between 19 and 91 years (M = 54.50, SD = 16.16) provided online responses between June 2021 and May 2022. The majority of participants were female (60.9%), employed either full-time (37.65%) or part-time (22.02%), and had completed an undergraduate degree or higher (70.1%). We implemented a triangulated approach, utilizing multiple modeling methodologies including the Gaussian graphical model (GGM), directed acyclic graph (DAG), and structural equation modeling (SEM). These methodologies were employed in three predetermined stages using a single cross-sectional dataset. Results A causal model elucidating the influence of the COVID-19 pandemic on oral health self-management behavior was uncovered. The resulting GGM displayed an acceptable fit to the observed data. The DAG structure conformed to that expected by ABM, with all directed edges being also present as edges in the GGM. The subsequent SEM model specified by the DAG provided excellent fit to the data. Conclusion Our study offers empirical evidence that factors such as appointment availability, dental care affordability, and the opportunity to engage in oral health discussions with individuals other than dentists during the pandemic directly contributed to inadequate oral health self-care.

Preventing Double Spending Attacks through Crow Search Algorithm to Enhance E-Voting System Security

Electronic voting system is the process of polling votes and counting votes. In most of the countries voting may now be done electronically, there are still several difficulties involved, including the expense of paper, how ballots are organized, the possibility of varying results when tallying the votes, and others. Duplicate votes pose a significant concern as they can be fraudulently cast by individuals. To focus on this issue, Distributed Ledger Technology (DLT) is employed to enhance the voting procedure in a secured manner. A directed acyclic graph is used by the Internet of Things Application (IOTA), a promising distributed ledger system. Faster transaction confirmation, high scalability and zero transaction fees are achieved via the Directed Acyclic Graph structure. In both IOTA tangle and blockchain technology, the public cast duplicate votes. The unauthorized user can create duplicate votes in the blockchain as well as IOTA tangle. This can be focused in this proposed method. The double spending problem can be solved by using Crow Search Algorithm (CSA). This Optimization problem produces an improved result for resolving double spending in e-voting systems.

LeCaSiM: Learning Causal Structure via Inverse of M-Matrices with Adjustable Coefficients

The objective of causal discovery is to uncover the causal relationships among natural phenomena or human behaviors, thus establishing the basis for subsequent prediction and inference. Traditional ways to reveal the causal structure between variables, such as interventions or random trials, are often deemed costly or impractical. With the diversity of means of data acquisition and abundant data, approaches that directly learn causal structure from observational data are attracting more interest in academia. Unfortunately, learning a directed acyclic graph (DAG) from samples of multiple variables has been proven to be a challenging problem. Recent proposals leverage the continuous optimization method to discover the structure of the target DAG from observational data using an acyclic constraint function. Although it has an elegant mathematical form, its optimization process is prone to numerical explosion or the disappearance of high-order gradients of constraint conditions, making the convergence conditions unstable and the result deviating from the true structure. To tackle these issues, we first combine the existing work to give a series of algebraic equivalent conditions for a DAG and its corresponding brief proofs, then leverage the properties of the M-matrix to derive a new acyclic constraint function. Based on the utilization of this function and linear structural equation models, we propose a novel causal discovery algorithm called LeCaSiM, which employs continuous optimization. Our algorithm exploits the spectral radius of the adjacency matrix to dynamically adjust the coefficients of the matrix polynomial, effectively solving the above problems. We conduct extensive experiments on both synthetic and real-world datasets, the results show our algorithm outperforms the existing algorithms on both computation time and accuracy under the same settings.

ST-HO: Symmetry-Enhanced Energy-Efficient DAG Task Offloading Algorithm in Intelligent Transport System

In Intelligent Transport Systems (ITSs), Internet of Vehicles (IoV) communications and computation offloading technology have been introduced to assist with the burdensome sensing task processing, thus prompting a new design paradigm called mobile sensing–communication–computation (MSCC) synergy. Most researchers have focused on offloading strategy design to reduce energy consumption or execution costs, but ignore the intrinsic characteristics of tasks, which may lead to poor performance. This paper studies the offloading strategy of vehicle MSCC tasks represented by a Directed Acyclic Graph (DAG) structure. According to the DAG dependency of the subtasks, this paper proposes a computation offloading strategy to optimize energy consumption under time constraints. An energy consumption model for task execution is established. Then, the Simulated Annealing and Tabu Search hybrid optimization algorithm (ST-HO) is designed to solve the problem of minimizing the energy consumption. Crucially, this research integrates the concept of symmetry into the typical DAG structure of MSCC tasks, ensuring the integrity and efficiency of task execution in ITS. The simulation results show that ST-HO reduces energy consumption by at least 5.58% compared to the conventional algorithm. Particularly, the convergence speed of ST-HO is improved by 52.63% when the replication strategy of symmetric task is considered.

A Graphical Multi-Fidelity Gaussian Process Model, with Application to Emulation of Heavy-Ion Collisions

With advances in scientific computing and mathematical modeling, complex scientific phenomena such as galaxy formations and rocket propulsion can now be reliably simulated. Such simulations can however be very time-intensive, requiring millions of CPU hours to perform. One solution is multi-fidelity emulation, which uses data of different fidelities to train an efficient predictive model which emulates the expensive simulator. For complex scientific problems and with careful elicitation from scientists, such multi-fidelity data may often be linked by a directed acyclic graph (DAG) representing its scientific model dependencies. We thus propose a new Graphical Multi-fidelity Gaussian Process (GMGP) model, which embeds this DAG structure (capturing scientific dependencies) within a Gaussian process framework. We show that the GMGP has desirable modeling traits via two Markov properties, and admits a scalable algorithm for recursive computation of the posterior mean and variance along at each depth level of the DAG. We also present a novel experimental design methodology over the DAG given an experimental budget, and propose a nonlinear extension of the GMGP via deep Gaussian processes. The advantages of the GMGP are then demonstrated via a suite of numerical experiments and an application to emulation of heavy-ion collisions, which can be used to study the conditions of matter in the Universe shortly after the Big Bang. The proposed model has broader uses in data fusion applications with graphical structure, which we further discuss.

Modified graph-based algorithm to analyze security threats in IoT.

In recent years, the growing and widespread usage of Internet of Things (IoT) systems has led to the emergence of customized structures dependent on these systems. Industrial IoT (IIoT) is a subset of IoT in terms of applications and usage areas. IIoT presents many participants in various domains, such as healthcare, transportation, agriculture, and manufacturing. Besides the daily life benefits, IIoT technology provides major contributions via the Industrial Control System (ICS) and intelligent systems. The convergence of IoT and IIoT systems brings some integration and interoperability problems. In IIoT systems, devices interact with each other using information technologies (IT) and network space. However, these common usages and interoperability led to some security risks. To avoid security risks and vulnerabilities, different systems and protocols have been designed and published. Various public databases and programs identify and provide some of the security threats to make it easier for system administrators' missions. However, effective and long-term security detection mechanisms are needed. In the literature, there are numerous approaches to detecting security threats in IoT-based systems. This article presents two major contributions: First, a graph-based threat detection approach for IoT-based network systems is proposed. Threat path detection is one of the most critical steps in the security of IoT-based systems. To represent vulnerabilities, a directed acyclic graph (DAG) structure is constructed using threat weights. General threats are identified using Common Vulnerabilities and Exposures (CVE). The proposed threat pathfinding algorithm uses the depth first search (DFS) idea and discovers threat paths from the root to all leaf nodes. Therefore, all possible threat paths are detected in the threat graph. Second, threat path-reducing algorithms are proposed considering the total threat weight, hop length, and hot spot thresholds. In terms of available threat pathfinding and hot spot detecting procedures, the proposed reducing algorithms provide better running times. Therefore, all possible threat paths are founded and reduced by the constructed IoT-based DAG structure. Finally, simulation results are compared, and remarkable complexity performances are obtained.

Minimum+1 ( s, t )-cuts and Dual-edge Sensitivity Oracle

Let G be a directed multi-graph on n vertices and m edges with a designated source vertex s and a designated sink vertex t . We study the ( s,t )-cuts of capacity minimum+1 and as an important application of them, we give a solution to the dual-edge sensitivity for ( s,t )-mincuts—reporting an ( s,t )-mincut upon failure or insertion of any pair of edges. Picard and Queyranne [Mathematical Programming Studies, 13(1): 8–16 (1980)] showed that there exists a directed acyclic graph (DAG) that compactly stores all minimum ( s,t )-cuts of G . This structure also acts as an oracle for the single-edge sensitivity of minimum ( s,t )-cut. For undirected multi-graphs, Dinitz and Nutov [STOC, 509–518 (1995)] showed that there exists an 𝒪( n ) size 2-level Cactus model that stores all global cuts of capacity minimum+1. However, for minimum+1 ( s,t )-cuts, no such compact structure exists till date. We present the following structural and algorithmic results on minimum+1 ( s,t )-cuts. (1) Structure: There is an 𝒪( m ) size 2-level DAG structure that stores all minimum+1 (s,t) -cuts of G such that each minimum+1 ( s,t )-cut appears as 3-transversal cut—it intersects any path in this structure at most thrice. We also show that there is an 𝒪( mn ) size structure for storing and characterizing all minimum+1 (s,t) -cuts in terms of 1-transversal cuts. (2) Data structure: There exists an 𝒪( n 2 ) size data structure that, given a pair of vertices {u,v} that are not separated by an ( s,t )-mincut, can determine in 𝒪(1) time if there exists a minimum+1 ( s,t )-cut, say ( A,B ), such that s,u ∊ A and v,t∊ B ; the corresponding cut can be reported in 𝒪(| B |) time. (3) Sensitivity oracle: There exists an 𝒪( n 2 ) size data structure that solves the dual-edge sensitivity problem for (s,t) -mincuts. It takes 𝒪(1) time to report the capacity of a resulting (s,t) -mincut (A,B) and 𝒪(| B |) time to report the cut. (4) Lower bounds: For the data structure problems addressed in results (2) and (3) above, we also provide a matching conditional lower bound. We establish a close relationship among three seemingly unrelated problems—all-pairs directed reachability problem, the dual-edge sensitivity problem for ( s,t )-mincuts, and the problem of reporting the capacity of ({ x,y }, { u,v })-mincut for any four vertices x,y,u,v in G . Assuming the Directed Reachability Hypothesis by Patrascu [SIAM J. Computing, 827–847 (2011)] and Goldstein et al. [WADS, 421–436 (2017)], this leads to \(\tilde{\Omega }(n^2)\) lower bounds on the space for the latter two problems.

Skeleton estimation of directed acyclic graphs using partial least squares from correlated data

Directed acyclic graphs (DAGs) are directed graphical models that are well known for discovering causal relationships between variables in a high-dimensional setting. When the DAG is not identifiable due to the lack of interventional data, the skeleton can be estimated using observational data, which is formed by removing the direction of the edges in a DAG. In real data analyses, variables are often highly correlated due to some form of clustered sampling, and ignoring this correlation will inflate the standard errors of the parameter estimates in the regression-based DAG structure learning framework. In this work, we propose a two-stage DAG skeleton estimation approach for highly correlated data. First, we propose a novel neighborhood selection method based on sparse partial least squares (PLS) regression, and a cluster-weighted adaptive penalty is imposed on the PLS weight vectors to exploit the local information. In the second stage, the DAG skeleton is estimated by evaluating a set of conditional independence hypotheses. Simulation studies are presented to demonstrate the effectiveness of the proposed method. The algorithm is also tested on publicly available datasets, and we show that our algorithm obtains higher sensitivity with comparable false discovery rates for high-dimensional data under different network structures.

Workflow performance prediction based on graph structure aware deep attention neural network

• In workflow performance prediction, DAG structure matters; • DAG-Transformer effectively embeds the DAG information and outperforms mainstream ML, DL , and GCN methods; • A new dataset for cloud workflow performance prediction is accompanied as well as the source code. With the rapid growth of cloud computing, efficient operational optimization and resource scheduling of complex cloud business processes rely on real-time and accurate performance prediction. Previous research on cloud computing performance prediction focused on qualitative (heuristic rules), model-driven, or coarse-grained time-series prediction, which ignore the study of historical performance, resource allocation status and service sequence relationships of workflow services. There are even fewer studies on prediction for workflow graph data due to the lack of available public datasets. In this study, from Alibaba Cloud's Cluster-trace-v2018, we extract nearly one billion offline task instance records into a new dataset, which contains approximately one million workflows and their corresponding directed acyclic graph (DAG) matrices. We propose a novel workflow performance prediction model (DAG-Transformer) to address the aforementioned challenges. In DAG-Transformer, we design a customized position encoding matrix and an attention mask for workflows, which can make full use of workflow sequential and graph relations to improve the embedding representation and perception ability of the deep neural network. The experiments validate the necessity of integrating graph-structure information in workflow prediction. Compared with mainstream deep learning (DL) methods and several classic machine learning (ML) algorithms, the accuracy of DAG-Transformer is the highest. DAG-Transformer can achieve 85-92% CPU prediction accuracy and 94-98% memory prediction accuracy, while maintaining high efficiency and low overheads. This study establishes a new paradigm and baseline for workflow performance prediction and provides a new way for facilitating workflow scheduling.

Tangle 2.0 Leaderless Nakamoto Consensus on the Heaviest DAG

We introduce the theoretical foundations of the Tangle 2.0, a probabilistic leaderless consensus protocol based on a directed acyclic graph (DAG) called the Tangle. The Tangle naturally succeeds the blockchain as its next evolutionary step as it offers features suited to establish more efficient and scalable distributed ledger solutions. Consensus is no longer found in the longest chain but on the heaviest DAG, where PoW is replaced by a stake- or reputation-based weight function. The DAG structure and the underlying Reality-based UTXO Ledger allow parallel validation of transactions without the need for total ordering. Moreover, it enables the removal of the intermediary of miners and validators, allowing a pure two-step process that follows the \emph{propose-vote} paradigm at the node level and not at the validator level. We propose a framework to analyse liveness and safety under different communication and adversary models. This allows providing impossibility results in some edge cases and in the asynchronous communication model. We provide formal proof of the security of the protocol assuming a common random coin.

Reflection on modern methods: understanding bias and data analytical strategies through DAG-based data simulations.

Directed acyclic graphs (DAGs) are increasingly used in epidemiology to identify and address different types of bias. The present work aims to demonstrate how DAG-based data simulation can be used to understand bias and compare data analytical strategies in an educational context. Examples based on classical confounding situations and an M-DAG are examined and used to introduce basic concepts and demonstrate some important features of regression analysis, as well as the harmful effect of adjusting for a collider variable. Other potential uses of DAG-based data simulation include systematic comparisons of data analytical strategies or the evaluation of the role of uncertainties in a hypothesized DAG structure, including other types of bias such as information bias. DAG-based data simulations, like those presented here, should facilitate the exploration of several key epidemiological concepts, DAG theory and data analysis. Some suggestions are also made on how to further expand the ideas from this study.

An improved list-based task scheduling algorithm for fog computing environment

A high-performance execution of programs predominately depends on the efficient scheduling of tasks. An application consists of a sequence of tasks that can be represented as a directed acyclic graph (DAG). The tasks in the DAG have precedence constraints between them and each task has a different timeline on different processors. In this paper, a new list-based scheduling algorithm is proposed which schedules the tasks which are represented as a DAG structure. The main focus of this algorithm is to schedule the tasks to the suitable processing node in fog environment as the fog nodes have limited processing capacity. The assignment of tasks on the fog node should consider both the computation cost of the node and the execution finishing time of the node. The proposed algorithm has three phases. (1) the level sorting phase, where the independent tasks are identified (2) in the Task prioritization phase the proposed algorithm assigns priority to the task which has more successors so that more tasks in the next level can start their execution and (3) in the task selection phase a balanced combination of local optimal and global optimal approach is considered to assign a task to a suitable processor which further enhances the processor selection phase results in minimizing both the makespan and overall computation cost of the processors. Extensive experiments are carried out using randomly generated graphs and graphs from the real-world to analyze the performance of the proposed algorithm. The results show that the proposed algorithm outperforms all other well-known algorithms like predict earliest finish time, heterogeneous earliest finish time algorithm, minimal optimistic processing time, and SDBBATS in terms of performance matrices like average scheduling length ratio, speedup, and makespan.

Consensus mechanism design based on structured directed acyclic graphs

Capacity limit is a bottleneck for broader applications of blockchain systems. Scaling up capacity while preserving security and decentralization are major challenges in blockchain infrastructure design. In this paper, we design a proof of work-based mechanism by endowing directed acyclic graphs (DAG) with a novel structure so that peers can reach consensus at a large scale. At a high level, we break large blocks into smaller ones to improve utilization of broadcast network and embed a Nakamoto chain inside the DAG in a decent way to ensure security. We further exploit the DAG structure and design a mempool transaction assignment method. The method reduces the probability that a transaction is processed by multiple miners and hence improves processing efficiency. Without sacrificing security and decentralization, our design significant scales up capacity and also addresses important issues such as high latency and mining power concentration in existing blockchain systems.

Multi-dimensional classification via stacked dependency exploitation

Multi-dimensional classification (MDC) aims to build classification models for multiple heterogenous class spaces simultaneously, where each class space characterizes the semantics of an object w.r.t. one specific dimension. Modeling dependencies among class spaces plays a key role in solving MDC tasks, where most approaches work by assuming directed acyclic graph (DAG) structure or random chaining structure over class spaces. Different from existing probabilistic strategies, a deterministic strategy named oursfor dependency modeling is proposed in this paper via stacked dependency exploitation. In the first-level, pairwise dependencies are considered which can be modeled more reliably than modeling full dependencies among all class spaces by DAG or chaining structure. In the second-level, the class label of unseen instance w.r.t. each class space is determined by adaptively stacking predictive outputs from first-level pairwise classifiers. Experimental results show that stacked dependency exploitation leads to superior performance against state-of-the-art MDC approaches.

On the Resilience of DAG-Based Distributed Ledgers in IoT Applications

Directed Acyclic Graph (DAG) based Distributed Ledgers can be useful in a number of applications in the IoT domain. A distributed ledger should serve as an immutable and irreversible record of transactions, however, a DAG structure is a more complicated mathematical object than its blockchain counterparts, and as a result, providing guarantees of immutability and irreversibility is more involved. In this paper, we analyse a commonly discussed attack scenario known as a parasite chain attack for the IOTA Foundation DAG based ledger. We analyse the efficacy of IOTA core MCMC algorithm using a matrix model and present an extension which improves the ledger resistance to these attacks.

Selection and integration of generalized instrumental variables for estimating total effects

We consider a situation where cause–effect relationships between variables can be described as a directed acyclic graph (DAG) and the corresponding linear structural equation model (linear SEM). When several pairs of instrumental variables (IVs) and covariates (IV-pairs; Pearl in: Proceedings of the 20th conference on uncertainty in artificial intelligence, AUAI Press, Arlington, Virginia, United States, UAI’04, 2004) are available, we propose (i) the graphical selection criteria of IV-pairs for total effects and (ii) an integrated estimator that combines them to estimate total effects with better accuracy. In this paper, in accordance with the paper by Brito and Pearl (in: Proceedings of the 18th conference on uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, UAI’02, 2002), the proposed estimator is called an integrated generalized instrumental variable (iGIV) estimator. The proposed graphical selection criteria imply that (a) the back-door criterion achieves better estimation accuracy than the traditional instrumental variable (IV) method of estimating total effects even when the treatment and covariates are highly correlated and (b) the conditional IV method can be superior to the back-door criterion in some situations. The iGIV estimator provides a general class that includes both the ordinary least squares (OLS) estimator based on the back-door criterion and the two-stage least squares (2SLS) estimator based on the (conditional) IV method. We clarify the properties of the iGIV estimator, some of which can be read off from the DAG structure. Furthermore, through numerical experiments and an application to a case study, we show that the performance of the iGIV estimator is superior to those of the OLS and IV estimators. The iGIV estimator can be a powerful tool to estimate the total effect when the proposed graphical selection criteria of the IV-pairs are not satisfied.

Bayesian Model Selection of Gaussian Directed Acyclic Graph Structures

SummaryDuring the last years, graphical models have become a popular tool to represent dependencies among variables in many scientific areas. Typically, the objective is to discover dependence relationships that can be represented through a directed acyclic graph (DAG). The set of all conditional independencies encoded by a DAG determines its Markov property. In general, DAGs encoding the same conditional independencies are not distinguishable from observational data and can be collected into equivalence classes, each one represented by a chain graph called essential graph (EG). However, both the DAG and EG space grow super exponentially in the number of variables, and so, graph structural learning requires the adoption of Markov chain Monte Carlo (MCMC) techniques. In this paper, we review some recent results on Bayesian model selection of Gaussian DAG models under a unified framework. These results are based on closed‐form expressions for the marginal likelihood of a DAG and EG structure, which is obtained from a few suitable assumptions on the prior for model parameters. We then introduce a general MCMC scheme that can be adopted both for model selection of DAGs and EGs together with a couple of applications on real data sets.

Semi-Clairvoyant Scheduling in Data Analytics Systems

Popular data analytics systems including Apache Hadoop, Dryad, and Apache Spark abstract jobs as directed acyclic graphs (DAGs). Speeding up completions for DAG jobs matter in practice in order to support real-time decisions. State-of-the-art works propose clairvoyant schedulers to optimize these goals, however, they assume complete job information as a prior knowledge which includes the precise DAG structure, and fine-grained resource requirement and duration time of each task. This assumption limits their applicability. In this paper, to be more practical, we relax the complete prior knowledge assumption and rely solely on partial prior information, based on which, we design a semi-clairvoyant task scheduler Cobra operating within each job. When managing resources for a job, Cobra adaptively adjusts its resource desires in a multiplicative-increase multiplicative-decrease manner on the basis of the its current resource utilization and the presence of current waiting tasks. When assigning tasks to run on the allocated resources, Cobra strives to satisfy task locality preference by tolerating each task waiting for some time that is bounded by a parameterized threshold. Even with the partial prior job information, when a set of jobs in which each employing Cobra as its task scheduler, run on a cluster that employs the fair job scheduler, we theoretically prove the produced makespan and average job response time are O(1)-competitive in different settings. We implement our design in Spark on YARN system, and use experiments from both real deployments and simulations on Google's trace to verify the performance promotion and sensitivity of Cobra.