Job Completion Time Research Articles

MATHEMATICAL MODELS AND METHODS OF COORDINATED PLANNING

Modern processes of globalization and economic competition require a significant increase in the requirements for the professional degree of top-level managers who manage the activities of international corporations, regional economies, branch ministries, etc. Their efficient operation is impossible without the use of basic scientific developments and appropriate software which implement the main qualitative law of complex organizational and production systems management: the law of coordinated management (planning), when management decisions at the top level take into account interests that may not coincide, or even be antagonistic in organizational and production subsystems connected by a certain structure of mutual relations within a single organizational and production complex system. In this work, we consider a two-level organizational and production system, which in terms of the generally known theory of active systems is defined as “decision-making center → elements (of an organizational and production subsystem)”. We consider formal models of elements of two classes, linear continuous and discrete, aggregated production models which belong to the same class of NP-hard single-stage scheduling problems. For both types of element models, we give compromise criteria and corresponding methods of constructing compromise solutions based on the results of Prof. A. A. Pavlov for multi-objective linear programming, as a result of his theoretical research for discrete optimization problems under uncertainty, and the theory of PSC-algorithms created by him and his students, that is, algorithms containing polynomial complexity subalgorithms for constructing feasible solutions that satisfy theoretically substantiated sufficient signs of optimality. In this work, we use the PSC-algorithm for the NP-hard scheduling problem “Minimization of the total weighted completion time of jobs on a single machine with precedence relations given by a directed acyclic graph”.

Non-clairvoyant Scheduling with Predictions

In the single-machine non-clairvoyant scheduling problem, the goal is to minimize the total completion time of jobs whose processing times are unknown a priori . We revisit this well-studied problem and consider the question of how to effectively use (possibly erroneous) predictions of the processing times. We study this question from ground zero by first asking what constitutes a good prediction; we then propose a new measure to gauge prediction quality and design scheduling algorithms with strong guarantees under this measure. Our approach to derive a prediction error measure based on natural desiderata could find applications for other online problems.

Multitasking scheduling with shared processing

AbstractRecently, the problem of multitasking scheduling has raised a lot of interest in the service industries. Hall et al. (Discrete Applied Mathematics, 2016) proposed a shared processing multitasking scheduling model which allows a team to continue to work on the primary tasks while processing the routinely scheduled activities as they occur. With a team being modeled as a single machine, the processing sharing of the machine is achieved by allocating a fraction of the processing capacity to routine jobs and the remaining fraction, which we denote as sharing ratio, to the primary jobs. In this paper, we generalize this model to parallel machines and allow the fraction of the processing capacity assigned to routine jobs to vary from one to another. The objectives are minimizing makespan and minimizing the total completion time of primary jobs. We show that for both objectives, there is no polynomial time approximation algorithm unless P=NP if the sharing ratios are arbitrary for all machines. Then we consider the problems where the sharing ratios on some machines have a constant lower bound. For each objective, we analyze the performance of the classical scheduling algorithms and their variations and then develop a polynomial time approximation scheme when the number of machines is a constant.

CarbonScaler: Leveraging Cloud Workload Elasticity for Optimizing Carbon-Efficiency

Cloud platforms are increasing their emphasis on sustainability and reducing their operational carbon footprint. A common approach for reducing carbon emissions is to exploit the temporal flexibility inherent to many cloud workloads by executing them in periods with the greenest energy and suspending them at other times. Since such suspend-resume approaches can incur long delays in job completion times, we present a new approach that exploits the elasticity of batch workloads in the cloud to optimize their carbon emissions. Our approach is based on the notion of "carbon scaling," similar to cloud autoscaling, where a job dynamically varies its server allocation based on fluctuations in the carbon cost of the grid's energy. We develop a greedy algorithm for minimizing a job's carbon emissions via carbon scaling that is based on the well-known problem of marginal resource allocation. We implement a CarbonScaler prototype in Kubernetes using its autoscaling capabilities and an analytic tool to guide the carbon-efficient deployment of batch applications in the cloud. We then evaluate CarbonScaler using real-world machine learning training and MPI jobs on a commercial cloud platform and show that it can yield i) 51% carbon savings over carbon-agnostic execution; ii) 37% over a state-of-the-art suspend-resume policy; and iii) 8 over the best static scaling policy.

A Novel Model for Dynamic Manufacturing Service Collaboration on Industrial Internet

Industrial Internet enables distributed manufacturing enterprises to efficiently and promptly respond to the requirements of stakeholders using a manufacturing service collaboration chain (MSCC) composed of networked enterprises. However, various dynamic uncertainties may interrupt the MSCC, such as device malfunctions, urgent order insertions, and dynamic logistics, resulting in inexactitude practical applications. This paper proposes a novel reliability-based dynamic manufacturing service collaboration optimization (R-DMSCO) model for uncertain manufacturing collaboration procedures on Industrial Internet. The RDMSCO model reformulates the MSCC reliability in the form of an expectation-standard deviation of uncertain job completion time described by discrete scenarios pertaining to the uncertain perturbation of processing time and logistics time. Subsequently, an enhanced multi-objective artificial bee colony (EMOABC) algorithm that embeds four improvements is intended to address the manufacturing service collaboration optimization (MSCO) problem. The experimental results demonstrate that EMOABC outperforms other typical multi-objective algorithms for MSCO problems. Additionally, the R-DMSCO model can cope with dynamic uncertainties with better robustness and stability than two other effective strategies for dynamic manufacturing service collaboration.

GPARS: Graph predictive algorithm for efficient resource scheduling in heterogeneous GPU clusters

Efficient resource scheduling in heterogeneous graphics processing unit (GPU) clusters are critical for maximizing system performance and optimizing resource utilization. However, prior research in resource scheduling algorithms typically employed machine learning (ML) algorithms to estimate job durations or GPU utilization in the cluster based on training progress and task speed. Regrettably, these studies often overlooked the performance variations among different GPU types within these clusters, as well as the presence of spatiotemporal correlations among jobs. To address these limitations, this paper introduces the graph predictive algorithm for efficient resource scheduling (GPARS) designed specifically for heterogeneous clusters. GPARS leverages spatiotemporal correlations among jobs and utilizes graph attention networks (GANs) for precise job duration prediction. Building upon the prediction results, we develop a dynamic objective function to allocate suitable GPU types for newly submitted jobs. By conducting a comprehensive analysis of Alibaba’s heterogeneous GPU cluster, we delve into the impact of GPU capacity and type on job completion time (JCT) and resource utilization. Our evaluation, using real traces from Alibaba and Philly, substantiates the effectiveness of GPARS. It achieves a remarkable 10.29% reduction in waiting time and an average improvement of 7.47% in resource utilization compared to the original scheduling method. These findings underscore GPARS’s superior performance in enhancing resource scheduling within heterogeneous GPU clusters.

A Hybrid Optimization Algorithm for Efficient Virtual Machine Migration and Task Scheduling Using a Cloud-Based Adaptive Multi-Agent Deep Deterministic Policy Gradient Technique

This To achieve optimal system performance in the quickly developing field of cloud computing, efficient resource management—which includes accurate job scheduling and optimized Virtual Machine (VM) migration—is essential. The Adaptive Multi-Agent System with Deep Deterministic Policy Gradient (AMS-DDPG) Algorithm is used in this study to propose a cutting-edge hybrid optimization algorithm for effective virtual machine migration and task scheduling. An sophisticated combination of the War Strategy Optimization (WSO) and Rat Swarm Optimizer (RSO) algorithms, the Iterative Concept of War and Rat Swarm (ICWRS) algorithm is the foundation of this technique. Notably, ICWRS optimizes the system with an amazing 93% accuracy, especially for load balancing, job scheduling, and virtual machine migration. The VM migration and task scheduling flexibility and efficiency are greatly improved by the AMS-DDPG technology, which uses a powerful combination of deterministic policy gradient and deep reinforcement learning. By assuring the best possible resource allocation, the Adaptive Multi-Agent System method enhances decision-making even more. Performance in cloud-based virtualized systems is significantly enhanced by our hybrid method, which combines deep learning and multi-agent coordination. Extensive tests that include a detailed comparison with conventional techniques verify the effectiveness of the suggested strategy. As a consequence, our hybrid optimization approach is successful. The findings show significant improvements in system efficiency, shorter job completion times, and optimum resource utilization. Cloud-based systems have unrealized potential for synergistic optimization, as shown by the integration of ICWRS inside the AMS-DDPG framework. Enabling a high-performing and sustainable cloud computing infrastructure that can adapt to the changing needs of modern computing paradigms is made possible by this strategic resource allocation, which is attained via careful computational utilization.

A Study Resource Optimization Techniques Based Job Scheduling in Cloud Computing

Cloud computing has revolutionized the way businesses and individuals utilize computing resources. It offers on-demand access to a vast pool of virtualized resources, such as processing power, storage, and networking, through the Internet. One of the key challenges in cloud computing is efficiently scheduling jobs to maximize resource utilization and minimize costs. Job scheduling in cloud computing involves allocating tasks or jobs to available resources in an optimal manner. The objective is to minimize job completion time, maximize resource utilization, and meet various performance metrics such as response time, throughput, and energy consumption. Resource optimization techniques play a crucial role in achieving these objectives. Resource optimization techniques aim to efficiently allocate resources to jobs, taking into account factors like resource availability, job priorities, and constraints. These techniques utilize various algorithms and optimization approaches to make intelligent decisions about resource allocation. Research on resource optimization techniques for job scheduling in cloud computing is of significant importance due to the following reasons: Efficient Resource Utilization: Cloud computing environments consist of a large number of resources that need to be utilized effectively to maximize cost savings and overall system performance. By optimizing job scheduling, researchers can develop algorithms and techniques that ensure efficient utilization of resources, leading to improved productivity and reduced costs. Performance Improvement: Job scheduling plays a crucial role in meeting performance metrics such as response time, throughput, and reliability. By designing intelligent scheduling algorithms, researchers can improve the overall system performance, leading to better user experience and customer satisfaction. Scalability: Cloud computing environments are highly scalable, allowing users to dynamically scale resources based on their needs. Effective job scheduling techniques enable efficient resource allocation and scaling, ensuring that the system can handle varying workloads without compromising performance. Energy Efficiency: Cloud data centres consume significant amounts of energy, and optimizing resource allocation can contribute to energy conservation. By scheduling jobs intelligently, researchers can reduce energy consumption, leading to environmental benefits and cost savings for cloud service providers. Quality of Service (QoS): Cloud computing service providers often have service-level agreements (SLAs) that define the QoS requirements expected by users. Resource optimization techniques for job scheduling can help meet these SLAs by ensuring that jobs are allocated resources in a timely manner, meeting performance guarantees, and maintaining high service availability. Here in this research, we have used the method of the weighted product model (WPM). For the topic of Resource Optimization Techniques Based Job Scheduling in Cloud Computing For calculating the values of alternative and evaluation parameters. A variation of the WSM called the weighted product method (WPM) has been proposed to address some of the weaknesses of The WSM that came before it. The main distinction is that the multiplication is being used in place of additional. The terms "scoring methods" are frequently used to describe WSM and WPM Execution time on Virtual machine, Transmission time (delay)on Virtual machine, Processing cost of a task on virtual machine resource optimization techniques based on job scheduling play a crucial role in maximizing the efficiency and performance of cloud computing systems. By effectively managing and allocating resources, these techniques help minimize costs, reduce energy consumption, and improve overall system throughput. One of the key findings is that intelligent job scheduling algorithms, such as genetic algorithms, ant colony optimization,

Interference-aware opportunistic job placement for shared distributed deep learning clusters

Distributed deep learning frameworks facilitate large deep learning workloads. These frameworks support sharing one GPU device among multiple jobs to improve resource utilization. Modern deep learning training jobs consume a large amount of GPU memory. Despite that, sharing GPU memory among jobs is still possible because a training job has iterative steps that its memory usage fluctuates over time. However, resource sharing also introduces the risk of job performance degradation. Co-located jobs sharing a GPU device may suffer from different levels of interference, mainly caused by memory oversharing. How to improve resource utilization while maintaining good job performance is a novel challenge for job placement strategies. This paper studies the job placement problem. We propose an opportunistic memory sharing model to describe the time-varying job memory requirements. Based on this model, we introduce an Opportunistic Job Placement Problem (OJPP) for shared GPU clusters that seek job placement configurations using a minimum number of GPU devices and guarantee user-defined performance requirements at the same time. We propose a greedy algorithm and a heuristic algorithm with computational complexities of O(nlog⁡n) and O(n2log⁡n), respectively, to solve the problem. We also propose an online adjustment algorithm with the computational complexity of O(nlog⁡n) to perform updates to job placement configurations in runtime. A machine-learning-based interference prediction method is used to prepare accurate interference estimations. Extensive experiments are conducted on a GPU cluster to verify the correctness and effectiveness of our algorithms. Compared with standalone training jobs on dedicated clusters, the proposed approach reduces resource consumption by 46% in a shared cluster, while guaranteeing over 92.97% of the job performance, in terms of average job completion time.

Empowering small businesses with the force of big data analytics and AI: A technological integration for enhanced business management

Small and medium-sized businesses (SMEs) in developing economies still face a number of obstacles that prevent them from adopting digital technologies. In contrast, SMEs have achieved greater success in emerging markets. Due to its potential benefits for numerous businesses, machine learning (ML) has become a hot topic in recent years. Particularly a few major organizations, for example, Amazon, Google and Microsoft have shown a few effective cases on coordinating simulated intelligence capacity in their own organizations. This research suggests a fresh method for bettering private companies by combining large-scale data analysis with artificial intelligence and enhanced safety measures. Here, cloud edge administration with task planning using a dynamic joined real channel Kubernetes obstruction task scheduler improves business for the executives. Then, the organization's security is bolstered by a form of differential encryption on the blockchain that takes into account the need for security. We also propose assigning new jobs to the load node with the lightest workload. Experiments show that our strategy shortens job completion times and distributes work evenly across edge nodes. The experimental investigation is conducted in terms of latency, quality of service, energy efficiency, data integrity, and scalability. The proposed technique attained latency of 0.8354, QoS of 0.9395, energy efficiency of 0.9879, data integrity of 0.1189, scalability of 0.8400.

Reasoning About Expected Job Completion Time in Dynamic Vehicular Clouds

Roughly a decade ago, inspired by the phenomenal success of cloud computing, a group of researchers have defined Vehicular Clouds as a group of vehicles whose sensing, communication, and computing resources can be coordinated and allocated to authorized users. While both conventional and Vehicular Clouds are instances of utility computing, a number of important characteristics set vehicular clouds apart from their conventional counterparts. These characteristics include the mobility of vehicles and the volatility of resources that fluctuate with the arrival and departure of vehicles. As in the conventional version of cloud computing, approximating job completion time is one of the key performance metrics of interest. Unfortunately, estimating job completion time with any degree of accuracy and confidence requires complete knowledge of the distributions of several relevant random variables. Typically, however, these probability distributions are not known. Luckily, in many cases of practical relevance, accumulated empirical evidence allows to estimate the first few moments of these random variables. The main contribution of this paper is to offer an accurate approximation of the expected job completion time in Dynamic Vehicular Clouds built on top of vehicles on a highway. For this purpose, we rely on empirical estimates of the first moment of the user job execution time in the absence of any overhead attributable to the Dynamic Vehicular Cloud itself. Our extensive simulations have confirmed that our approximations of the expected job execution time are very accurate.

Hydra: Deadline-Aware and Efficiency-Oriented Scheduling for Deep Learning Jobs on Heterogeneous GPUs

With the rapid proliferation of deep learning (DL) jobs running on heterogeneous GPUs, scheduling DL jobs to meet various scheduling requirements, such as meeting deadlines and reducing job completion time (JCT), is critical. Unfortunately, existing efficiency-oriented and deadline-aware efforts are still rudimentary. They lack the capability of scheduling jobs to meet deadline requirements while reducing total JCT, especially when the jobs have various execution times on heterogeneous GPUs. Therefore, we present Hydra, a novel quantitative cost comparison approach, to address this scheduling issue. Here, the cost represents the total JCT plus a dynamic penalty calculated from the total tardiness (i.e., the delay time of exceeding the deadline) of all jobs. Hydra adopts a sampling approach that exploits the inherent iterative periodicity of DL jobs to estimate job execution times accurately on heterogeneous GPUs. Then, Hydra considers various combinations of job sequences and GPUs to obtain the minimized cost by leveraging an efficient branch-and-bound algorithm. Finally, the results of evaluation experiments on Alibaba traces show that Hydra can reduce total tardiness by 85.8% while reducing total JCT as much as possible, compared with state-of-the-art efforts.

Dependent task offloading with deadline-aware scheduling in mobile edge networks

In the field of the Internet of Things (IoT), Edge computing has emerged as a revolutionary paradigm that offers unprecedented benefits by serving the IoT at the network edge. One of the primary advantages of edge computing is that it reduces the job completion time by offloading tasks at the edge server from the IoT. Typically, a job is made up of dependent tasks in which the output of one task is required as the input to the other. This work proposes a directed cyclic graph model that represents the dependencies among these tasks focusing on jointly optimizing task dependencies with deadline constraints for tasks that are delay-sensitive. Thus, dependent tasks are scheduled while considering their deadlines using priority-aware scheduling. For tasks with no deadlines, the processing is done with First-Come-First-Serve (FCFS) scheduling. The tasks with a priority are offloaded to the suitable edge server for processing by using a priority queue to enhance the task satisfaction rate under deadline constraints. To model the suitable edge server decision, we use the Markov decision process (MDP) that minimizes the total completion time. Additionally, we model the mobility of users while offloading tasks to the edge servers. The throughput results demonstrate that the proposed strategy outperforms random offloading, the highest data rate offloading (HDR), the highest computing device (HCD), and delay-dependent priority-aware offloading (DPTO), by 66.67%, 43.75%, 27.78%, and 4.55%, respectively. Furthermore, the proposed strategy surpasses random, HDR, and HCD offloading in terms of task satisfaction rate by 20.48%, 16.28%, and 12.36%, respectively.

SLearn: A Case for Task Sampling Based Learning for Cluster Job Scheduling

The ability to accurately estimate job runtime properties allows a scheduler to effectively schedule jobs. State-of-the-art online cluster job schedulers use history-based learning, which uses past job execution information to estimate the runtime properties of newly arrived jobs. However, with fast-paced development in cluster technology (in both hardware and software) and changing user inputs, job runtime properties can change over time, which lead to inaccurate predictions. In this paper, we explore the potential and limitation of real-time learning of job runtime properties, by proactively sampling and scheduling a small fraction of the tasks of each job. Such a task-sampling-based approach exploits the similarity among runtime properties of the tasks of the same job and is inherently immune to changing job behavior. Our analytical and experimental analysis of 3 production traces with different skew and job distribution shows that learning in space can be substantially more accurate. Our simulation and testbed evaluation on Azure of the two learning approaches anchored in a generic job scheduler using 3 production cluster job traces shows that despite its online overhead, learning in space reduces the average Job Completion Time (JCT) by 1.28×, 1.56×, and 1.32× compared to the prior-art history-based predictor.

GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning

Training data pre-processing pipelines are essential to deep learning (DL). As the performance of model training keeps increasing with both hardware advancements (e.g., faster GPUs) and various software optimizations, the data pre-processing on CPUs is becoming more resource-intensive and a severe bottleneck of the pipeline. This problem is even worse in the cloud, where training jobs exhibit diverse CPU-GPU demands that usually result in mismatches with fixed hardware configurations and resource fragmentation, degrading both training performance and cluster utilization. We introduce GoldMiner, an input data processing service for stateless operations used in pre-processing data for DL model training. GoldMiner decouples data pre-processing from model training into a new role called the data worker. Data workers facilitate scaling of data pre-processing to anywhere in a cluster, effectively pooling the resources across the cluster to satisfy the diverse requirements of training jobs. GoldMiner achieves this decoupling in a fully automatic and elastic manner. The key insight is that data pre-processing is inherently stateless, thus can be executed independently and elastically. This insight guides GoldMiner to automatically extract stateless computation out of a monolithic training program, efficiently disaggregate it across data workers, and elastically scale data workers to tune the resource allocations across jobs to optimize cluster efficiency. We have applied GoldMiner to industrial workloads, and our evaluation shows that GoldMiner can transform unmodified training programs to use data workers, accelerating individual training jobs by up to 12.1x. GoldMiner also improves average job completion time and aggregate GPU utilization by up to 2.5x and 2.1x in a 64-GPU cluster, respectively, by scheduling data workers with elasticity.

Solving a new bi-objective mathematical model for a hybrid flow shop scheduling problem with robots and fuzzy maintenance time

This paper presents a bi-objective mathematical model of a hybrid flow shop scheduling problem (HFSSP) with robots and fuzzy maintenance time. The proposed model minimizes the mean completion time of jobs and the total cost of the processes based on maintenance costs and incurred transportation costs via robots. A fuzzy chance-constrained programming (FCCP) model is used to present its deterministic-equivalent problem. It is then solved by two multi-objective decision-making approaches, namely LP-metric and goal attainment (GA). A sensitivity analysis of the weights of the objective functions is conducted and the behavior of the model is studied for each instance. Also, a TOPSIS (the technique for order of preference by similarity to ideal solution) method is used to rank the efficiency of the proposed solution approaches. Future research areas are also suggested.

Towards Accelerating Data Intensive Application's Shuffle Process Using SmartNICs

The wide adoption of the emerging SmartNIC technology creates new opportunities to offload application-level computation into the networking layer, which frees the burden of host CPUs, leading to performance improvement. Shuffle, the all-to-all data exchange process, is a critical building block for network communication in distributed data-intensive applications and can potentially benefit from SmartNICs. In this paper, we develop SmartShuffle, which accelerates the data-intensive application's shuffle process by offloading various computation tasks into the SmartNIC devices. SmartShuffle supports offloading both low-level network functions, including data partitioning and network transport, and high-level computation tasks, including filtering, aggregation, and sorting. SmartShuffle adopts a coordinated offload architecture to make sender-side and receiver-side SmartNICs jointly contribute to the benefits of shuffle computation offload. SmartShuffle carefully manages the tight and time-varying computation and memory constraints on the device. We propose a liquid offloading approach, which dynamically migrates operators between the host CPU and the SmartNIC at runtime such that resources in both devices are fully utilized. We prototype SmartShuffle on the Stingray SoC SmartNICs and plug it into Spark. Our evaluation shows that SmartShuffle improves host CPU efficiency and I/O efficiency with lower job completion time. SmartShuffle outperforms Spark, and Spark RDMA by up to 40% on TPC-H.

An Algebraic Approach to the Solutions of the Open Shop Scheduling Problem

The open shop scheduling problem (OSSP) is one of the standard scheduling problems. It consists of scheduling jobs associated with a finite set of tasks developed by different machines. In this case, each machine processes at most one operation at a time, and the job processing order on the machines does not matter. The goal is to determine the completion times of the operations processed on the machines to minimize the largest job completion time, called Cmax. This paper proves that each OSSP has associated a path algebra called Brauer configuration algebra whose representation theory (particularly its dimension and the dimension of its center) can be given using the corresponding Cmax value. It has also been proved that the dimension of the centers of Brauer configuration algebras associated with OSSPs with minimal Cmax are congruent modulo the number of machines.

Online scheduling of coflows by attention-empowered scalable deep reinforcement learning

With the abstraction of parallel data transmission flows being a coflow, data transmissions in large-scale computing jobs can be modeled by a coflow directed acyclic graph (coflow DAG) in which nodes are coflows and edges represent dependencies between coflows. Efficient scheduling of coflows on network links is crucial for reducing the overall communication and job completion time. The known best coflow scheduling method deploying deep reinforcement learning (DRL), DeepWeave (Sun et al., 2020), suffers from poor scalability due to the requirement of O(dn)-size policy network for processing n coflows of d dimensions which is difficult to train. This paper extends the directed acyclic graph neural network (DAGNN) to Pipelined-DAGNN that embeds the features of different stages of input coflow DAGs in pipeline to effectively speed up the feature extraction process. To effectively process the feature vectors of coflow DAGs of arbitrary size and shape without compromising scheduling accuracy (quality), we propose a novel self-attention empowered DRL coflow scheduling model to generate coflow scheduling policies, which enables the scale of policy network depends only on features (dimensions) rather than coflows, without the need of packing all individual embedding vectors from Pipelined-DAGNN into a long flat vector. Our model reduces the size of the policy network in DRL from previously O(dn) to O(d), achieving a high scalability independent of the number of coflows. Simulation results on Facebook trace show that our model reduces the average weighted job completion time by up to 33.88%, apart from being more scalable and robust, compared with the state-of-the-art methods.

Multi-Stage Geo-Distributed Data Aggregation With Coordinated Computation and Communication in Edge Compute First Networking

With the explosion of geo-distributed data, the huge treasures hidden in them are waiting to be explored to obtain valuable insights. This results in the need for an effective geo- distributed data analysis method. The traditional approach to geo-distributed data analytics is to gather all the required data into a single edge datacenter (edge DC) through one transmission and aggregation (centralized data aggregation). However, as the volume of data grows exponentially, the centralized data aggregation scheme becomes inefficient or infeasible due to the limitations of the computing and network resources. In this paper, we propose the geo-distributed data aggregation scheme in edge compute first networking (CFN) with joint consideration of computation and communication resources. The proposed scheme optimizes two objectives: the first is to minimize the job completion time (JCT) by selecting cluster centers, dividing clusters and provisioning lightpaths; the second objective is to reduce bandwidth consumption by reallocating routing and frequency slots based on JCT. To achieve the two objectives, we first formulate the optimization problem of multi-stage geo- distributed data aggregation as a linear programming (LP) model. To tackle the computational complexity issue of the LP model, a multi-stage geo-distributed data aggregation algorithm jointly with computation and communication resources (MGDD-CC) is proposed. Simulation results show that the proposed scheme can reduce JCT, alleviate the competition for bandwidth resources and is more suitable for scenarios with better data aggregation effects and larger quantities of geo-distributed data.