Queue Delay Research Articles

Edge-Intelligence-Powered Joint Computation Offloading and Unmanned Aerial Vehicle Trajectory Optimization Strategy

UAV-based air-ground integrated networks offer a significant benefit in terms of providing ubiquitous communications and computing services for Internet of Things (IoT) devices. With the empowerment of edge intelligence (EI) technology, they can efficiently deploy various intelligent IoT applications. However, the trajectory of UAVs can significantly affect the quality of service (QoS) and resource optimization decisions. Joint computation offloading and UAV trajectory optimization bring many challenges, including coupled decision variables, information uncertainty, and long-term queue delay constraints. Therefore, this paper introduces an air-ground integrated architecture with EI and proposes a TD3-based joint computation offloading and UAV trajectory optimization (TCOTO) algorithm. Specifically, we use the principle of the TD3 algorithm to transform the original problem into a cumulative reward maximization problem in deep reinforcement learning (DRL) to obtain the UAV trajectory and offloading strategy. Additionally, the Lyapunov framework is used to convert the original long-term optimization problem into a deterministic short-term time-slot problem to ensure the long-term stability of the UAV queue. Based on the simulation results, it can be concluded that our novel TD3-based algorithm effectively solves the joint computation offloading and UAV trajectory optimization problems. The proposed algorithm improves the performance of the system energy efficiency by 3.77%, 22.90%, and 67.62%, respectively, compared to the other three benchmark schemes.

Hybrid Clustering Approach (SG-MFOA) using Multipath Cross-Layer Design in MANET Network

Mobile Ad-hoc Networks (MANETs) are used in a wide range of applications because of their unique capabilities. The routing in MANET is regarded as a main challenge on account of its permanent movement and nodes' randomness that can cause a continual change of network topology, thus finding correct paths among the nodes is the fundamental task of routing protocols. There is a rise in protocols that depend on cross-layer communication across several layers to increase the MANET performance. In this manuscript, a Hybrid Clustering Approach (SG-MFOA) using a Multipath Cross-Layer Design in a MANET Network is proposed. Here, multiple routes are selected for data packet transmission with the help of hybrid routing. Furthermore, a cross-layer metric is acquired depending upon Expected Transmission Time (ETT), Residual Energy and Load Balancing Factor. It is necessary to select a Cluster Head (CH) depending on this trade for proficient routing. So, the cluster head is optimally chosen to utilize SG-MFOA. Thus, the multipath route selection is performed by the multi-objective functions containing bandwidth, congestion delay, transmission delay and to ensure the prolonging of network lifetime, queue delay at each connection Access Category from the available routes to a destination. When compared with the existing methods, like a proficient self-attention-based conditional variational auto-encoder generative adversarial networks-based multipath cross-layer design routing paradigm for MANET (SCVAGAN-MCDR-MANET), Energy Aware Cross Layer Routing Protocol to Increase Route Reliability in MANETs (EACLRP-RR-MANET), Cross-layer Adaptive Fuzzy-based Ad hoc On-Demand Distance Vector Routing Protocol for MANET (CLAF-DVRP-MANET), respectively.

Queues with resetting: a perspective

Abstract Performance modeling is a key issue in queuing theory and operation research. It is well-known that the length of a queue that awaits service or the time spent by a job in a queue depends not only on the service rate, but also crucially on the fluctuations in service time. The larger the fluctuations, the longer the delay becomes and hence, this is a major hindrance for the queue to operate efficiently. Various strategies have been adapted to prevent this drawback. In this perspective, we investigate the effects of one such novel strategy namely resetting or restart, an emerging concept in statistical physics and stochastic complex process, that was recently introduced to mitigate fluctuations-induced delays in queues. In particular, we show that a service resetting mechanism accompanied with an overhead time can remarkably shorten the average queue lengths and waiting times. We examine various resetting strategies and further shed light on the intricate role of the overhead times to the queuing performance. Our analysis opens up future avenues in operation research where resetting-based strategies can be universally promising.

A Fuzzy-PI Clock Servo with Window Filter for Compensating Queue-Induced Delay Asymmetry in IEEE 1588 Networks.

Clock synchronization is one of the popular research topics in Distributed Measurement and Control Systems (DMCSs). In most industrial fields, such as Smart Grid and Flight Test, the highest requirement for synchronization accuracy is 1 μs. IEEE 1588 Precision Time Protocol-2008 (PTPv2) can theoretically achieve sub-microsecond accuracy, but it relies on the assumption that the forward and backward delays of PTP packets are symmetrical. In practice, PTP packets will experience random queue delays in switches, making the above assumption challenging to satisfy and causing poor synchronization accuracy. Although using switches supporting the Transparent Clock (TC) can improve synchronization accuracy, these dedicated switches are generally expensive. This paper designs a PTP clock servo for compensating Queue-Induced Delay Asymmetry (QIDA), which can be implemented based on ordinary switches. Its main algorithm comprises a minimum window filter with drift compensation and a fuzzy proportional-integral (PI) controller. We construct a low-cost hardware platform (the cost of each node is within USD 10) to test the performance of the clock servo. In a 100 Mbps network with background (BG) traffic of less than 70 Mbps, the maximum absolute time error (max |TE|) does not exceed 0.35 μs, and the convergence time is about half a minute. The accuracy is improved hundreds of times compared with other existing clock servos.

A Priority-Based Self-Adaptive Random Early Detection Algorithm in IoT Gateways

Effective algorithms for queue management are crucial in place of guaranteeing maximum efficiency in gateway routers since network traffic continues to expand dramatically. An online researcher has suggested the Active Queue Management (AQM) strategy regarding the upcoming generation of gateway switches. The common active queue scheme remains (RED) Random Early Detection. Random early detection is susceptible to parameterization issues and lacks a self-adaptation mechanism. Several RED variants have been formed; nevertheless, variations in traffic load have an adverse effect on all of them. Due to the fact that each has a static drop pattern, to address the RED and its variation schemes, the SARED system, or the design of self-adaptive random early detection was created. But in order to prevent congestion, during the time when the queue length surpasses a present maximum threshold limit, SARED aggressively removes packets. This causes networks having a lot of traffic situations the average is expected to increase queue delay, so in those cases, SARED should be less aggressive. This paper develops a priority-based queuing congestion control method for IoT gateways to manage network congestion. Our method (priority-based algorithms) performs substantially better with regard to throughput, delay, and packet loss than the present methods of SARED. The outcomes of the conducted simulation experiments have shown that in scenarios with heavy traffic loads, priority-based self-adaptive random early detection (PSARED) has greatly decreased average queuing delay by 3%, minimized average throughput by 1%, and decreased the rate of packet loss by 10% in contrast to SARED.

Software-Defined Time-Sensitive Networking for Cross-Domain Deterministic Transmission

With the rapid development of Industrial 4.0, massive emerging time-critical applications pose new demands for industrial networks, such as strict latency boundaries, ultra-reliable transmission, and so on. Time-Sensitive Networking (TSN) is well suited for these demanding scenarios due to its support for deterministic transmission based on Ethernet. However, many use cases in real industrial scenarios involve non-TSN domains, and, thus, the network requires providing deterministic transmission across non-TSN domains to support them. To this end, this paper proposes a novel Software-Defined Time-Sensitive Networking (SD-TSN) framework that integrates the determinism of TSN and scalability of Software-Defined Networking (SDN). SD-TSN exploits the coordination between the coordinated controller and different domain controllers to bound non-deterministic queuing delays in a way that guarantees determinism. In particular, we designed a multi-domain time-aware traffic scheduling model, which harmonizes the time-aware shaper (TAS) in TSN and time-slots in non-TSN domains based on a global view, generating transmission schedules for each domain. A prototype testbed was built to conduct the experiments. The evaluation results demonstrate that the proposed method can efficiently achieve bounded delay and low delay variations in the transmission across non-TSN domains.

Distributed Traffic Signal Optimization at V2X Intersections

This paper presents our research on a traffic signal control system (TSCS) at V2X intersections. The overall objective of the study is to create an implementable TSCS. The specific objective of this paper is to investigate a distributed system towards implementation. The objective function of minimizing queue delay is formulated as the integral of queue lengths. The discrete queueing estimation is mixed with macro and micro traffic flow models. The novel proposed architecture alleviates the communication network bandwidth constraint by processing BSMs and computing queue lengths at the local intersection. In addition, a two-stage distributed system is designed to optimize offsets, splits, and cycle length simultaneously and in real time. The paper advances TSCS theories by contributing a novel analytic formulation of delay functions and their first degree of derivatives for a two-stage optimization model. The open-source traffic simulation engine Enhanced Transportation Flow Open-Source Microscopic Model (ETFOMM version 1.2) was selected as a simulation environment to develop, debug, and evaluate the models and the system. The control delay of the major direction, minor direction, and the total network were collected to assess the system performance. Compared with the optimized TSCS timing plan by the Virginia Department of Transportation, the system generated a 21% control delay reduction in the major direction and a 7% control delay reduction in the minor direction at just a 10% penetration rate of connected vehicles. Finally, the proposed distributed and centralized systems present similar performances in the case study.

SOFTWARE GENERATORS IN THE GPSS WORLD ENVIRONMENT FOR DISTRIBUTIONS-PROBABILITY MIXTURES WITH QUALITYASSESSMENT RESULTS

This article presents quality assessment results for the software generators of pseudo-random sequences (PRS) presented in [1; 2] for QS simulation, including second-order hyper-Erlang and hyper-exponential distributions of the second order, that flows organically from these works. These distribution laws are in demand in queuing theory as it provides a big range of changes in the variation coefficient which plays an important role in the average queue delay calculating. There is no data on such generators neither in scientific literature, no in the GPSS WORLD library. Software generator quality evaluation, regardless of the programming language, is usually made according to statistical tests. In this case, chi-squared and Kolmogorov-Smirnov’s tests are used. In addition, the comparison between theoretical and statistical aspects of the corresponding order is provided. Unlike simple distribution laws, for which the use of the above mentioned statistical tests is provided for in well-known software products and does not cause any difficulties, in our case automatic testing is impossible. The well-known packages STATISTICA, STATGAPHICS, Matlab/Simulink, etc. do not provide for the use of the hyper-Erlang and hyper-exponential distribution laws we consider. Therefore, the Pearson statistical test is calculated and checked manually. The presented results should be useful for professionals of discrete event modeling in the GPSS WORLD environment.

Experimental Analysis and Optimization Approach of Self-Clocked Rate Adaptation for Multimedia Congestion Control Algorithm in Emulated 5G Environment.

The congestion problem has driven many researchers to address it, among other networking issues. In a packet-switched network, congestion is essential; it leads to a high response time to deliver packets due to heavy traffic, which eventually causes packet loss. Hence, congestion control mechanisms are utilized to prevent such cases. Several interesting algorithms are proposed to focus on this dilemma, such as the Self-Clocked Rate Adaptation for Multimedia (SCReAM) designed for interactive real-time video streaming applications. One of the main issues of SCReAM is the high design complexity due to the large size of its documentation and coding. Furthermore, there is a considerable number of parameters that can be adjusted to accomplish the desired performance. This study proposes a guided parameters' tuning approach to assess and optimize the SCReAM algorithm in an emulated 5G environment through a detailed exploration of its parameters. The proposed approach consists of three phases, namely, the initialization phase, the standalone experimentation phase, and the hybrid experimentation phase. In the first phase, we illustrate the method of initializing and implementing the environment, followed by specifying the investigated parameters' settings, testing, and validation. The second phase aims to investigate SCReAM parameters in isolation to identify the effect on the performance in relation to network queue delay, smoothed Round Trip Time (sRTT), and throughput. The final phase discusses the possibility of achieving the optimum performance by combining various sets to provide researchers with clear and explicit guidelines to establish an adequate SCReAM behavior for the desired application. To the best of our knowledge, this is the first study that proposes a preliminary and comprehensive analysis of the SCReAM algorithm. Based on the proposed approach, when L4S/ECN is disabled, we reduced the network queue delay by 63.36% and increased the network throughput by 48.6% as compared to the results generated by the original design. In L4S/ECN-enabled mode, the network queue delay is reduced by 16.17% while the network throughput increased by 93%.

Improving QoS in mobile multimedia streaming with SCTP-PQ

The Stream Control Transmission Protocol (SCTP) is often the preferred transport layer protocol in streaming applications. This protocol combines the best aspects of Transmission Control Protocol (TCP) and User Datagram Protocol (UDP), but also offers additional features. SCTP supports multihoming and multi-streaming applications and has a congestion mechanism like TCP. Media streaming consists of different types of frames with different levels of importance. For example, I-frames carry more information than B-frames in Moving Picture Experts Group (MPEG). Usually, MPEG frames are processed using the First-In-First-Out (FIFO) algorithm. In this paper, a four-level priority queue integrated protocol named SCTP Priority Queue (SCTP-PQ) has been developed to reduce jitter and delay in real-time multimedia streaming for mobile devices. As part of the development, priority and retransmitted packets are determined on the sending side and these packets are processed faster by using the priority queue on the receiving side. In this way, the average queue delay of priority packets on the receiving side is reduced by 90 % and the throughput values are increased by an average of 10 times. The developed protocol has been extensively tested and compared with SCTP. The results show that the SCTP-PQ outperforms the standard SCTP in terms of jitter and delay.

Adaptive Modulation Based on Nondata-Aided Error Vector Magnitude for Smart Systems in Smart Cities

A smart city involves big data transmission (BDT) between smart systems, which increases queue delays and leads to difficulty in enhancing the spectral efficiency. Adaptive modulation is an effective technique for enhancing data transmission rates in smart systems. However, traditional adaptive modulation approaches are not suitable for BDT in smart systems because the delays caused by the large amount of transmitted data lead to difficulty in evaluating the channel quality. In this paper, we propose a nondata-aided error vector (NDA-EVM) that can be employed in adaptive modulation over wireless channels. The proposed NDA-EVM can be used to evaluate the channel quality and symbol error rate (SER), which reflect the quality of service (QoS) of the system. We formulated the relationship between the NDA-EVM and SER, which provides a basis for designing adaptive modulation techniques for smart systems. To address the low average spectrum efficiency (ASE) caused by BDT queue delays, an adaptive modulation strategy based on the finite-state Markov chain (FSMC) of the NDA-EVM (i.e., NDA-EVM-AM) was designed. This method simplifies the adaptive modulation algorithm for smart systems to search for the optimal transfer probability in the FSMC matrix based on two typical states: the resident state and transient state. Moreover, we proposed an analytical procedure to describe queuing behavior to analyze the performance of the NDA-EVM-AM algorithm for smart systems in smart cities. The performance is compared with that of a conventional adaptive modulation algorithm through simulations. The results show that compared with traditional adaptive modulation, NDA-EVM-AM obtains a lower packet loss rate and higher spectral efficiency for smart systems.

A deep learning based latency aware predictive routing model for network‐on‐chip architectures

SummaryNetwork on Chip (NoC) is an evolving platform for communications related applications, which are executed on a single silicon chip. There are several routing models in NoC architectures, but the accuracy of these models is limited, and the existing models are degraded because of over and under fitting issues. This research introduces the new deep learning‐based latency aware predictive routing model for on‐chip networks to route packets with better performance and power efficiency. The deep learning model used in this research is a new convolutional residual gated recurrent unit (CRGRU) with queuing theory. Moreover, the source and channel queuing delay is comprised of features to learn spatial and sequential information that improves the overall prediction accuracy. This router is modified by the intrusion of the Router States Monitor unit and the CRGRU hardware engine. The work is executed using the Xilinx platform, and the performance measures like latency and throughput are obtained by varying the network size as , , and and also varying the buffer space and length as , , and , respectively. In addition, the squared correlation coefficient (SCC) and normalized root mean square error (NRMSE) are evaluated and compared with existing learning models to validate the proposed model.

Edge–Cloud Collaborative Computation Offloading for Mixed Traffic

Edge computing can avoid long network transmission delay, thus reducing the task response time. However, edge servers cannot respond quickly to massive computing requests due to limited computing power, which may lead to long queuing delays, thus failing to meet low latency requirements of delay-sensitive Internet of Things (IoT) applications. Furthermore, the per-computing-unit energy consumption in edge computing is much higher than that in cloud computing. This article studies the edge–cloud collaborative computing for energy efficiency while meeting the delay requirements of IoT applications in mobile edge computing systems. We formulate the energy efficient computation offloading problem for mixed tasks, including divisible and indivisible tasks. We theoretically analyze the relationship between delay guarantee and energy consumption in edge–cloud collaborative environment. We further analyze the system stability based on Lyapunov stability theory. Then, an energy efficient and delay guaranteed edge–cloud collaborative computation offloading algorithm is proposed to provide delay guarantee for delay-sensitive IoT applications while minimizing the energy consumption of the edge–cloud computing system. Theoretical analysis and experimental results show that, compared with the benchmark algorithms, our algorithm can provide strict delay guarantee for stringent-delay-sensitive IoT applications, and low delay for soft-delay-sensitive IoT applications, while consuming lower energy.

High Throughput VMs Placement With Constrained Communication Overhead and Provable Guarantees

Placement of VMs in the cloud is one of the most fundamental problems in systems research. Traditionally, placement algorithms assume that the schedulers have complete information about the currently available resources at each host. However, this assumption is in many cases unrealistic, as gathering fresh status information from each of the thousands of hosts in a large data center incurs excessive communication overhead, which results in long queueing delays. Efforts to resolve this problem by employing several parallel schedulers typically exhibit collisions when several schedulers are simultaneously trying to place VMs on the same host. Our work analyzes the performance of various placement algorithms and provides empirical evidence that using multiple randomized schedulers obtains high throughput, while significantly decreasing both the communication overhead, and the number of collisions between schedulers. We, therefore, introduce Adaptive Partial State Random (APSR) – an efficient parallel random resource management algorithm that samples only from a small number of hosts and dynamically adjusts the degree of parallelism to provide provable guarantees on the probability of collisions between distinct schedulers. We formally analyze APSR, evaluate it on real workloads, and integrate it into the popular OpenStack cloud management platform. Our evaluation shows that APSR matches the throughput provided by other parallel schedulers, while achieving up to 13x lower decline ratio and a reduction of over 85% in communication overheads.

On Performance Analysis for Random 3D Mobile AUV Networks With Limited Data Buffers

In this paper, we consider a finite network of multiple mobile autonomous underwater vehicles (AUVs) collectingdata in a given region and transferring them to a surface sink. All AUVs patrol a cylindrical region and dynamically control their 3-Dimensional (3D) location according to the random way-point (RWP) mobility model. Our specific interest focuses on the system throughput under different locations of the serving AUV. The following three serving AUV selection policies are investigated: 1)The serving AUV is the closest AUV among all active AUVs; 2) The serving AUV is randomly selected among the active AUVs; 3)The serving AUV is the farthest AUV among all active AUVs. Under the time slotted network scenario, the system throughput is defined as the probability that the sink can successfully recover the data packet from the serving AUV in one time slot. Further, we consider a practical constraint that all AUVs are equipped with limited data buffers. Based on the probability distributions of the distances between the AUVs and the surface sink, this paper presents a mathematical framework to characterize the performance of the random 3D mobile network. With the help of this framework, we obtain the theoretical performance in terms of the system throughput of the surface sink and the average queue delay of any AUV. Finally, extensive simulations are presented to verify the accuracy of our analysis and study the effects of various parameters.

A Sketch-Based Fine-Grained Proportional Integral Queue Management Method

The phenomenon “bufferbloat” occurs when the buffers of the network intermediary nodes fill up, causing long queuing delays. This has a significant negative impact on the quality of service of network applications, particularly those that are sensitive to time delay. Many active queue management (AQM) algorithms have been proposed to overcome this problem. Those AQMs attempt to maintain minimal queuing delays and good throughput by purposefully dropping packets at network intermediary nodes. However, the existing AQM algorithms mostly drop packets randomly based on a certain metric such as queue length or queuing delay, which fails to achieve fine-grained differentiation of data streams. In this paper, we propose a fine-grained sketch-based proportional integral queue management algorithm S-PIE, which uses an additional measurement structure Sketch for packet frequency share judgment based on the existing PIE algorithm for the fine-grained differentiation between data streams and adjust the drop policy for a differentiated packet drop. Experimental results on the NS-3 simulation platform show that the S-PIE algorithm achieves lower average queue length and RTT and higher fairness than PIE, RED, and CoDel algorithms while maintaining a similar throughput performance, maintaining network availability and stability, and improving network quality of service.

A Distance-Weighted Dynamic Bandwidth Allocation Algorithm for Improved Performance in Long-Reach Passive Optical Networks for Next Generation Networks

In recent years, there has been an increasing trend towards extending the coverage of passive optical networks (PONs) over large geographical areas. Long-reach PONs (LRPONs) are capable of extending the distance covered by PONs from 20 km to 100 km, leading to cost savings in the network operation by reducing the number of central offices. They have become widely deployed due to their ability to provide high-speed, long-distance data transmission over optical fibers. In addition, the next generation of optical access networks are expected to provide high-capacity mobile and wireless backhauling over a wide coverage area. However, this extended reach also requires the design of efficient dynamic bandwidth allocation (DBA) schemes to address the performance degradation caused by the increased propagation delay in LRPONs. The DBA schemes commonly used for upstream traffic transmission in traditional PONs are not well-suited for use in LRPONs due to their inefficiency in bandwidth utilization due to the increased round-trip time (RTT) between the optical line terminal (OLT) and the optical network unit (ONU). In this study, we present an efficient DBA algorithm, the Distance-Weighted Bandwidth Allocation DWDBA Algorithm, specifically enhanced for multi-wavelength LRPONs. Our DBA algorithm utilizes a scheduling policy that assigns weight vectors to Optical Network Units (ONUs) based on their distance from the Optical Line Terminal (OLT), sorting them accordingly without penalizing any ONU due to their distance. The DWDBA takes the laser tuning time into consideration. We conducted extensive simulations to evaluate the performance of the proposed algorithm under various scenarios and compared it to the IPACT algorithm. The results of the simulations show that the proposed algorithm outperformed the IPACT algorithm in terms of bandwidth utilization and queue delay.

Using submodularity in solving the robust bandwidth packing problem with queuing delay guarantees

This paper addresses the robust bandwidth packing problem with queuing delay guarantees. The queuing delays are approximated using the M/G/1 model. The problem involves nonlinear constraints that aim to limit the total queuing delays and satisfy edge capacities. We present two types of reformulations for this intractable problem. The first type consists of conic quadratically constrained or linear programs with integer variables, which can be implemented using commercial solvers. The second type is mixed-integer linear programs with an exponential number of constraints. To address the nonlinear constraints, we reformulate them into multiple linear constraints by combining polymatroid or submodular inequalities with the supporting hyperplanes of a concave function. The computational results demonstrate the effectiveness of the proposed formulations.

MODELING TRAFFIC OPERATIONS AT A TOLL PLAZA USING DISCRETE-EVENT SIMULATION

The objective of this study is to develop a discrete-event simulation model of traffic operations at a toll plaza to compare the performance measures between the manual and electronic toll collection (ETC) systems. This study also attempts to simulate the scenarios where the number of servers in these systems is reallocated to improve the overall operational performance. The field data were collected at a toll plaza on an expressway on-ramp during a PM peak period via video recording. The collected data were summarized to determine the characteristics of parameters as inputs for the simulation model. The simulation model was developed in EZStrobe, while the results were carried out via STROBOSCOPE. The simulation model was calibrated with the field data prior to making modifications and simulating various scenarios. The results show that, under the existing conditions, switching an electronic tollgate to a manual tollgate reduces the average overall queue delay and queue length by 71% and 78%, respectively. After reallocation, the average queue delays in these systems equalize as the proportion of ETC users reaches 72%. The additional throughput time of 0.4 s/veh in the ETC system is small compensation for the 6.8-second deduction in the average overall throughput time. In conclusion, shortening the queue lengths at the toll plaza by efficiently utilizing the existing tollgates is possible for improving the overall operational performance of the toll plaza without the additional tollgate installation.

Multi-Agent Learning-Based Optimal Task Offloading and UAV Trajectory Planning for AGIN-Power IoT

UAV-based air-ground integrated computing networks (AGIN) have gained significant traction in remote areas for the Power Internet of Things (PIoT). This paper considers an AGIN-PIoT, where computing tasks generated by ground PIoT devices are offloaded to aerial UAVs that perform edge computing. Jointly optimizing task offloading and UAV trajectory poses challenges such as many decision variables, information uncertainty, and long-term queue delay constraints. Due to the limited battery capacity of PIoT devices and UAVs, our objective is to minimize system energy consumption under long-term queue delay constraints by jointly optimizing task offloading, trajectory planning, and computing resource assignment. In light of Lyapunov optimization, we decompose the original challenging optimization problem into two sub-problems: (1) task offloading and UAV trajectory planning and (2) aerial edge resource allocation. Accordingly, we develop a multi-agent deep reinforcement learning-based algorithm called AGIN-MADDPG for the former to achieve the maximum accumulative reward and propose a greedy solution for the latter. Extensive experiments and numerical results demonstrate that our approach can avoid the problem of gradient vanishing and outperforms other benchmark methods in terms of power consumption, task backlog, queue delay, and system throughput.