Dynamic Computation Offloading Research Articles

Optimization of resource allocation strategy for high-speed railway based on deep reinforcement learning

With the accelerated development of high-speed railway (HSR), the contradiction between the surge of user services and the demand for resource has become increasingly prominent. Mobile edge computing (MEC) has emerged to improve performance, reduce communication delay and ease network load. In this paper, we design a multi-user MEC system framework that aims to solve the joint optimization problem of computation offloading and resource allocation in HSR communication scenario with deep reinforcement learning algorithm. The framework dynamically allocates computation resource and network bandwidth through the real-time distance between users and base station (BS) to achieve optimal resource utilization and maximize user experience. To achieve this goal, we use a deep reinforcement learning based dynamic computation offloading and resource allocation (DDCORA) optimization algorithm. The algorithm minimizes the system cost by sharing state information among different users and making collaborative decisions to rationally allocate spectrum resource and computation resource. Simulation results show that DDCORA algorithm can significantly decrease the system cost while enhancing the overall system performance and user experience.

QoS and Fairness Oriented Dynamic Computation Offloading in the Internet of Vehicles Based on Estimate Time of Arrival

QoS and Fairness Oriented Dynamic Computation Offloading in the Internet of Vehicles Based on Estimate Time of Arrival

Online dynamic multi-user computation offloading and resource allocation for HAP-assisted MEC: an energy efficient approach

Nowadays, the paradigm of mobile computing is evolving from a centralized cloud model towards Mobile Edge Computing (MEC). In regions without ground communication infrastructure, incorporating aerial edge computing nodes into network emerges as an efficient approach to deliver Artificial Intelligence (AI) services to Ground Devices (GDs). The computation offloading and resource allocation problem within a HAP-assisted MEC system is investigated in this paper. Our goal is to minimize the energy consumption. Considering the randomness and dynamism of the task arrival of GDs and the quality of wireless communication, stochastic optimization techniques are utilized to transform the long-term dynamic optimization problem into a deterministic optimization problem. Subsequently, the problem is further decomposed into three sub-problems which can be solved in parallel. An online Energy Efficient Dynamic Offloading (EEDO) algorithm is proposed to address these problems. Then, we conduct the theoretical performance analysis for EEDO. Finally, we carry out parameter analysis and comparative experiments, demonstrating that the EEDO algorithm can effectively reduce system energy consumption while maintaining the stability of the system.

Dynamic Computation Offloading with Deep Reinforcement Learning in Edge Network

With the booming proliferation of user requests in the Internet of Things (IoT) network, Edge Computing (EC) is emerging as a promising paradigm for the provision of flexible and reliable services. Considering the resource constraints of IoT devices, for some delay-aware user requests, a heavy-workload IoT device may not respond on time. EC has sparked a popular wave of offloading user requests to edge servers at the edge of the network. The orchestration of user-requested offloading schemes creates a remarkable challenge regarding the delay in user requests and the energy consumption of IoT devices in edge networks. To solve this challenge, we propose a dynamic computation offloading strategy consisting of the following: (i) we propose the concept of intermediate nodes, which can minimize the delay in user requests and the energy consumption of the current tasks handled by IoT devices by dynamically combining task-offloading and service migration strategies; (ii) based on the workload of the current network, the intermediate node selection problem is modeled as a multi-dimensional Markov Decision Process (MDP) space, and a deep reinforcement learning algorithm is implemented to reduce the large MDP space and make a fast decision. Experimental results show that this strategy is superior to the existing baseline methods to reduce delays in user requests and the energy consumption of IoT devices.

Multi-User Dynamic Computation Offloading and Resource Allocation in 5G MEC Heterogeneous Networks with Static and Dynamic Subchannels

With the rapid development of Mobile Edge Computing (MEC) technology, the computationally intensive requests of end devices can be offloaded to MEC servers directly, which equipped at the edge of wireless networks. Through offloading, the performances such as the execution delay as well as the energy consumption can be effectively improved, which can significantly enhance the quality of user experience. Given the dynamics and randomness of computation requests arrival, the energy in the battery, the radio network environment, and the computation resource in the MEC server, it is a challenge to perform efficient offloading. Based on these problems, this paper proposes dynamic optimization schemes with queuing theory for the cases of the static subchannel and dynamic subchannel during a time slot separately in 5G MEC heterogeneous networks with multiple MDs equipped with the function of energy harvesting. In the schemes, offloading decisions and radio allocation strategies will be dynamically coordinated. They are also jointly allocated along with changing wireless communication resources and computation demands aiming to minimize the system average execution delay. Specifically, it is assumed that the offloading requests can be transmitted through either macro base stations or small base stations. In the case of the static subchannel, a joint resource allocation and computation offloading scheme based on Lyapunov optimization and Simulated Annealing Genetic Algorithm (SAGA) is put forward. As for the dynamic subchannel, the master-and-slave model is adopted and solved by SAGA and Sequential Quadratic Programming (SQP) method. At last, the effectiveness of the proposed schemes is verified through several simulations.

Lyapunov-Based Partial Computation Offloading for Multiple Mobile Devices Enabled by Harvested Energy in MEC

Mobile-edge computing (MEC) has been garnering considerable level of interests by processing computation tasks nearby mobile devices (MDs). With limited computation and communication resources and strict task deadline, balancing the energy consumption and time delay of computational tasks will be highly focused. MDs deployed energy harvesting (EH) modules can always provide service to continuous task requests, and finer-grained offloading schemes of the MEC system will significantly affect the time delay of computation tasks. However, when combined them together, the energy causal constraint and the coupling between offloading ratios and resources allocation will cause new challenges for the computation offloading problem. To address these issues, we investigate the partial computation offloading schemes for multiple MDs enabled by harvested energy in MEC. Specifically, we build models for two computing modes and EH process. Subsequently, we formulate a nonconvex optimization problem by minimizing the energy consumption of all the MDs while satisfying the constraint of time delay. Furthermore, we propose and design a novel algorithm based on the Lyapunov optimization to achieve optimal solution, that is, Lyapunov-optimization-based partial computation offloading for multiuser (LOMUCO). Then, we take the long-term average energy consumption and the discarding ratio of computation tasks as the quantitative metrics and conduct extended simulation experiments to confirm the performance of LOMUCO. Finally, compared to several baseline or state-of-the-art algorithms, including local computing all (LCA), offloading computing all (OCA), randomly partial computation offloading (RPCO), and Lyapunov-optimization-based dynamic computation offloading (LODCO), we can demonstrate the superiority of LOMUCO.

Discontinuous Computation Offloading for Energy-Efficient Mobile Edge Computing

We propose a novel strategy for energy-efficient dynamic computation offloading, in the context of edge-computing-aided beyond 5G networks. The goal is to minimize the energy consumption of the overall system, comprising multiple User Equipment (UE), an access point (AP), and an edge server (ES), under constraints on the end-to-end service delay and the packet error rate performance over the wireless interface. To reduce the energy consumption, we exploit low-power sleep operation modes for the users, the AP and the ES, shifting the edge computing paradigm from an always on to an always available architecture, capable of guaranteeing an on-demand target service quality with the minimum energy consumption. To this aim, we propose an online algorithm for dynamic and optimal orchestration of radio and computational resources called Discontinuous Computation Offloading (DisCO). In such a framework, end-to-end delay constraints translate into constraints on overall queueing delays, including both the communication and the computation phases of the offloading service. DisCO hinges on Lyapunov stochastic optimization, does not require any prior knowledge on the statistics of the offloading traffic or the radio channels, and satisfies the long-term performance constraints imposed by the users. Several numerical results illustrate the advantages of the proposed method.

NOMA-Based Multi-User Mobile Edge Computation Offloading via Cooperative Multi-Agent Deep Reinforcement Learning

Mobile edge computing (MEC) is a promising solution to enable resource-limited mobile devices to offload computation-intensive tasks to nearby edge servers. In this paper, dynamic computation offloading in a non-orthogonal multiple access (NOMA) based multi-user network is investigated by considering stochastic task arrivals and wireless channels. Specifically, a cooperative multi-agent deep reinforcement learning (CMADRL) based framework is proposed to learn decentralized offloading policies to allocate task offloading and local execution powers at each user, which minimizes long-term average network computation cost in terms of power consumption and buffering delay. By leveraging a centralized training and distributed execution strategy, the proposed framework can not only learn efficient decentralized policies, but also relieves users’ computational pressure and works well in coordinating the interference of NOMA-based networks. To reduce the training complexity, it is further improved to train one parameter shared policy network that can be exploited by all users with slightly sacrificed performance. Numerical simulations demonstrate that the proposed CMADRL based framework can learn efficient dynamic offloading policies at each user, which significantly outperforms the conventional independent Q-learning based framework as well as some greedy strategies with lower network computation cost.

Optimal dynamic spectrum allocation-assisted latency minimization for multiuser mobile edge computing

Mobile Edge Computing (MEC) has been envisioned as an efficient solution to provide computation-intensive yet latency-sensitive services for wireless devices. In this paper, we investigate the optimal dynamic spectrum allocation-assisted multiuser computation offloading in MEC for overall latency minimization. Specifically, we first focus on a static multiuser computation offloading scenario and jointly optimize users' offloading decisions, transmission durations, and Edge Servers' (ESs) resource allocations. Owing to the nonconvexity of our joint optimization problem, we identify its layered structure and decompose it into two problems: a subproblem and a top problem. For the subproblem, we propose a bisection search-based algorithm to efficiently find the optimal users' offloading decisions and ESs’ resource allocations under a given transmission duration. Second, we use a linear search-based algorithm for solving the top problem to obtain the optimal transmission duration based on the result of the subproblem. Further, after solving the static scenario, we consider a dynamic scenario of multiuser computation offloading with time-varying channels and workload. To efficiently address this dynamic scenario, we propose a deep reinforcement learning-based online algorithm to determine the near-optimal transmission duration in a real-time manner. Numerical results are provided to validate our proposed algorithms for minimizing the overall latency in both static and dynamic offloading scenarios. We also demonstrate the advantages of our proposed algorithms compared to the conventional multiuser computation offloading schemes.

Dynamic Computation Offloading With Energy Harvesting Devices: A Graph-Based Deep Reinforcement Learning Approach

We study a joint partial offloading and resource allocations (JPORA) problem for the mobile edge computing (MEC) with energy harvesting (EH), where the number of MDs, computation task, and harvested energy of mobile device (MD) are highly dynamic. It is critical to acquire an algorithm that optimally adapts JPORA decisions. Recent studies employ deep deterministic policy gradient (DDPG) agent to tackle the JPORA problem. However, traditional DDPG cannot generalize well to different MEC network scale, due to the deep neural networks in DDPG can only extract latent representation from Euclidean data, with the characteristics of MEC network structural information ignored. To this end, by taking advantage of the graph-based relationship deduction ability form graph convolutional networks (GCN) and the self-evolution ability from the experience training of DDPG, we propose a centralized GCN-DDPG agent to learn making decisions for MDs, including offloading ratio, local computation capacity, and uplink transmission power. Experimental results show that the proposed GCN-DDPG provides significant performance improvement over a number of state-of-the-art DRL agents.

Exploring computation offloading in IoT systems

Internet of Things (IoT) paradigm raises challenges for devising efficient strategies that offload applications to the fog or the cloud layer while ensuring the optimal response time for a service.Traditional computation offloading policies assume the response time is only dominated by the execution time. However, the response time is a function of many factors including contextual parameters and application characteristics that can change over time.For the computation offloading problem, the majority of existing literature presents efficient solutions considering a limited number of parameters (e.g., computation capacity and network bandwidth) neglecting the effect of the application characteristics and dataflow configuration. In this paper, we explore the impact of the computation offloading on total application response time in three-layer IoT systems considering more realistic parameters, e.g., application characteristics, system complexity, communication cost, and dataflow configuration.This paper also highlights the impact of a new application characteristic parameter defined as Output–Input Data Generation (OIDG) ratio and dataflow configuration on the system behavior.In addition, we present a proof-of-concept end-to-end dynamic computation offloading technique, implemented in a real hardware setup, that observes the aforementioned parameters to perform real-time decision-making.

Dynamic Computation Offloading for MIMO Mobile Edge Computing Systems With Energy Harvesting

By providing spatial diversity gain, the incorporation of multiple antennas into mobile edge computing (MEC) systems can improve the transmission performance. Meanwhile, employing energy harvesting (EH) helps enhance the system sustainability. In this paper, we focus on multi-input multi-output (MIMO) MEC systems with EH and studies the computation offloading. The design objective is to minimize the time average of a weighted sum of energy consumption and execution delay, meanwhile stabilizing the battery energy queue. To this end, we formulate the problem as a statistic program and propose a dynamic computation offloading (DCO) algorithm in which the transmitter covariance matrix, CPU-cycle frequencies for local computing, and partial offloading ratio are jointly optimized. Based on Lyapunov optimization, the program is first transformed into a nonconvex per-time slot problem. Then, we solve it by the successive convex approximation (SCA) technique, where a sequence of convex problems are created and solved. Simulation results demonstrate that the proposed algorithm is asymptotically optimal and outperforms several benchmark schemes in terms of both the average system cost and task drop ratio.

Decentralized computation offloading for multi-user mobile edge computing: a deep reinforcement learning approach

Mobile edge computing (MEC) emerges recently as a promising solution to relieve resource-limited mobile devices from computation-intensive tasks, which enables devices to offload workloads to nearby MEC servers and improve the quality of computation experience. In this paper, an MEC enabled multi-user multi-input multi-output (MIMO) system with stochastic wireless channels and task arrivals is considered. In order to minimize long-term average computation cost in terms of power consumption and buffering delay at each user, a deep reinforcement learning (DRL)-based dynamic computation offloading strategy is investigated to build a scalable system with limited feedback. Specifically, a continuous action space-based DRL approach named deep deterministic policy gradient (DDPG) is adopted to learn decentralized computation offloading policies at all users respectively, where local execution and task offloading powers will be adaptively allocated according to each user’s local observation. Numerical results demonstrate that the proposed DDPG-based strategy can help each user learn an efficient dynamic offloading policy and also verify the superiority of its continuous power allocation capability to policies learned by conventional discrete action space-based reinforcement learning approaches like deep Q-network (DQN) as well as some other greedy strategies with reduced computation cost. Besides, power-delay tradeoff for computation offloading is also analyzed for both the DDPG-based and DQN-based strategies.

Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems

Mobile Edge Computing (MEC) is one of the most promising techniques for next-generation wireless communication systems. In this paper, we study the problem of dynamic caching, computation offloading, and resource allocation in cache-assisted multi-user MEC systems with stochastic task arrivals. There are multiple computationally intensive tasks in the system, and each Mobile User (MU) needs to execute a task either locally or remotely in one or more MEC servers by offloading the task data. Popular tasks can be cached in MEC servers to avoid duplicates in offloading. The cached contents can be either obtained through user offloading, fetched from a remote cloud, or fetched from another MEC server. The objective is to minimize the long-term average of a cost function, which is defined as a weighted sum of energy consumption, delay, and cache contents' fetching costs. The weighting coefficients associated with the different metrics in the objective function can be adjusted to balance the tradeoff among them. The optimum design is performed with respect to four decision parameters: whether to cache a given task, whether to offload a given uncached task, how much transmission power should be used during offloading, and how much MEC resources to be allocated for executing a task. We propose to solve the problems by developing a dynamic scheduling policy based on Deep Reinforcement Learning (DRL) with the Deep Deterministic Policy Gradient (DDPG) method. A new decentralized DDPG algorithm is developed to obtain the optimum designs for multi-cell MEC systems by leveraging on the cooperations among neighboring MEC servers. Simulation results demonstrate that the proposed algorithm outperforms other existing strategies, such as Deep Q-Network (DQN).

Dynamic Computation Offloading With Energy Harvesting Devices: A Hybrid-Decision-Based Deep Reinforcement Learning Approach

Mobile-edge computing (MEC) with energy harvesting (EH) is becoming an emerging paradigm to improve the computation experience for the Internet-of-Things (IoT) devices. For a multidevice multiserver MEC system, the frequently varied harvested energy, along with changeable computation task loads and time-varying computation capacities of servers, increase the system’s dynamic. Therefore, each device should learn to make coordinated actions, such as the offloading ratio, local computation capacity, and server selection, to achieve a satisfactory computation quality. Thus, the MEC system with EH devices is highly dynamic and face two challenges: 1) continuous–discrete hybrid action spaces and 2) coordination among devices. To deal with such problem, we propose two deep reinforcement learning (DRL)-based algorithms: 1) hybrid-decision-based actor–critic learning (Hybrid-AC) and 2) multidevice hybrid-AC (MD-Hybrid-AC) for dynamic computation offloading. Hybrid-AC solves the hybrid action space with an improvement of actor–critic architecture, where the ${{actor}}$ outputs continuous actions (offloading ratio and local computation capacity) corresponding to every server, and the ${{critic}}$ evaluates the continuous actions and outputs the discrete action of server selection. MD-Hybrid-AC adopts the framework of centralized training with decentralized execution. It learns coordinated decisions by constructing a centralized ${{critic}}$ to output server selections, which considers the continuous action policies of all devices. Simulation results show that the proposed algorithms achieve a good balance between consumed time and energy, and have a significant performance improvement compared with baseline offloading policies.

Dynamic Computation Offloading With Imperfect State Information in Energy Harvesting Small Cell Networks: A Partially Observable Stochastic Game

The combination of energy harvesting small cell networks (EH-SCNs) and mobile edge computing (MEC) has been considered as an effective means to improve the performance of mobile networks and provide users with a higher quality of service (QoS). In this letter, we investigate the decentralized computation offloading problem in heterogeneous EH-SCNs with MEC, where heterogeneous small cell base stations (SBSs) are rational individuals with interests to maximize their own benefits while considering their QoS requirements. Different from existing works, we address the challenge that heterogeneous SBSs may unwilling to expose their own information about the system state and offloading decisions. We formulate the problem as a partially observable stochastic game (POSG), in which SBSs can make optimal offloading decisions with imperfect state information. We analyze the local equilibrium, and propose a stochastic offloading algorithm to obtain the approximate optimal solution. Numerical results validate the effectiveness of the proposed scheme.

Dynamic Computation Offloading in Multi-Access Edge Computing via Ultra-Reliable and Low-Latency Communications

The goal of this work is to propose an energy-efficient algorithm for dynamic computation offloading, in a multi-access edge computing scenario, where multiple mobile users compete for a common pool of radio and computational resources. We focus on delay-critical applications, incorporating an upper bound on the probability that the overall time required to send the data and process them exceeds a prescribed value. In a dynamic setting, the above constraint translates into preventing the sum of the communication and computation queues’ lengths from exceeding a given value. Ultra-reliable low latency communications (URLLC) are also taken into account using finite blocklengths and reliability constraints. The proposed algorithm, based on stochastic optimization, strikes an optimal balance between the service delay and the energy spent at the mobile device, while guaranteeing a target out-of-service probability. Starting from a long-term average optimization problem, our algorithm is based on the solution of a convex problem in each time slot, which is provided with a very fast iterative strategy. Finally, we extend the approach to mobile devices having energy harvesting capabilities, typical of Internet of Things scenarios, thus devising an energy efficient dynamic offloading strategy that stabilizes the battery level of each device around a prescribed operating level.

Energy-Latency Tradeoff for Dynamic Computation Offloading in Vehicular Fog Computing

Vehicular Fog Computing (VFC) provides solutions to relieves overload cloudlet nodes, reduces service latency during peak times, and saves energy for battery-powered cloudlet nodes by offloading user tasks to a vehicle (vehicular node) by exploiting the under-utilized computation resources of nearby vehicular node. However, the wide deployment of VFC still confronts several critical challenges: lack of energy-latency tradeoff and efficient resource allocation mechanisms. In this paper, we address the challenges and provide an Energy-efficient dynamic Computation Offloading and resources allocation Scheme (ECOS) to minimize energy consumption and service latency. We first formulate the ECOS problem as a joint energy and latency cost minimization problem while satisfying vehicular node mobility and end-to-end latency deadline constraints. We then propose an ECOS scheme with three phases. In the first phase, we propose an overload cloudlet node detection policy based on resource utilization. In the second phase, we propose a computational offloading selection policy to select a task from an overloaded cloudlet node for offloading, which minimizes offloading cost and the risk of overload. Next, we propose a heuristic approach to solve the resource allocation problem between the vehicular node and selected user tasks for energy-latency tradeoff. Extensive simulations have been conducted under realistic highway and synthetic scenarios to examine the ECOS scheme's performance. In comparison, our proposed scheme outperforms the existing schemes in terms of energy-saving, service latency, and joint energy-latency cost.

Dynamic Social-Aware Computation Offloading for Low-Latency Communications in IoT

Internet of Things (IoT) as a prospective platform to develop mobile applications, is facing with significant challenges posed by the tension between resource-constrained mobile smart devices and low-latency demanding applications. Recently, mobile edge computing (MEC) is emerging as a cornerstone technology to address such challenges in IoT. In this paper, by leveraging social ties in human social networks, we investigate the optimal dynamic computation offloading mode selection to jointly minimize the total tasks' execution latency and the mobile smart devices' energy consumption in MEC-aided low-latency IoT. Different from the previous studies, which mostly focus on how to exploit social tie structure among mobile smart device users to construct the permutation of all the feasible modes, we consider dynamic computation offloading mode selection with social awareness-aided network resource assignment, involving both the computing resources and transmit power from heterogeneous mobile smart devices. On the one hand, we formulate the dynamic computation offloading mode selection into the infinite-horizon time-average renewal-reward problems subject to time average latency constraints on a collection of penalty processes. On the other hand, an efficient solution is also developed, which elaborates on a Lyapunov optimization-based approach, i.e., drift-plus-penalty (DPP) algorithm. Numerical simulations are provided to validate the theoretical analysis and assess the performance of the proposed dynamic social-aware computation offloading mode selection method considering different configurations of the IoT network parameters.

Socially Aware Dynamic Computation Offloading Scheme for Fog Computing System With Energy Harvesting Devices

Fog computing is considered as a promising technology to meet the ever-increasing computation requests from a wide variety of mobile applications. By offloading the computation-intensive requests to the fog node or the central cloud, the performance of the applications, such as energy consumption and delay, are able to be significantly enhanced. Meanwhile, utilizing the recent advances of social network and energy harvesting (EH) techniques, the system performance could be further improved. In this paper, we take the social relationships of the EH mobile devices (MDs) into the design of computational offloading scheme in fog computing. With the objective to minimize the social group execution cost, we advocate game theoretic approach and propose a dynamic computation offloading scheme designing the offloading process in fog computing system with EH MDs. Different queue models are applied to model the energy cost and delay performance. It can be seen that the proposed problem can be formulated as a generalized Nash equilibrium problem (GNEP) and we can use exponential penalty function method to transform the original GNEP into a classical Nash equilibrium problem and address it with semi-smooth Newton method with Armijo line search. The simulation results demonstrate the effectiveness of the proposed scheme.