Online Resource Allocation Algorithm Research Articles

IRATS: A DRL-based intelligent priority and deadline-aware online resource allocation and task scheduling algorithm in a vehicular fog network

Cloud computing platforms support the Internet of Vehicles, but the main bottlenecks are high latency and massive data transmission in cloud-based processing. Vehicular fog computing has emerged as a promising paradigm to accommodate the increasing computational needs of vehicles. It provides low latency network services that are most important for latency-sensitive tasks. The dynamic nature of VFC, having vehicles with heterogeneous computing resources, vehicle mobility, and diverse tasks with different priorities are the main challenges in vehicular fog networks. In VFC, vehicles can share their idle compute resources with other task-generating vehicles. So, scheduling the tasks on the idle resources of resource-limited vehicles is very important. Existing solutions use a heuristic approach to solve this issue but lack generalizability and adaptability. In this paper, we describe a PPO-based intelligent, priority and deadline-aware online and distributed resource allocation and task scheduling algorithm, called IRATS, in vehicular fog networks. IRATS formulates the resource allocation problem as a Markov decision process to minimize the waiting time and delay of tasks. For vehicles sharing their idle resources, we design a task scheduler for the orderly execution of received tasks according to their priorities using multi-level queues. We conducted extensive simulations using SUMO, OMNeT++, Veins, and veins-gym to validate the effectiveness of the presented algorithm. The simulation results confirm that the proposed algorithm improves the percentage of in-time completed tasks and decreases the packet loss, waiting time, and end-to-end delay as compared to random, A2C, and DQN algorithms considering the task priority and link duration of vehicles.

Cellular-Connected Multi-UAV MEC Networks: An Online Stochastic Optimization Approach

In this paper, we consider a mobile edge computing (MEC) network where multiple cellular-connected unmanned aerial vehicles (UAVs) can offload their computation tasks to multiple ground base stations (GBSs). In practice, the UAVs are generally unable to master stochastic information of task arrival and channel changes in advance, which may cause a severe issue in terms of energy consumption. Therefore, we formulate a stochastic optimization problem with the goal of minimizing the average weighted sum energy consumption, by jointly optimizing UAV-GBS associations, communication and computation resource allocation, and three-dimensional (3D) UAV trajectories, during which a velocity-triggered penalty term (VTPT) is designed to suppress a large amount of the energy consumption of the UAVs. To handle the stochastic problem, we propose an online resource allocation and trajectory optimization algorithm with outer and inner structures. The outer structure transforms the original problem to a deterministic one by applying the Lyapunov-based optimization framework. The inner structure solves the obtained deterministic problem via the Lagrange duality method and the successive convex approximation technique, based on the block coordinate descent framework. Numerical results demonstrate that: 1) VTPT dramatically decreases the UAVs’ energy consumption, and 2) the proposed algorithm not only reduces the energy consumption but also ensures the computation queue stability compared with other benchmark schemes.

Energy Efficiency and Delay Tradeoff in an MEC-Enabled Mobile IoT Network

Mobile-edge computing (MEC) has recently emerged as a promising technology in the 5G era. It is deemed an effective paradigm to support computation intensive and delay-critical applications even at energy-constrained and computation-limited Internet of Things (IoT) devices. To effectively exploit the performance benefits enabled by MEC, it is imperative to jointly allocate radio and computational resources by considering nonstationary computation demands, user mobility, and wireless fading channels. This article aims to study the tradeoff between energy efficiency (EE) and service delay for multiuser multiserver MEC-enabled IoT systems when provisioning offloading services in a user mobility scenario. Particularly, we formulate a stochastic optimization problem with the objective of minimizing the long-term average network EE with the constraints of the task queue stability, peak transmit power, maximum CPU-cycle frequency, and maximum user number. To tackle the problem, we propose an online offloading and resource allocation algorithm by transforming the original problem into several individual subproblems in each time slot based on the Lyapunov optimization theory, which are then solved by convex decomposition and submodular methods. Theoretical analysis proves that the proposed algorithm can achieve a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$[O(1/V), O(V)]$ </tex-math></inline-formula> tradeoff between EE and service delay. Simulation results verify the theoretical analysis and demonstrate our proposed algorithm can offer much better EE-delay performance in task offloading challenges, compared to several baselines.

Federated Deep Reinforcement Learning for Online Task Offloading and Resource Allocation in WPC-MEC Networks

Mobile edge computing (MEC) is considered a more effective technological solution for developing the Internet of Things (IoT) by providing cloud-like capabilities for mobile users. This article combines wireless powered communication (WPC) technology with an MEC network, where a base station (BS) can transfer wireless energy to edge users (EUs) and execute computation-intensive tasks through task offloading. Traditional numerical optimization methods are time-consuming approaches for solving this problem in time-varying wireless channels, and centralized deep reinforcement learning (DRL) is not stable in large-scale dynamic IoT networks. Therefore, we propose a federated DRL-based online task offloading and resource allocation (FDOR) algorithm. In this algorithm, DRL is executed in EUs, and federated learning (FL) uses the distributed architecture of MEC to aggregate and update the parameters. To further solve the problem of the non-IID data of mobile EUs, we devise an adaptive method that automatically adjusts the FDOR algorithm’s learning rate. Simulation results demonstrate that the proposed FDOR algorithm is superior to the traditional numerical optimization method and the existing DRL algorithm in four aspects: convergence speed, execution delay, overall calculation rate and stability in large-scale and dynamic IoT.

Mobility-Aware Offloading and Resource Allocation in a MEC-Enabled IoT Network With Energy Harvesting

Mobile-edge computing (MEC)-enabled Internet of Things (IoT) networks have been deemed a promising paradigm to support massive energy-constrained and computation-limited IoT devices. Energy harvesting (EH) further enhances the operating capabilities of IoT devices that normally only possess very limited energy support. Nevertheless, many studies show that IoT devices using EH can experience uncertainty and unpredictability, which can complicate the EH-based IoT network design. Furthermore, with many new services in 5G and the forthcoming 6G eras, such as autonomous driving and vehicular communications, mobility consideration in IoT networks becomes more and more important. In this article, we study the computing offloading and resource allocation problems in an IoT network that supports both mobility and EH. The long-term average sum service cost of all the mobile IoT devices (MIDs) is minimized by optimizing the harvested energy, task-partition factors, the central process unit frequencies, the transmit power, and the association vector of MIDs. An online mobility-aware offloading and resource allocation (OMORA) algorithm is proposed based on the Lyapunov optimization and semidefinite programming (SDP). This online algorithm optimizes the offloading scheme without the need to have prior knowledge of the user mobility, EH model, and channel condition. Theoretical analysis shows that the proposed OMORA algorithm can achieve asymptotic optimality. Simulation results demonstrate that the proposed algorithm can effectively balance the system service cost and energy queue length, and outperform other offloading benchmark algorithms on the system service cost and packet losses.

Evaluating the impact of delay constraints in network services for intelligent network slicing based on SKM model

Solving the problem of network resource allocation with delay constraint is a significant challenge for realizing future Internet and 5G networks services such as advanced mobile broadband services and Internet of things (IoT), especially under the network slicing scenario. The impact of delay constraints may lead to rejection of demands, resulting in low resource utilization of network resources. This is especially severe when dynamic traffic is considered. Therefore, intelligent resource allocation algorithms are required to use the network resources in delay constrained scenario efficiently. Moreover, these algorithms should guarantee quality of service (QoS) between different priority slices during congestion case. Therefore, in this paper, we analyze the impact of delay constraint on the performance of an online resource allocation algorithm based on an intelligent efficient squatting and kicking model (SKM), proved in other works to be the most effective up to the present time yet. SKM incorporates kicking and squatting of resources as innovative techniques enabling it to achieve 100% resource utilization and acceptance ratio for higher priority slices in scenarios where the other state of art algorithms not able to reach by far in some scenarios. Simulation results showed that incorporating delay constraints has a significant impact on the performance, resulting in up to 10% and 4% reduction in terms of average resource utilization and acceptance ratios respectively.

Stochastic joint rate control and resource allocation for wireless video surveillance

Wireless video surveillance has become increasingly prevalent due to the rapid development of cellular communication technology and the low deployment cost of cellular surveillance cameras. However, it is still a challenging task to deliver high-quality low-latency surveillance videos in resource-limited cellular networks due to the fluctuating characteristics of wireless channels. In this paper, we consider a wireless video surveillance system consisting of one base station and multiple cellular surveillance cameras, which upload their videos to the base station. To optimize the long-term system performance in terms of surveillance video quality and latency, we formulate a stochastic optimization problem that jointly considers video rate control and uplink resource allocation. The formulation captures the tradeoff between video quality and delivery latency while considering the different time scales of video rate control and channel resource allocation. With the application of Lyapunov optimization, we transform the problem into a sequence of per-slot deterministic problems, and propose an online joint rate control and resource allocation algorithm to solve these per-slot problems. Specifically, the proposed algorithm decomposes each of the per-slot problems into two independent subproblems, namely, the rate control subproblem at large time scale and the uplink resource allocation subproblem at small time scale. We derive the closed-form solution for the rate control subproblem, and design a fast-convergent iterative algorithm to solve the uplink resource allocation subproblem optimally. We prove the asymptotical optimality of our proposed algorithm. Extensive simulation results validate our theoretical analysis and demonstrate that the proposed algorithm can well balance the surveillance video quality and delivery latency.

Online Resource Allocation With Machine Variability: A Bandit Perspective

Approximation jobs that allow partial execution of their many tasks to achieve valuable results have played an important role in today's large-scale data analytics. This fact can be utilized to maximize the system utility of a big data computing cluster by choosing proper tasks in scheduling for each approximation job. A fundamental challenge herein, however, is that the machine service capacity may fluctuate substantially during a job's lifetime, which makes it difficult to assign valuable tasks to well-performing machines. In addition, the cluster scheduler needs to make online scheduling decisions without knowing future job arrivals according to machine availabilities. In this paper, we tackle this online resource allocation problem for approximation jobs in parallel computing clusters. In particular, we model a cluster with heterogeneous machines as a multi-armed bandit where each machine is treated as an arm. By making estimations on machine service rates while balancing the exploration-exploitation trade-off, we design an efficient online resource allocation algorithm from a bandit perspective. The proposed algorithm extends existing online convex optimization techniques and yields a sublinear regret bound. Moreover, we also examine the performance of the proposed algorithm via extensive trace-driven simulations and demonstrate that it outperforms the baselines substantially.

Online Offloading Scheduling and Resource Allocation Algorithms for Vehicular Edge Computing System

To accommodate the exponentially increasing computation demands of vehicle-based applications, vehicular edge computing (VEC) system was introduced. This paper considers a three-layer VEC architecture and proposes an online offloading scheduling and resource allocation (OOSRA) algorithm to improve the system performance. Specifically, this study designs a game-theoretic online algorithm to solve the problem of computation task offloading scheduling, and employs an online bin-packing algorithm to compute the resource allocation modified from the First Fit algorithm, which can be adapted to various traffic flow and service attributes. Extensive simulations are conducted, and a numerical analysis of simulation results verifies the effectiveness of the OOSRA-VEC system. The algorithms proposed in this paper are online, adaptive, and distributed, which can provide useful references for future development in VEC system protocols.

Energy Efficiency and Delay Tradeoff for Wireless Powered Mobile-Edge Computing Systems With Multi-Access Schemes

The integration of Mobile-edge Computing (MEC) and Wireless Energy Transfer (WET) has been recognized as a promising technique to enhance computation capability and to prolong battery lifetime of resource-constrained wireless devices in the Internet of Things (IoT) era. However, it is challenging to jointly schedule energy, radio, and computational resources for coordinating heterogeneous performance requirements in wireless powered MEC systems. To fill this gap, this paper investigates the fundamental tradeoff between Energy Efficiency (EE) and delay in a multi-user wireless powered MEC system. Considering the random channel conditions and task arrivals, we formulate a stochastic optimization problem to study the EE-delay tradeoff, which optimizes network EE subject to network stability, maximum central processing unit frequency, peak transmission power, available communication resource, and energy causality constraints. Further, we propose the online computation offloading and resource allocation algorithm by transforming the original problem into a series of deterministic optimization problems in each time block based on Lyapunov optimization theory. In addition, theoretical analysis shows that the algorithm achieves the EE-delay tradeoff as [ ${O}(1/{V}), {O}({V})$ ] and introduces a control parameter ${V}$ to balance the EE-delay performance. Numerical results verify the theoretical analysis and reveal the impact of various parameters to the system performance.

An online resource allocation algorithm to minimize system interference for inband underlay D2D communications

SummaryThis paper addresses the research question of total system interference minimization while maintaining a target system sum rate gain in an inband underlay device‐to‐device (D2D) communication. To the best of our knowledge, most of the state of the art research works exploit offline resource allocation algorithms to address the research problem. However, in Long‐Term Evolution (LTE) and beyond systems (4G, 5G, or 5G+), offline resource allocation algorithms do not comply with the fast scheduling requirements because of the high data rate demand. In this paper, we propose a bi‐phase online resource allocation algorithm to minimize the total system interference for inband underlay D2D communication. Our proposed algorithm assumes D2D pairs as a set of variable elements whereas takes the cellular user equipment (UEs) as a set of constant elements. The novelty of our proposed online resource allocation algorithm is that it incurs a minimum number of changes in radio resource assignment between two successive allocations among the cellular UEs and the D2D pairs. Graphical representation of the simulation results suggests that our proposed algorithm outperforms the existing offline algorithm considering number of changes in successive allocation for a certain percentage of sum rate gain maintaining the total system interference and total system sum rate very similar.

Optimal 3D-Trajectory Design and Resource Allocation for Solar-Powered UAV Communication Systems

In this paper, we investigate the resource allocation algorithm design for multicarrier solar-powered unmanned aerial vehicle (UAV) communication systems. In particular, the UAV is powered by the solar energy enabling sustainable communication services to multiple ground users. We study the joint design of the 3D aerial trajectory and the wireless resource allocation for maximization of the system sum throughput over a given time period. As a performance benchmark, we first consider an off-line resource allocation design assuming non-causal knowledge of the channel gains. The algorithm design is formulated as a mixed-integer non-convex optimization problem taking into account the aerodynamic power consumption, solar energy harvesting, a finite energy storage capacity, and the quality-of-service requirements of the users. Despite the non-convexity of the optimization problem, we solve it optimally by applying monotonic optimization to obtain the optimal 3D-trajectory and the optimal power and subcarrier allocation policy. Subsequently, we focus on the online algorithm design that only requires real-time and statistical knowledge of the channel gains. The optimal online resource allocation algorithm is motivated by the off-line scheme and entails a high computational complexity. Hence, we also propose a low-complexity iterative suboptimal online scheme based on the successive convex approximation. Our simulation results reveal that both the proposed online schemes closely approach the performance of the benchmark off-line scheme and substantially outperform two baseline schemes. Furthermore, our results unveil the tradeoff between solar energy harvesting and power-efficient communication. In particular, the solar-powered UAV first climbs up to a high altitude to harvest a sufficient amount of solar energy and then descends again to a lower altitude to reduce the path loss of the communication links to the users it serves.

Self-adaptive resource allocation for energy-aware virtual machine placement in dynamic computing cloud

Cloud computing is a type of Internet-based computing that provides shared computing resources and data on demand. To support the massive cloud services, a data center is built with thousands of servers and switches connected by a communication network called a data center network (DCN). Along with the development of virtualization, the data center manager conducts virtual machine (VM) resizing and live migrations to adjust the allocation of VMs and improve the server utilization. However, compared to VM resizing, live migrations can negatively impact the performance of the system depending on the amount of transferred data and the path length. In this paper, we propose an online resource allocation algorithm using VM consolidation to achieve energy efficiency and reduce service-level agreement (SLA) violations of data centers while considering the power usage of servers, the number of migrations, and the path length of migrations in DCNs. Compared to the state-of-the-art mechanism, the proposed algorithm successfully reduces the power consumption, the number of migrations, and the path length of migrations for a dynamic cloud service.

Resource allocation algorithm with worst case delay guarantees in energy harvesting body area networks

In Body Area Networks, for energy harvesting-powered, managing the renewable energy to provide delay-sensitive services is of significant importance. In this paper, we propose an online algorithm to allocate resources, i.e., energy and channel, to maximize the user utility while guaranteeing the worst-case delay. To this end, we first formulate a user utility optimization problem, characterizing the stochastic nature of energy harvesting and energy consumption. Furthermore, a priori knowledge of any this processes is not required. Using Lyapunov optimization techniques, we decompose the problem into four sub-problems, i.e., battery management, collecting rate control, transmission power allocation and dropping rate control. Low-complexity online resource allocation algorithm is proposed to address these problems for user utility maximization, while we further analyze the performance of the online algorithm, in terms of the upper bounds of queues, required battery capacity, and the optimality of the proposed algorithm. Simulation results verify our analysis and the efficacy of the proposed algorithm.

An Optimal Online Resource Allocation Algorithm for Energy Harvesting Body Area Networks

In Body Area Networks (BANs), how to achieve energy management to extend the lifetime of the body area networks system is one of the most critical problems. In this paper, we design a body area network system powered by renewable energy, in which the sensors carried by patient with energy harvesting module can transmit data to a personal device. We do not require any a priori knowledge of the stochastic nature of energy harvesting and energy consumption. We formulate a user utility optimization problem. We use Lyapunov Optimization techniques to decompose the problem into three sub-problems, i.e., battery management, collecting rate control and transmission power allocation. We propose an online resource allocation algorithm to achieve two major goals: (1) balancing sensors’ energy harvesting and energy consumption while stabilizing the BANs system; and (2) maximizing the user utility. Performance analysis addresses required battery capacity, bounded data queue length and optimality of the proposed algorithm. Simulation results verify the optimization of algorithm.

Utility-Optimal Resource Management and Allocation Algorithm for Energy Harvesting Cognitive Radio Sensor Networks

In this paper, we study resource management and allocation for energy harvesting cognitive radio sensor networks (EHCRSNs). In these networks, energy harvesting supplies the network with a continual source of energy to facilitate the self-sustainability of the power-limited sensors. Furthermore, cognitive radio enables access to the underutilized licensed spectrum to mitigate the spectrum-scarcity problem in the unlicensed band. We develop an aggregate network utility optimization framework for the design of an online energy management, spectrum management, and resource allocation algorithm based on Lyapunov optimization. The framework captures three stochastic processes: energy harvesting dynamics, inaccuracy of channel occupancy information, and channel fading. However, a priori knowledge of any of these processes statistics is not required. Based on the framework, we propose an online algorithm to achieve two major goals: first, balancing sensors’ energy consumption and energy harvesting while stabilizing their data and energy queues; second, optimizing the utilization of the licensed spectrum while maintaining a tolerable collision rate between the licensed subscriber and unlicensed sensors. The performance analysis shows that the proposed algorithm achieves a close-to-optimal aggregate network utility while guaranteeing bounded data and energy queue occupancy. The extensive simulations are conducted to verify the effectiveness of the proposed algorithm and the impact of various network parameters on its performance.

Suboptimal Online Resource Allocation in Hybrid Energy Supplied OFDMA Cellular Networks

In this letter, by jointly considering the harvested energy control, power allocation, and sub-carrier assignment, we address the grid energy conservation problem in QoS guaranteed orthogonal frequency division multiple Access (OFDMA) cellular networks, in which the base station is supplied by both grid and renewable energy. By using the stochastic optimization theory and the Lagrange dual method, we propose a suboptimal online resource allocation (SORA) algorithm, which is of low computational complexity. In particular, the SORA algorithm is asymptotically optimal as the battery capacity is to be infinite. Simulation results demonstrate the advantages of the SORA algorithm.

Power-Delay Tradeoff With Predictive Scheduling in Integrated Cellular and Wi-Fi Networks

The explosive growth of global mobile traffic has lead to a rapid growth in the energy consumption in communication networks. In this paper, we focus on the energy-aware design of the network selection, subchannel, and power allocation in cellular and Wi-Fi networks, while taking into account the traffic delay of mobile users. The problem is particularly challenging due to the two-timescale operations for the network selection (large timescale) and subchannel and power allocation (small timescale). Based on the two-timescale Lyapunov optimization technique, we first design an online Energy-Aware Network Selection and Resource Allocation (ENSRA) algorithm. The ENSRA algorithm yields a power consumption within O(1/V) bound of the optimal value, and guarantees an O(V) traffic delay for any positive control parameter V. Motivated by the recent advancement in the accurate estimation and prediction of user mobility, channel conditions, and traffic demands, we further develop a novel predictive Lyapunov optimization technique to utilize the predictive information, and propose a Predictive Energy-Aware Network Selection and Resource Allocation (P-ENSRA) algorithm. We characterize the performance bounds of P-ENSRA in terms of the power-delay tradeoff theoretically. To reduce the computational complexity, we finally propose a Greedy Predictive Energy-Aware Network Selection and Resource Allocation (GP-ENSRA) algorithm, where the operator solves the problem in P-ENSRA approximately and iteratively. Numerical results show that GP-ENSRA significantly improves the power-delay performance over ENSRA in the large delay regime. For a wide range of system parameters, GP-ENSRA reduces the traffic delay over ENSRA by 20~30% under the same power consumption.

Online Resource Allocation for Energy Harvesting Downlink Multiuser Systems: Precoding With Modulation, Coding Rate, and Subchannel Selection

This paper proposes an online resource allocation algorithm for weighted sum rate maximization in energy harvesting downlink multiuser multiple-input-multiple-output (MIMO) systems, where the base station transmitter is powered by both a regular energy source and an energy buffer that is connected to an energy harvester. Taking into account the discrete nature of the modulation and coding rates (MCRs) used in practice, we formulate a stochastic dynamic programming (SDP) problem to jointly design the MIMO precoders, select the MCRs, assign the subchannels, and optimize the energy consumption over multiple time slots with causal and statistical energy arrival information and statistical channel state information. Solving this high-dimensional SDP entails several difficulties: the SDP has a nonconcave objective function, the optimization variables are of mixed binary and continuous types, and the number of optimization variables is on the order of thousands. We propose a new method to solve this NP-hard SDP by decomposing the high-dimensional SDP into an equivalent three-layer optimization problem and develop efficient algorithms to solve each layer separately. The decomposition reduces the computational burden and breaks the curse of dimensionality successfully. We analyze the complexity of the proposed algorithm and demonstrate the performance gains based on numerical examples.

Energy-Efficient Resource Allocation in OFDMA Systems with Hybrid Energy Harvesting Base Station

We study resource allocation algorithm design for energy-efficient communication in an orthogonal frequency division multiple access (OFDMA) downlink network with hybrid energy harvesting base station (BS). Specifically, an energy harvester and a constant energy source driven by a non-renewable resource are used for supplying the energy required for system operation. We first consider a deterministic offline system setting. In particular, assuming availability of non-causal knowledge about energy arrivals and channel gains, an offline resource allocation problem is formulated as a non-convex optimization problem over a finite horizon taking into account the circuit energy consumption, a finite energy storage capacity, and a minimum required data rate. We transform this non-convex optimization problem into a convex optimization problem by applying time-sharing and exploiting the properties of non-linear fractional programming which results in an efficient asymptotically optimal offline iterative resource allocation algorithm for a sufficiently large number of subcarriers. In each iteration, the transformed problem is solved by using Lagrange dual decomposition. The obtained resource allocation policy maximizes the weighted energy efficiency of data transmission (weighted bit/Joule delivered to the receiver). Subsequently, we focus on online algorithm design. A conventional stochastic dynamic programming approach is employed to obtain the optimal online resource allocation algorithm which entails a prohibitively high complexity. To strike a balance between system performance and computational complexity, we propose a low complexity suboptimal online iterative algorithm which is motivated by the offline algorithm. Simulation results illustrate that the proposed suboptimal online iterative resource allocation algorithm does not only converge in a small number of iterations, but also achieves a close-to-optimal system energy efficiency by utilizing only causal channel state and energy arrival information.