Long-term Optimization Problem Research Articles

Multi-agent DRL for edge computing: A real-time proportional compute offloading

In the Industrial Internet of Things, devices with limited computing power and energy storage often rely on offloading tasks to edge servers for processing. However, existing methods are plagued by the high cost of device communication and unstable training processes. Consequently, Deep reinforcement learning (DRL) has emerged as a promising solution to tackle the computation offloading problem. In this paper, we propose a framework called multi-agent twin delayed shared deep deterministic policy gradient algorithm (MASTD3) based on DRL. Firstly, we formulate the task offloading conundrum as a long-term optimization problem, which aids in mitigating the challenge of deciding between local or remote task execution by a device, leading to more effective task offloading management. Secondly, we enhance MASTD3 by introducing a priority experience replay buffer mechanism and a model sample replay buffer mechanism, thus improving sample utilization and overcoming the cold-start problem associated with long-term optimization. Moreover, we refine the actor-critic structure, enabling all agents to share the same critic network. This modification accelerates convergence speed during the training process and reduces computational costs during runtime. Finally, experimental results demonstrate that MASTD3 effectively addresses the proportional offloading problem, which is optimized by 44.32%, 29.26%, and 17.47% compared to DDPQN, MADDPG, and FLoadNet.

6G Goal-Oriented Communications: How to Coexist with Legacy Systems?

6G will connect heterogeneous intelligent agents to make them natively operate complex cooperative tasks. When connecting intelligence, two main research questions arise to identify how artificial intelligence and machine learning models behave depending on (i) their input data quality, affected by errors induced by interference and additive noise during wireless communication; (ii) their contextual effectiveness and resilience to interpret and exploit the meaning behind the data. Both questions are within the realm of semantic and goal-oriented communications. With this paper, we investigate how to effectively share communication spectrum resources between a legacy communication system (i.e., data-oriented) and a new goal-oriented edge intelligence one. Specifically, we address the scenario of an enhanced Mobile Broadband (eMBB) service, i.e., a user uploading a video stream to a radio access point, interfering with an edge inference system, in which a user uploads images to a Mobile Edge Host that runs a classification task. Our objective is to achieve, through cooperation, the highest eMBB service data rate, subject to a targeted goal effectiveness of the edge inference service, namely the probability of confident inference on time. We first formalize a general definition of a goal in the context of wireless communications. This includes the goal effectiveness, (i.e., the goal achievability rate, or the probability of achieving the goal), as well as goal cost (i.e., the network resource consumption needed to achieve the goal with target effectiveness). We argue and show, through numerical evaluations, that communication reliability and goal effectiveness are not straightforwardly linked. Then, after a performance evaluation aiming to clarify the difference between communication performance and goal effectiveness, a long-term optimization problem is formulated and solved via Lyapunov stochastic network optimization tools to guarantee the desired target performance. Finally, our numerical results assess the advantages of the proposed optimization and the superiority of the goal-oriented strategy against baseline 5G-compliant legacy approaches, under both stationary and non-stationary communication (and computation) environments.

Online User and Power Allocation in Dynamic NOMA-Based Mobile Edge Computing

This study tackles the online user allocation problem in mobile edge computing (MEC) systems powered by non-orthogonal multiple access. App vendors need to determine a proper wireless channel in a base station/edge server and sufficient transmit power for every user. We consider a stochastic MEC system where users arrive and depart over time. When an edge server runs out of computing resources, some users will have to wait until the resources become available again, which incurs an allocation delay cost. This cost is often not investigated in many studies, which also do not consider a multi-cell, multi-channel system as we do in this work, due to its complexity. We aim to minimize the allocation delay and transmit power costs, increasing the system’s energy efficiency. To achieve this objective while guaranteeing users’ data rate requirements over time, we adopt the Lyapunov framework to convert this long-term optimization problem into a series of subproblems to be solved in every time slot. To solve the aforementioned subproblems efficiently, we present a distributed game theory-based approach. The proposed algorithm is theoretically evaluated and experimentally demonstrated to outperform several baseline and state-of-the-art methods, highlighting the significance of systematic consideration for both computation and communication aspects of this problem.

Low-Latency Federated Learning via Dynamic Model Partitioning for Healthcare IoT.

Federated learning (FL) is receiving much attention in the Healthcare Internet of Things (H-IoT) to support various instantaneous E-health services. Today, the deployment of FL suffers from several challenges, such as high training latency and data privacy leakage risks, especially for resource-constrained medical devices. In this article, we develop a three-layer FL architecture to decrease training latency by introducing split learning into FL. We formulate a long-term optimization problem to minimize the local model training latency while preserving the privacy of the original medical data in H-IoT. Specially, a Privacy-ware Model Partitioning Algorithm (PMPA) is proposed to solve the formulated problem based on the Lyapunov optimization theory. In PMPA, the local model is partitioned properly between a resource-constrained medical end device and an edge server, which meets privacy requirements and energy consumption constraints. The proposed PMPA is separated into two phases. In the first phase, a partition point set is obtained using Kullback-Leibler (KL) divergence to meet the privacy requirement. In the second phase, we employ the model partitioning function, derived through Lyapunov optimization, to select the partition point from the partition point set that that satisfies the energy consumption constraints. Simulation results show that compared with traditional FL, the proposed algorithm can significantly reduce the local training latency. Moreover, the proposed algorithm improves the efficiency of medical image classification while ensuring medical data security.

Joint Client Selection and CPU Frequency Control in Wireless Federated Learning Networks with Power Constraints.

Federated learning (FL) represents a distributed machine learning approach that eliminates the necessity of transmitting privacy-sensitive local training samples. However, within wireless FL networks, resource heterogeneity introduces straggler clients, thereby decelerating the learning process. Additionally, the learning process is further slowed due to the non-independent and identically distributed (non-IID) nature of local training samples. Coupled with resource constraints during the learning process, there arises an imperative need for optimizing client selection and resource allocation strategies to mitigate these challenges. While numerous studies have made strides in this regard, few have considered the joint optimization of client selection and computational power (i.e., CPU frequency) for both clients and the edge server during each global iteration. In this paper, we initially define a cost function encompassing learning latency and non-IID characteristics. Subsequently, we pose a joint client selection and CPU frequency control problem that minimizes the time-averaged cost function subject to long-term power constraints. By utilizing Lyapunov optimization theory, the long-term optimization problem is transformed into a sequence of short-term problems. Finally, an algorithm is proposed to determine the optimal client selection decision and corresponding optimal CPU frequency for both the selected clients and the server. Theoretical analysis provides performance guarantees and our simulation results substantiate that our proposed algorithm outperforms comparative algorithms in terms of test accuracy while maintaining low power consumption.

Joint Device Selection and Bandwidth Allocation for Cost-Efficient Federated Learning in Industrial Internet of Things

Along with the deployment of Industrial Internet of Things (IIoT), massive amounts of industrial data have been generated at the network edge, driving the evolution of edge machine learning (ML). But during the ML model training, it may bring privacy leakage by traditional central methods. To address this issue, federated learning (FL) has been proposed as a distributed learning framework for training a global model without uploading raw data to protect data privacy. Since the communication and computing resources are usually limited in IIoT networks, how to reasonably select device and allocate bandwidth is crucial for the FL model training. Therefore, this paper proposes a joint edge device selection and bandwidth allocation scheme for FL to minimize the time-averaged cost under the given long-term energy budget and delay constraints in IIoT system. To tackle with this long-term optimization problem, we construct a virtual energy deficit queue and leverage the Lyapunov optimization theory to transform it into a list of round-wise drift-plus-cost minimization problems firstly. Then, we design an iterative algorithm to allocate reasonable bandwidth and select appropriate devices to achieve cost minimization while satisfying the energy consumption constraints. Besides, we develop an optimality analysis of the average cost and energy violation for our proposed scheme. Extensive experiments verify that our proposed scheme can achieve superior performance in cost efficiency over other schemes while guaranteeing FL training performance.

Joint Optimization of Trajectory and User Association via Reinforcement Learning for UAV-Aided Data Collection in Wireless Networks

Unmanned Aerial Vehicles (UAVs) can be used as aerial base stations for data collection in next-generation wireless networks due to their high adaptability and maneuverability. This paper investigates the scenario where multiple UAVs cooperatively fly over heterogeneous ground users (GUs) and collect data without a central controller. With the consideration of signal-to-interference-and-noise ratio (SINR) and fairness among users, we jointly optimize the trajectories of UAVs and the GUs associations to maximize the total throughput and energy efficiency. We formulate the long-term optimization problem as a decentralized partially observed Markov decision processes (DEC-POMDP) and derive an approach combining the coalition formation game (CFG) and multi-agent deep reinforcement learning (MADRL). We first formulate the discrete association scheduling problem as a non-cooperative theoretical game and use the CFG algorithm to achieve a decentralized scheme converging to Nash equilibrium (NE). Then, a MARL-based technique is developed to optimize the trajectories and energy consumption continuously in a centralized-training but decentralized-execution manner. Simulation results demonstrate that the proposed algorithm outperforms the commonly used schemes in the literature, regarding the fair throughput and energy consumption in a distributed manner.

Power and rate control in wireless communication systems with energy harvesting and rateless codes

In this paper, for a wireless communication system with energy harvesting and rateless error correction codes, the joint optimization of transmit power, modulation scheme and code rate to maximize the long-term average achievable transmission rate under the constraint of available energy is studied. The method is given first to determine the codeword length (or code rate) for a signal-to-noise ratio (SNR) under the constraint of a pre-defined decoding error probability. Then the formula of the actual transmission rate is deduced with a specific modulation and code rate, and the optimization problem to maximize the long-term average actual transmission rate is constructed under the constraints of available harvested energy and the decoding complexity of the rateless code. Since energy harvesting and channel fading are both stochastic processes, the optimization problem is difficult to solve. By using the Lyapunov optimization framework, the original long-term optimization problem is transformed into a per time slot one. Then an efficient numerical method is proposed to obtain the solution to the problem. The proposed algorithm is verified by simulation, and the results show that the proposed algorithm can achieve a higher average actual transmission rate than the comparison algorithms aiming at optimizing the channel capacity.

A Lyapunov-Optimized Dynamic Task Offloading Strategy for Satellite Edge Computing

Satellite edge computing (SEC) has garnered significant attention for its potential to deliver services directly to users. However, the uneven distribution of receiving tasks among satellites in the constellation can lead to uneven utilization of computing resources. This paper proposes a task offloading strategy for SEC that aims to minimize the average delay and energy consumption of tasks by assigning them to appropriate satellite nodes. The approach uses Lyapunov optimization to convert the long-term optimization problem with task queue length constraints into an assignment problem within a single time slot and solve it based on the Hungarian algorithm. Experimental simulations have shown that the proposed algorithm performs better than other baseline algorithms.

Jointly optimized resource allocation for SDN control and forwarding planes in edge-cloud SDN-based networks

Today’s resource allocation strategies for edge-cloud network based on Software-Defined Network (SDN) and Network Function Virtualization (NFV) architectures often make a decision of placing resource based on network locations, i.e. edge or core cloud. Unfortunately, these strategies are not optimally tailored for a growing class of Internet-of-Things (IoT) services whose traffic is designated to go through a Virtual Network Functions (VNF) service chain. Without considering the communication between SDN control plane and forwarding plane, inappropriate SDN controllers could be selected for switches, which may result in a significant setup delay of the whole service chain. In addition, the fluctuate demand of IoT services requires the entire system, that is composed of controllers, forwarding elements, i.e. switches, and physical machines, being stable during service provisioning. In this paper, we jointly address two issues, i.e. SDN controller-switch assignment and VNF placement, both are high-complexity problems. We formulate the joint optimization problem of dynamically provisioning resource for SDN controllers and VNFs in a long run. We then apply the Lyapunov optimization framework to transform a long-term optimization problem into a series of real-time problems and employ exponentiated gradient ascent (EGA) method to find a near-optimal solution. Extensive simulation and experiments show that our approach helps to optimize the system cost up to 11 ∼ 40% depending on service demand and network size.

An Energy Efficient Task Scheduling for a Cloud-edge System

With the widespread popularity of edge computing, the energy consumption of edge devices has attracted lots of research interests. This paper studies the cloud-edge system under the solar energy supply scenario, and optimizes the system parameter configuration according to the energy supply of photovoltaic power generation, to reduce the system energy consumption and prolong the system running time. Specifically, we built an edge video detection system based on Raspberry Pi, and deeply analyzed the energy consumption model of the system. Based on this, the problem of dynamically adjusting the video detection operating parameters is modeled as a long-term optimization problem. In order to solve this problem effectively, the long-term optimization problem is transformed to an instantaneous optimization problem, so that the problem can be solved efficiently in real time, and the rationality of this transformation is proved. Finally, we propose a low-energy-consumption scheduling algorithm to optimize the task scheduling and energy consumption of the system. Through simulation experiments, the performance of the proposed algorithm is evaluated. The results show that the scheduling algorithm can reduce the energy consumption of the system by an average of 18%.

A Learning-Based Context-Aware Quality Test System in B5G-Aided Advanced Manufacturing

The booming of the industrial Internet of Things (IIoT) brings an exponential increase in industrial devices, calling for more flexible and low-cost communications. The fifth generation and beyond (B5G) communication technologies provide a dedicated solution by supporting two industry-targeted technologies: Massive machine-type communications (mMTC) and ultra reliable low-latency communications (URLLC). In this article, we design a B5G-aided quality test system in advanced manufacturing, where various sensors are connected to the base station (BS) and send contextual information via mMTC. The BS and quality test machine transmit short length commands and small size feedback to each other, respectively, via URLLC. We formulate a long-term optimization problem to improve the product qualification rate by maximizing the expected average reward with limited testing capacity and changing configurations. To address this problem, we develop a novel context-aware combinatorial quality test (CC-QT) algorithm based on bandit learning (BL), which integrates contextual information to predict the product quality, and a combinatorial method to decrease the complexity of the BL process. Furthermore, we derive a performance upper bound of the proposed CC-QT and analyze its computational complexity. Experimental results illustrate the performance of CC-QT and substantiate its superiority over the existing algorithms.

Service function chain migration with the long-term budget in dynamic networks

Mobile edge computing emerges as a new paradigm to provide low-latency network services in the close proximity to users. Based on the network function virtualization (NFV) technology, network services can be flexibly provisioned as service function chain (SFC) deployed at edge servers. In some scenarios, such as the vehicular or UAV-assisted edge computing, the network topology varies rapidly due to the mobile edge servers, which changes the routing path between adjacent VNFs in an SFC. Migrating SFC to adapt to the frequent topology change can reduce the SFC latency, and improve the quality of users’ experience. However, frequent SFC migration will unavoidably increase the operation cost. In this paper, to optimize the system performance in a cost-efficient manner, we study the SFC migration problem in dynamic networks with a long-term cost budget constraint. We then propose the Topology-aware Min-latency SFC Migration (TMSM) method to strike a desirable balance between the SFC latency and the migration cost. Specifically, we first apply the Lyapunov optimization to decompose the long-term optimization problem into a series of real-time optimization sub-problems. Since the decomposed problem is still NP-hard, a Markov approximation based heuristic is proposed to seek a near-optimal solution for each sub-problem. Compared with the rerouting-only strategy, which does not migrate any VNF, our TMSM reduces the latency by at least 21% on average in each time slot. Extensive evaluations show that the proposed algorithm achieves a better tradeoff between the SFC latency and migration cost than the baselines.

A bandwidth-aware service migration method in LEO satellite edge computing network

With the development of modern sensing technology, large amounts of data are generated in areas that lack terrestrial networks, such as the field and space. This data is transmitted by satellite networks, where the interesting data is only a small part. By introducing edge computing technology, identifying and transmitting interesting data will improve the efficiency of satellite network and the quality of service for users. A satellite edge computing (SatEC) system is proposed in this paper that provides computing services through LEO satellite network. For satellite characteristics with limited resources and mobility, a bandwidth-aware service migration problem is proposed to minimize the long-term total delay cost of the system while all tasks do not exceed its deadline. Converting long-term optimization problem into a series of one-time MINLP problems, an online lazy migration algorithm is firstly proposed to solve optimal service migration decision scheme with ϵ(1+1β)-competitive ratio. Then a hierarchical iterative algorithm is proposed to decouple communication and computing resources. Finally, a bandwidth-aware virtual CPU scheduling for post-copy migration algorithm is proposed to improve the migration speed. Simulation results show that the proposed scheme effectively deal with the topology change of satellite network and reduce the long-term total delay cost of the SatEC system.

Joint Application Placement and Request Routing Optimization for Dynamic Edge Computing Service Management

As mobile edge computing (MEC) hosting applications at the network edge with limited capacities, service providers are facing the new challenge of how to make full use of the scarce edge resources to maximize the system performance. Accommodating this challenge requires careful application placement and request routing to coordinate diverse MEC nodes. However, frequent application re-placement would greatly increase the system reconfiguration cost, indicating a performance-cost trade-off. In response, in this paper, we study the problem of joint optimization on application placement and request routing to maximize the system performance, under a long-term budget of the application reconfiguration cost. Solving this problem is non-trivial since the long-term budget is coupled with the future system states (e.g., user request arrivals) that are typically unpredictable. To address this challenge, we first advocate an approximated dynamic optimization framework to decompose the long-term optimization problem into a series of one-shot problems which do not require the future system states. Moreover, since the decomposed problem is a mixed integer linear program (MILP) which is proven to be NP-hard, we then devise an efficient dependent rounding based approximation algorithm, which can achieve the near-optimal performance in a fast manner. Both rigorous theoretical analysis and extensive trace-driven evaluations demonstrate the proposed framework can achieve superior performance gain over existing schemes.

Incorporating demand response effect on transformer replacement planning

• Long term effect of loading profile on failure rate of transformers. • Effect of DR (peak load shifting) on amount of load shed at peak hours. • Failure rate related value of transformers. • Although all power market participant benefits from DR application but the cost of DR is added to the objective function and yet it was beneficiary. Transformer replacement planning is a long-term optimization problem of power system engineering. Therefore, long-term effects as loading and ageing should be considered in the problem. The most crucial reason in such a long term problem in power system is reliability issues. Accordingly, in this paper, long-term effect of loading on transformer failure rates has been modelled and demand response (DR) has been implemented to reduce loading rate of the system. In order to optimize effectiveness of DR, its implementation cost has been added to the objective function of the problem. In addition, reducing the failure rate of existing transformer may decrease the need for a new transformer so that the reliability of the system has a desirable level. On the other hand, a decrease in existing transformer loading rate in the long run, makes them more valuable at the end of the planning horizon. Hence, the value of transformers should be embedded in the objective function. Additionally, reducing loading rate causes lower loss of load at contingencies caused by transformers from two point of view, first reduced failure rate and second reduced peak load. The developed model has been implemented on transformers of THE IEEE 24-bus reliability test system (IEEE RTS) in two case studies. Results show DR can enhance network performance technically by increasing network reliability and decreasing transformer loading and can improve network operation economically by decreasing operational and expansion costs

Dynamic wireless networks assisted by RIS mounted on aerial platform: Joint active and passive beamforming design

The design of dynamic wireless networks assisted by reconfigurable intelligent surfaces (RIS) mounted on aerial platforms (RIS-APs) is conceived, where the connection status among users and RIS-APs are selected according to the average channel quality dynamically and timely. Taking into account the time-varying selection status and the mobility of users, we construct a long-term dynamic process. The goal is to minimize the time-averaged power consumption under the requirements of the time-averaged minimum rate for users as well as the constraint of the maximum transmit power for the base station (BS), via jointly optimizing the active beamforming at the BS and passive beamforming at RIS-APs. With the aid of Lyapunov concept-based drift-plus-penalty (DPP) algorithm, the long-term optimization problem is transformed into short-term sub-problems related to each other at each frame. Subsequently, the fractional programming method based on Lagrangian dual theory is applied to derive the solutions for active-passive beamforming in a closed form. Finally, simulation results validate the convergence and effectiveness of the proposed algorithm.

Intelligent Reflecting Surface Aided Wireless Networks: Dynamic User Access and System Sum-Rate Maximization

In this paper, we conceive the design of dynamic wireless networks assisted by multiple intelligent reflecting surfaces (IRSs), where the connection states between users and IRSs are capable of being updated timely. Taking into account the time-varying states of the system, we further construct a long-term dynamic process. Our goal is to maximize the time average sum-rate of the dynamic system under the time average rate and power constraints of users, via jointly optimizing the power allocation at users and the reflecting coefficients at IRSs. With the aid of Lyapunov concept-based drift-plus-penalty (DPP) algorithm, the long-term optimization problem is formulated as an infinite-horizon time-average one. Subsequently, the fractional programming method based on Lagrangian dual transform is applied to optimize power allocation and reflecting coefficients in an iterative manner, and the closed-form solutions of power and reflecting coefficients can be obtained at each iteration. Finally, simulation results demonstrate the convergence and effectiveness of the proposed algorithm. Further performance comparisons indicate that the proposed algorithm can maintain a balance between supply and demand for resource allocation and improve the fairness of users.

GreenEdge: Joint Green Energy Scheduling and Dynamic Task Offloading in Multi-Tier Edge Computing Systems

As mobile edge computing (MEC) emerges as a paradigm to meet the ever-increasing computation demands from real-time Internet of Things (IoT) applications in 5 G era, the development trends of which are mainly divided into two, with one being MEC with advanced computing architectures, and the other being MEC with high efficiency for sustainable operations. We are committed to taking advantage of these two trends to explore a novel multi-tier edge computing scenario with hierarchical task offloading and green energy provisioning via leveraging the energy harvesting (EH) technique. Specifically, we focus on the key problem of joint task offloading and energy scheduling in such green multi-tier edge computing systems. We aim to minimize the task execution cost by jointly considering the system cost that covers latency, energy consumption, and cloud rental fees. By formulating the problem as a stochastic optimization problem, we invoke the Lyapunov technique to decompose the long-term optimization problem into a series of one-slot optimization problems which only use the current system information. To solve the one-slot optimization problem which is a mixed-integer linear problem (MILP) proved to be <i>NP-hard</i>, we first relax the integer variables into real ones to obtain the optimal fractional solutions. Considering the capacity of the physical resources of each edge server, we propose a resource-constrained randomized dependent rounding algorithm to properly round up or down the fractional variables to get a feasible yet near-optimal solution. We conduct rigorous theoretical analysis and extensive simulations to verify the superior performance of the proposed schemes.

Joint Data Downloading and Resource Management for Small Satellite Cluster Networks

As an attractive alternative to traditional and large satellite, small satellite cluster (SSC) with low cost and short development time will play an important role in Earth observation. Generally, SSC networks for Earth observation are energy-limited, the design of the joint data downloading and resource management scheme remains challenging. On the one hand, with the ever-increasing amount of sensing data, multiple gateway stations and relay satellites have been deployed to receive the sensing data from SSC. The problem that where to download the sensing data should be addressed. On the other hand, due to the limited communication, storage, computing and power supply capacities of small satellite, an efficient multi-dimensional resource management scheme should be adopted. In this paper, the joint data downloading scheduling and multi-dimensional resource management problem for satellite cluster networks with relay satellites is investigated, which is formulated as a long-term optimization problem to minimize the energy consumption and guarantee the data drop rate performance. Then, this problem is decomposed into a series of mixed-integer programming problems by Lyapunov optimization theory. Leveraging many-to-many matching theory with empty node and the simplex method, an online joint data downloading and resource management (OJDDRM) scheme is proposed. Simulation results validate the proposed OJDDRM scheme&#x0027;s performance and demonstrate that there exists a balance point for the proposed OJDDRM scheme. Under the balance point, the data drop rate is zero and the energy consumption can be reduced compared with the existed schemes.