Learning Based Resource Allocation Research Articles

Deep Reinforcement Learning Based Resource Allocation Strategy in Cloud-Edge Computing System.

The rapid development of mobile device applications put tremendous pressure on edge nodes with limited computing capabilities, which may cause poor user experience. To solve this problem, collaborative cloud-edge computing is proposed. In the cloud-edge computing, an edge node with limited local resources can rent more resources from a cloud node. According to the nature of cloud service, cloud service can be divided into private cloud and public cloud. In a private cloud environment, the edge node must allocate resources between the cloud node and the edge node. In a public cloud environment, since public cloud service providers offer various pricing modes for users’ different computing demands, the edge node also must select the appropriate pricing mode of cloud service; which is a sequential decision problem. In this stydy, we model it as a Markov decision process and parameterized action Markov decision process, and we propose a resource allocation algorithm cost efficient resource allocation with private cloud (CERAI) and cost efficient resource allocation with public cloud (CERAU) in the collaborative cloud-edge environment based on the deep reinforcement learning algorithm deep deterministic policy gradient and P-DQN. Next, we evaluated CERAI and CERAU against three typical resource allocation algorithms based on synthetic and real data of Google datasets. The experimental results demonstrate that CERAI and CERAU can effectively reduce the long-term operating cost of collaborative cloud-side computing in various demanding settings. Our analysis can provide some useful insights for enterprises to design the resource allocation strategy in the collaborative cloud-side computing system.

Deep Learning-Based Resource Allocation for Device-to-Device Communication

In this paper, a deep learning (DL) framework for the optimization of the resource allocation in multi-channel cellular systems with device-to-device (D2D) communication is proposed. Thereby, the channel assignment and discrete transmit power levels of the D2D users, which are both integer variables, are optimized for maximization of the overall spectral efficiency whilst maintaining the quality-of-service (QoS) of the cellular users. Depending on the availability of channel state information (CSI), two different configurations are considered, namely 1) centralized operation with full CSI and 2) distributed operation with partial CSI, where in the latter case, the CSI is encoded according to the capacity of the feedback channel. Instead of solving the resulting resource allocation problem for each channel realization, a DL framework is proposed, where the optimal resource allocation strategy for arbitrary channel conditions is approximated by deep neural network (DNN) models. Furthermore, we propose a new training strategy that combines supervised and unsupervised learning methods and a local CSI sharing strategy to achieve near-optimal performance while enforcing the QoS constraints of the cellular users and efficiently handling the integer optimization variables based on a few ground-truth labels. Our simulation results confirm that near-optimal performance can be attained with low computation time, which underlines the real-time capability of the proposed scheme. Moreover, our results show that not only the resource allocation strategy but also the CSI encoding strategy can be efficiently determined using a DNN. Furthermore, we show that the proposed DL framework can be easily extended to communication systems with different design objectives.

Deep Reinforcement Learning-based Resource Allocation for 5G Machine-type Communication in Active Distribution Networks with Time-varying Interference

Deep Reinforcement Learning-based Resource Allocation for 5G Machine-type Communication in Active Distribution Networks with Time-varying Interference

Deep learning–based resource allocation for secure transmission in a non-orthogonal multiple access network

Machine learning techniques, especially deep learning algorithms have been widely utilized to deal with different kinds of research problems in wireless communications. In this article, we investigate the secrecy rate maximization problem in a non-orthogonal multiple access network based on deep learning approach. In this non-orthogonal multiple access network, the base station intends to transmit two integrated information: a confidential information to user 1 (the strong user) and a broadcast information to user 1 and user 2. In addition, there exists an eavesdropper that intends to decode the confidential information due to the broadcast nature of radio waves. Hence, we formulate the optimization problem as a secrecy rate maximization problem. We first solve this problem by employing convex optimization technique, then we generate the training, validation, and test dataset. We propose a deep neural network–based approach to learn to optimize the resource allocations. The advantages of the proposed deep neural network are the capabilities to achieve low complexity and latency resource allocations. Simulation results are provided to show that the proposed deep neural network approach is capable of reaching near-optimal secrecy rate performance with significantly reduced computational time, when compared with the benchmark conventional approach.

DRAM: A DRL-based resource allocation scheme for MAR in MEC

Mobile Edge Computing (MEC) and 5G technology allow clients to access computing resources at the network frontier, which paves the way for applying Mobile Augmented Reality (MAR) applications. Under the MEC paradigm, MAR clients can offload complex tasks to the MEC server and enhance the human perception of the world by merging the received virtual information with the real environment. However, the resource allocation problem arises as a critical challenge in circumstances where several MAR clients compete for limited resources at the network frontier. In this paper, we aim to design an online resource allocation scheme on the MEC server that takes both high quality of experience and good fairness performance for MAR clients into consideration. We first formulate this problem as a Markov decision process and tackle the challenge of applying the deep reinforcement learning paradigm. Then, we propose DRAM, a Deep reinforcement learning-based Resource allocation scheme for mobile Augmented reality service in MEC. We also propose a self-adaptive algorithm on the MAR client that is derived based on the analysis of the MAR service to tackle client adaptation problems. The simulation results demonstrated that DRAM can provide high quality of experience and simultaneously achieve good fairness performance by coordinating with clients’ adaptation algorithms.

Deep Reinforcement Learning-Based Resource Allocation for Satellite Internet of Things with Diverse QoS Guarantee.

Large-scale terminals’ various QoS requirements are key challenges confronting the resource allocation of Satellite Internet of Things (S-IoT). This paper presents a deep reinforcement learning-based online channel allocation and power control algorithm in an S-IoT uplink scenario. The intelligent agent determines the transmission channel and power simultaneously based on contextual information. Furthermore, the weighted normalized reward concerning success rate, power efficiency, and QoS requirement is adopted to balance the performance between increasing resource efficiency and meeting QoS requirements. Finally, a practical deployment mechanism based on transfer learning is proposed to promote onboard training efficiency and to reduce computation consumption of the training process. The simulation demonstrates that the proposed method can balance the success rate and power efficiency with QoS requirement guaranteed. For S-IoT’s normal operation condition, the proposed method can improve the power efficiency by 60.91% and 144.44% compared with GA and DRL_RA, while its power efficiency is only 4.55% lower than that of DRL-EERA. In addition, this method can be transferred and deployed to a space environment by merely 100 onboard training steps.

Deep Reinforcement Learning-Based Resource Allocation for Cellular Vehicular Network Mode 3 with Underlay Approach.

Vehicle-to-vehicle (V2V) communication has attracted increasing attention since it can improve road safety and traffic efficiency. In the underlay approach of mode 3, the V2V links need to reuse the spectrum resources preoccupied with vehicle-to-infrastructure (V2I) links, which will interfere with the V2I links. Therefore, how to allocate wireless resources flexibly and improve the throughput of the V2I links while meeting the low latency requirements of the V2V links needs to be determined. This paper proposes a V2V resource allocation framework based on deep reinforcement learning. The base station (BS) uses a double deep Q network to allocate resources intelligently. In particular, to reduce the signaling overhead for the BS to acquire channel state information (CSI) in mode 3, the BS optimizes the resource allocation strategy based on partial CSI in the framework of this article. The simulation results indicate that the proposed scheme can meet the low latency requirements of V2V links while increasing the capacity of the V2I links compared with the other methods. In addition, the proposed partial CSI design has comparable performance to complete CSI.

Deep Learning Based Auction-Driven Beamforming for Wireless Information and Power Transfer

In this paper, we design a deep learning based resource allocation framework, in the form of an auction, for simultaneous information and power transfer from a hybrid access point (AP) to information devices and energy harvesting devices, respectively. Using Myerson’s lemma and the concept of virtual welfare maximization, we develop an optimal dominant-strategy incentive-compatible mechanism for the AP to maximize its expected revenue, based on the devices’ bid profiles, valuation distributions, demand profiles, and channel state information. In so doing, we formulate the revenue maximization problem, which is a mixed-integer non-linear program, and propose an efficient Branch-and-Bound (BnB) algorithm to solve the problem using semidefinite relaxation technique in each branch. Since the problem has exponential time complexity, using BnB algorithms can be impractical for real-time applications. To circumvent this, a deep neural network (DNN) is proposed, and trained to predict the optimal mechanism for beamforming the data and the energy towards the information and energy devices, respectively. We use the BnB algorithm to solve the problem offline and populate the training dataset. The proposed DNN architecture is indeed a multi-layer perceptron, which is trained well to map the heterogeneous input to the desired output with high accuracy. Furthermore, we propose a heuristic iterative solution whose accuracy performance is comparable to that of the DNN-based solution. The heuristic solution has polynomial time complexity whereas the DNN-based solution has linear time complexity.

Multiagent Deep-Reinforcement-Learning-Based Resource Allocation for Heterogeneous QoS Guarantees for Vehicular Networks

Vehicle-to-vehicle communications can offer direct information interaction, including security-centered information and entertainment information. However, the rapid proliferation of vehicles and the diversity of communications services demand for a more intelligent and efficient resource allocation framework to enhance network performance. In this article, a multi-agent deep reinforcement learning-based resource allocation framework is developed to jointly optimize the channel allocation and power control to satisfy the heterogeneous Quality-of-Service (QoS) requirements in heterogeneous vehicular networks. In the proposed framework, the utility maximization problem is formulated by considering two types of traffics, i.e., the strict ultrareliable and low-latency requirements for safety-centric applications and the high-capacity requirements for entertainment applications. The utility of each vehicular users is formulated as a multicriterion objective function by taking into account the heterogeneous traffic requirements. To overcome the drawbacks of the traditional totally centralized and distributed deep reinforcement learning-based resource allocation approaches, we propose a multi-agent deep deterministic policy gradient algorithm with centralized learning and decentralized execution to solve the formulated optimization problem. The normalization of the input states and reward functions is introduced to speed up the training and learning progress of the proposed algorithm. Simulation results show the superiority of the proposed algorithm in terms of the convergence and system performance through the comparison with the other methods and schemes for the delay-sensitive applications and delay-tolerant applications.

Deep learning driven physical layer security for a simultaneously wireless information and power transfer network

Artificial intelligent approaches have been considered as the promising techniques to enable smart communications in future wireless networks. In this paper, we investigate the deep learning based resource allocation approach for secure transmission in a simultaneously wireless information and power transfer (SWIPT) network. In particular, we design the resource allocations to maximize the minimum achievable secrecy rate of the legitimate user under the constraints of energy harvesting requirements of the energy receivers (ERs). Conventionally, the optimal or suboptimal solutions of resource allocation problems can be obtained by exploiting convex optimization approaches, which are often developed based on iterative algorithms, and always result in long computational time. To satisfy ultra low latency demands and achieve physical layer security for future SWIPT systems, we develop a DNN based approach that has the capability to optimize the power allocations for a SWIPT network, where the computational time and complexity have been significantly cut down. Numerical results are provided to illustrate that the effectiveness of our proposed DNN based approach, which is capable to achieve near optimal secrecy rate performances in comparing with convex optimization approach.

Distributed Learning-Based Resource Allocation for Self-Organizing C-V2X Communication in Cellular Networks

In this paper, we investigate a resource allocation problem for a Cellular Vehicle to Everything (C-V2X) network to improve energy efficiency of the system. To address this problem, self-organizing mechanisms are proposed for joint and disjoint subcarrier and power allocation procedures which are performed in a fully distributed manner. A multi-agent Q-learning algorithm is proposed for the joint power and subcarrier allocation. In addition, for the sake of simplicity, it is decoupled into two sub-problems: a subcarrier allocation sub-problem and a power allocation sub-problem. First, to allocate the subcarrier among users, a distributed Q-learning method is proposed. Then, given the optimal subcarriers, a dynamic power allocation mechanism is proposed where the problem is modeled as a non-cooperative game. To solve the problem, a no-regret learning algorithm is utilized. To evaluate the performance of the proposed approaches, other learning mechanisms are used which are presented in fig.8. Simulation results show the multi-agent joint Q-learning algorithm yields significant performance gains of up to about 11% and 18% in terms of energy efficiency compared to proposed disjoint mechanism and the third disjoint Q-learning mechanism for allocating the power and subcarrier to each user; however, the multi-agent joint Q-learning algorithm uses more memory than disjoint methods.

Multi-Agent Deep Reinforcement Learning-Based Resource Allocation in HPC/AI Converged Cluster

As the complexity of deep learning (DL) networks and training data grows enormously, methods that scale with computation are becoming the future of artificial intelligence (AI) development. In this regard, the interplay between machine learning (ML) and high-performance computing (HPC) is an innovative paradigm to speed up the efficiency of AI research and development. However, building and operating an HPC/AI converged system require broad knowledge to leverage the latest computing, networking, and storage technologies. Moreover, an HPC-based AI computing environment needs an appropriate resource allocation and monitoring strategy to efficiently utilize the system resources. In this regard, we introduce a technique for building and operating a high-performance AI-computing environment with the latest technologies. Specifically, an HPC/AI converged system is configured inside Gwangju Institute of Science and Technology (GIST), called GIST AI-X computing cluster, which is built by leveraging the latest Nvidia DGX servers, high-performance storage and networking devices, and various open source tools. Therefore, it can be a good reference for building a small or middle-sized HPC/AI converged system for research and educational institutes. In addition, we propose a resource allocation method for DL jobs to efficiently utilize the computing resources with multi-agent deep reinforcement learning (mDRL). Through extensive simulations and experiments, we validate that the proposed mDRL algorithm can help the HPC/AI converged cluster to achieve both system utilization and power consumption improvement. By deploying the proposed resource allocation method to the system, total job completion time is reduced by around 20% and inefficient power consumption is reduced by around 40%.

Deep Reinforcement Learning Based Resource Allocation with Radio Remote Head Grouping and Vehicle Clustering in 5G Vehicular Networks

With increasing data traffic requirements in vehicular networks, vehicle-to-everything (V2X) communication has become imperative in improving road safety to guarantee reliable and low latency services. However, V2X communication is highly affected by interference when changing channel states in a high mobility environment in vehicular networks. For optimal interference management in high mobility environments, it is necessary to apply deep reinforcement learning (DRL) to allocate communication resources. In addition, to improve system capacity and reduce system energy consumption from the traffic overheads of periodic messages, a vehicle clustering technique is required. In this paper, a DRL based resource allocation method is proposed with remote radio head grouping and vehicle clustering to maximize system energy efficiency while considering quality of service and reliability. The proposed algorithm is compared with three existing algorithms in terms of performance through simulations, in each case outperforming the existing algorithms in terms of average signal to interference noise ratio, achievable data rate, and system energy efficiency.

Deep Reinforcement Learning-Based Resource Allocation in Cooperative UAV-Assisted Wireless Networks

We consider the downlink of an unmanned aerial vehicle (UAV) assisted cellular network consisting of multiple cooperative UAVs, whose operations are coordinated by a central ground controller using wireless fronthaul links, to serve multiple ground user equipments (UEs). A problem of jointly designing UAVs’ positions, transmit beamforming, as well as UAV-UE association is formulated in the form of mixed integer nonlinear programming (MINLP) to maximize the sum UEs’ achievable rate subject to limited fronthaul capacity constraints. Solving the considered problem is hard owing to its non-convexity and the unavailability of channel state information (CSI) due to the movement of UAVs. To tackle these effects, we propose a novel algorithm comprising of two distinguishing features: (i) exploiting a deep Q-learning approach to tackle the issue of CSI unavailability for determining UAVs’ positions, (ii) developing a difference of convex algorithm (DCA) to efficiently solve for the UAV’s transmit beamforming and UAV-UE association. The proposed algorithm recursively solves the problem of interest until convergence, where each recursion executes two steps. In the first step, the deep Q-learning (DQL) algorithm allows UAVs to learn the overall network state and account for the joint movement of all UAVs to adapt their locations. In the second step, given the determined UAVs’ positions from the DQL algorithm, the DCA iteratively solves a convex approximate subproblem of the original non-convex MINLP problem with the updated parameters, where the problem’s variables are transmit beamforming and UAV-UE association. Numerical results show that our design outperforms the existing algorithms in terms of algorithmic convergence and network performance with a gain of up to 70%.

Deep reinforcement learning-based resource allocation for D2D communications in heterogeneous cellular networks

Device-to-Device (D2D) communication-enabled Heterogeneous Cellular Networks (HCNs) have been a promising technology for satisfying the growing demands of smart mobile devices in fifth-generation mobile networks. The introduction of Millimeter Wave (mm-wave) communications into D2D-enabled HCNs allows higher system capacity and user data rates to be achieved. However, interference among cellular and D2D links remains severe due to spectrum sharing. In this paper, to guarantee user Quality of Service (QoS) requirements and effectively manage the interference among users, we focus on investigating the joint optimization problem of mode selection and channel allocation in D2D-enabled HCNs with mm-wave and cellular bands. The optimization problem is formulated as the maximization of the system sum-rate under QoS constraints of both cellular and D2D users in HCNs. To solve it, a distributed multiagent deep Q-network algorithm is proposed, where the reward function is redefined according to the optimization objective. In addition, to reduce signaling overhead, a partial information sharing strategy that does not observe global information is proposed for D2D agents to select the optimal mode and channel through learning. Simulation results illustrate that the proposed joint optimization algorithm possesses good convergence and achieves better system performance compared with other existing schemes.

Machine Learning-Based Resource Allocation in Satellite Networks Supporting Internet of Remote Things

Satellite networks have been regarded as a promising architecture for supporting the Internet of remote things (IoRT) due to their advantages of wide coverage and high communication capacity in remote areas, which further promotes the development of the satellites for IoRT networks (SIoRTNs). The effectiveness of multi-dimensional resource collaboration has significant impacts on the IoRT data downloading performance. However, the environment ’ s dynamics, e.g., channel conditions and solar infeed process, are unknown in practical scenarios, which poses daunting challenges in making efficient utilization of multi-dimensional resources. Motivated by this fact, we model the joint resource scheduling and IoRT data scheduling problem with the aim of maximizing the amount of the IoRT data of the overall network by applying the model-free reinforcement learning framework. To overcome the limitations of traditional reinforcement learning algorithms, we propose several feature functions by investigating the natural attributes of the multi-dimensional resources of the SIoRTNs, and further exploit the concept of function approximation to approximate the expected downloaded IoRT data given the network state. Furthermore, we propose a state-action-reward-state-action (SARSA) based actor-critic reinforcement learning (SACRL) resource allocation strategy to achieve the optimal resource allocation and IoRT data scheduling with casual information at LEO satellites. Simulations validate the convergence property and the effectiveness of the proposed SACRL algorithm in terms of the amount of the downloaded IoRT data. Particularly, we investigate the impact of typical network parameters on network performance to further provide guidance for future SIoRTN system design.

Meta-Reinforcement Learning Based Resource Allocation for Dynamic V2X Communications

This paper studies the allocation of shared resources between vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) links in vehicle-to-everything (V2X) communications. In existing algorithms, dynamic vehicular environments and quantization of continuous power become the bottlenecks for providing an effective and timely resource allocation policy. In this paper, we develop two algorithms to deal with these difficulties. First, we propose a deep reinforcement learning (DRL)-based resource allocation algorithm to improve the performance of both V2I and V2V links. Specifically, the algorithm uses deep Q-network (DQN) to solve the sub-band assignment and deep deterministic policy-gradient (DDPG) to solve the continuous power allocation problem. Second, we propose a meta-based DRL algorithm to enhance the fast adaptability of the resource allocation policy in the dynamic environment. Numerical results demonstrate that the proposed DRL-based algorithm can significantly improve the performance compared to the DQN-based algorithm that quantizes continuous power. In addition, the proposed meta-based DRL algorithm can achieve the required fast adaptation in the new environment with limited experiences.

Deep reinforcement learning-based resource allocation and seamless handover in multi-access edge computing based on SDN

Deep reinforcement learning-based resource allocation and seamless handover in multi-access edge computing based on SDN

Smart Resource Allocation for Mobile Edge Computing: A Deep Reinforcement Learning Approach

The development of mobile devices with improving communication and perceptual capabilities has brought about a proliferation of numerous complex and computation-intensive mobile applications. Mobile devices with limited resources face more severe capacity constraints than ever before. As a new concept of network architecture and an extension of cloud computing, Mobile Edge Computing (MEC) seems to be a promising solution to meet this emerging challenge. However, MEC also has some limitations, such as the high cost of infrastructure deployment and maintenance, as well as the severe pressure that the complex and mutative edge computing environment brings to MEC servers. At this point, how to allocate computing resources and network resources rationally to satisfy the requirements of mobile devices under the changeable MEC conditions has become a great aporia. To combat this issue, we propose a smart, Deep Reinforcement Learning based Resource Allocation (DRLRA) scheme, which can allocate computing and network resources adaptively, reduce the average service time and balance the use of resources under varying MEC environment. Experimental results show that the proposed DRLRA performs better than the traditional OSPF algorithm in the mutative MEC conditions.

A novel approach to efficient resource allocation in load-balanced cellular networks using hierarchical DRL

5G wireless networks require highly spectral-efficient multiple access techniques, which play an important role in determining the performance of mobile communication systems. Multiple access techniques can be classified into orthogonal and nonorthogonal based on the way the resources are allocated to the users. This paper investigates the joint user association and resource allocation problem in an uplink multicast NOMA system to maximize the power efficiency with guaranteeing the quality-of-experience of all subscribers. We also introduce an adaptive load balancing approach that aspires to obtain “almost optimal” fairness among servers from the quality of service (QoS) perspective in which learning automata (LA) has been used to find the optimal solution for this dynamic problem. This approach contains a sophisticated learning automata which consists of time-separation and the “artificial” ergodic paradigms. Different from conventional model-based resource allocation methods, this paper suggested a hierarchical reinforcement learning based frameworks to solve this non-convex and dynamic power optimization problem, referred to as hierarchical deep learning-based resource allocation framework. The entire resource allocation policies of this framework are adjusted by updating the weights of their neural networks according to feedback of the system. The presented learning automata find the $$\epsilon$$ -optimal solution for the problem by resorting to a two-time scale-based SLA paradigm. Numerical results show that the suggested hierarchical resource allocation framework in combination with the load balancing approach, can significantly improve the energy efficiency of the whole NOMA system compared with other approaches.