Federated Reinforcement Learning Research Articles

SFL-TUM: Energy efficient SFRL method for large scale AI model's task offloading in UAV-assisted MEC networks

The convergence of mobile edge computing (MEC) network with unmanned aerial vehicles (UAVs) presents an auspicious opportunity to revolutionize wireless communication and facilitate high-speed internet access in remote regions for mobile devices (MDs) as well as large scale artificial intelligence (AI) models. However, the substantial amount of data produced by the UAVs-assisted MEC network necessitates the integration of efficient distributed learning techniques in AI models. In recent times, distributed learning algorithms, including federated reinforcement learning (FRL) and split learning (SL), have been explored for the purpose of learning machine learning (ML) models that are distributed by sharing model parameters, as opposed to large raw data-sets as seen in traditional centralized learning algorithms. To implement the hybrid method, the model is first trained locally on each UAV-assisted MEC network using SL. Subsequently, the model parameters that have been encrypted are sent to a central server for federated averaging. Finally, after the model has been updated, it is distributed to each UAV-assisted MEC network for local fine-tuning. Our simulations indicate that the proposed split and federated reinforcement learning (SFRL) framework yields comparable high-test accuracy performance while consuming less energy compared to extant distributed learning algorithms. Furthermore, the SFRL algorithm efficiently realizes energy-efficient selection between the SL and FRL methods under different distributions. Numerical results shows that the proposed scheme improves the accuracy by 29.31% and reduced the energy consumption by around 67.34% and time delay by about 7.37%. as compared to the existing baseline schemes.

Adaptive federated reinforcement learning for critical realtime communications in UAV assisted vehicular networks

This paper sheds the light on road active safety measurements implemented in unmanned aerial vehicles assisted vehicular networks. Despite the great potential of deploying high computing drones, the drone battery life is the major concern, on one hand. On the other hand, road active safety is a critical real-time process that should be tackled in a tight time window in vehicular networks. To meet the mentioned concerns, we adopt federated machine learning on the local vehicles, sending local updates to drone servers. Moreover, a dynamic frequency adaptation framework is proposed to achieve the optimal trade-off between the road active safety performance and drone’s energy consumption. The thresholds for the local update frequency are calibrated according to road safety measurements (i.e., collision rate, risky and impolite driving time on the road) and drone energy consumption. Additionally, an accurate mathematical modeling based on M/G/1 multi-class was conducted in order to access the queuing time at the drone.

Collaborative Policy Learning for Dynamic Scheduling Tasks in Cloud–Edge–Terminal IoT Networks Using Federated Reinforcement Learning

Collaborative Policy Learning for Dynamic Scheduling Tasks in Cloud–Edge–Terminal IoT Networks Using Federated Reinforcement Learning

A Concurrent Federated Reinforcement Learning for IoT Resources Allocation With Local Differential Privacy

A Concurrent Federated Reinforcement Learning for IoT Resources Allocation With Local Differential Privacy

FedRDR: Federated Reinforcement Distillation-Based Routing Algorithm in UAV-Assisted Networks for Communication Infrastructure Failures

Traditional Internet of Things (IoT) networks have limited coverage and may experience failures due to natural disasters affecting critical IoT devices, making it difficult for them to provide communication services. Therefore, how to establish network communication service more efficiently in the presence of fault points is the problem we solve in this paper. To address this issue, this study constructs a hierarchical multi-domain data transmission architecture for an emergency network with unmanned aerial vehicles (UAVs) employed as core communication devices. This architecture expands the functionality of UAVs as key network devices and provides a theoretical basis for their feasibility as intelligent network controllers and switches. Firstly, the UAV controllers perceive the network status and learn the spatio-temporal characteristics of air-to-ground network links. Secondly, a routing algorithm within the domain based on federated reinforcement distillation (FedRDR) is developed, which enhances the generalization capability of the routing decision model by increasing the training data samples. Simulation experiments are conducted, and the results show that the average communication data size between each domain controller and the server is approximately 45.3 KB when using the FedRDR algorithm. Compared to the transmission of parameters through federated reinforcement learning algorithms, FedRDR reduces the transmitted parameter size by approximately 29%. Therefore, the FedRDR routing algorithm helps to facilitate knowledge transfer, accelerate the training process of intelligent agents within the domain, and reduce communication costs in resource-constrained scenarios for UAV networks and has practical value.

Blockchain and Federated Reinforcement Learning for Vehicle-to-Everything Energy Trading in Smart Grids

The proliferation of electric vehicles (EVs) and the advancement of Vehicle-to-Everything (V2X) energy trading systems are expected to play a crucial role in alleviating the stress on the electric grid during peak hours. However, the wide adoption of these paradigms requires intelligent mechanisms that protect the security and privacy of EV users. This paper proposes a novel Federated Reinforcement Learning (FRL) system combined with blockchain technology to maximize EV users' utility while preserving the security and privacy of trading transactions. Furthermore, we develop the concept of Proof of State of Charge (PoSOC) as a consensus mechanism to determine the winning EVs and reward them as block miners in the blockchain. The proposed system is validated through comprehensive simulation experiments utilizing a real-world dataset. The model is implemented on the Avalanche blockchain platform to demonstrate its real-world feasibility. The test results show that the proposed scheme improves EV users' utility significantly compared to the existing studies. The obtained simulation results indicate the effectiveness and robustness of the proposed system.

Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

Energy Maximization for Wireless Powered Communication Enabled IoT Devices With NOMA Underlaying Solar Powered UAV Using Federated Reinforcement Learning for 6G Networks

Energy Maximization for Wireless Powered Communication Enabled IoT Devices With NOMA Underlaying Solar Powered UAV Using Federated Reinforcement Learning for 6G Networks

On-Device Indoor Positioning: A Federated Reinforcement Learning Approach With Heterogeneous Devices

On-Device Indoor Positioning: A Federated Reinforcement Learning Approach With Heterogeneous Devices

A Hybrid Task Offloading and Resource Allocation Approach for Digital Twin-Empowered UAV-Assisted MEC Network Using Federated Reinforcement Learning for Future Wireless Network

A Hybrid Task Offloading and Resource Allocation Approach for Digital Twin-Empowered UAV-Assisted MEC Network Using Federated Reinforcement Learning for Future Wireless Network

Dynamic Multi-Sleeping Control with Diverse Quality-of-Service Requirements in Sixth-Generation Networks Using Federated Learning

The intensive deployment of sixth-generation (6G) base stations is expected to greatly enhance network service capabilities, offering significantly higher throughput and lower latency compared to previous generations. However, this advancement is accompanied by a notable increase in the number of network elements, leading to increased power consumption. This not only worsens carbon emissions but also significantly raises operational costs for network operators. To address the challenges arising from this surge in network energy consumption, there is a growing focus on innovative energy-saving technologies designed for 6G networks. These technologies involve strategies for dynamically adjusting the operational status of base stations, such as activating sleep modes during periods of low demand, to optimize energy use while maintaining network performance and efficiency. Furthermore, integrating artificial intelligence into the network’s operational framework is being explored to establish a more energy-efficient, sustainable, and cost-effective 6G network. In this paper, we propose a small base station sleeping control scheme in heterogeneous dense small cell networks based on federated reinforcement learning, which enables the small base stations to dynamically enter appropriate sleep modes, to reduce power consumption while ensuring users’ quality-of-service (QoS) requirements. In our scheme, double deep Q-learning is used to solve the complex non-convex base station sleeping control problem. To tackle the dynamic changes in QoS requirements caused by user mobility, small base stations share local models with the macro base station, which acts as the central control unit, via the X2 interface. The macro base station aggregates local models into a global model and then distributes the global model to each base station for the next round of training. By alternately performing model training, aggregation, and updating, each base station in the network can dynamically adapt to changes in QoS requirements brought about by user mobility. Simulations show that compared with methods based on distributed deep Q-learning, our proposed scheme effectively reduces the performance fluctuations caused by user handover and achieves lower network energy consumption while guaranteeing users’ QoS requirements.

An Optimal Transport-Based Federated Reinforcement Learning Approach for Resource Allocation in Cloud–Edge Collaborative IoT

An Optimal Transport-Based Federated Reinforcement Learning Approach for Resource Allocation in Cloud–Edge Collaborative IoT

Modeling Practically Private Wireless Vehicle to Grid System With Federated Reinforcement Learning

Modeling Practically Private Wireless Vehicle to Grid System With Federated Reinforcement Learning

Federated Reinforcement Learning with Knowledge Transfer for Network Selection in Hybrid WiFi-VLC Networks

Federated Reinforcement Learning with Knowledge Transfer for Network Selection in Hybrid WiFi-VLC Networks

Location Privacy Preservation Crowdsensing with Federated Reinforcement Learning

Location Privacy Preservation Crowdsensing with Federated Reinforcement Learning

FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning

FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning

UAV Communication Against Intelligent Jamming: A Stackelberg Game Approach With Federated Reinforcement Learning

UAV Communication Against Intelligent Jamming: A Stackelberg Game Approach With Federated Reinforcement Learning

Federated and Online Dynamic Spectrum Access for Mobile Secondary Users

Users in dynamic spectrum access (DSA) with federated reinforcement learning (FRL) autonomously access channels, avoiding centralized coordination and protecting users’ privacy. However, existing FRL-based DSA mechanisms are limited to ideal network states, i.e., assuming that channel states and users’ interference relationships are unchanged. Besides, users should upload intermediate results simultaneously for federated aggregation. The above conditions are impractical for mobile users since their network states and locations are unstable. Meanwhile, newly connected users have to train their models through local data with numerous computing resources since global models are unsuitable for them. We propose FRDSA, an FRL-based secure and lightweight channel selection mechanism in DSA for mobile users under dynamic network states. An independent channel selection environment with a virtual group strategy is presented to avoid interference between users under unstable channel states. Furthermore, an asynchronous parameter aggregation method in FRDSA dynamically adjusts the aggregation factors without users simultaneously uploading intermediate results. Simulations based on real trajectory data show that FRDSA significantly reduces approximately 60% interference between mobile users under unstable network states. Newly connected users can directly apply the well-trained global model to access channels autonomously instead of retraining a model, effectively reducing mobile users’ computing resource requirements.

Meta Federated Reinforcement Learning for Distributed Resource Allocation

Meta Federated Reinforcement Learning for Distributed Resource Allocation

Complex Network Cognition-based Federated Reinforcement Learning for End-to-end Urban Autonomous Driving

Complex Network Cognition-based Federated Reinforcement Learning for End-to-end Urban Autonomous Driving