Learning In Wireless Networks Research Articles

Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks

Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks

The Threat of Disruptive Jamming to Blockchain-Based Decentralized Federated Learning in Wireless Networks.

Machine learning techniques have attracted considerable attention for wireless networks because of their impressive performance in complicated scenarios and usefulness in various applications. However, training with and sharing raw data obtained locally from each wireless node does not guarantee privacy and requires a large communication overhead. To mitigate such issues, federated learning (FL), in which sharing parameters for model updates are shared instead of raw data, has been developed. FL has also been studied using blockchain techniques to efficiently perform learning in distributed wireless systems without having to deploy a centralized server. Although blockchain-based decentralized federated learning (BDFL) is a promising technique for various wireless sensor networks, malicious attacks can still occur, which result in performance degradation or malfunction. In this study, we analyze the impact of a jamming threats from malicious miners to BDFL in wireless networks. In a wireless BDFL system, it is possible for malicious miners with jamming capability to interfere with the collection of model parameters by normal miners, thus preventing the victim miner from generating a global model. By disrupting normal miners participating in BDFL systems, malicious miners with jamming capability can more easily add malicious data to the mainstream. Through various simulations, we evaluated the success probability performance of malicious block insertion and the participation rate of normal miners in a wireless BDFL system.

Graph Representation Learning for Wireless Communications

Wireless networks are inherently graph-structured, in which, graph representation learning can be utilized to solve complex network optimization problems. In graph representation learning, feature vectors for each entity in the network are calculated such that they could capture spatial and temporal dependencies in their local and global neighborhoods. Specifically, graph neural networks (GNNs) are powerful tools to solve these complex problems because of their expressive representation and reasoning power. In this paper, the potential of graph representation learning and GNNs in wireless networks is presented. An overview of graph representation learning is provided which covers the fundamentals and concepts such as feature design over graphs, GNNs, and their design principles. The potential of graph representation learning in wireless networks is presented via a few exemplary use cases and some initial results on the GNNbased access point selection for cell-free massive Multiple-Input Multiple-Output (MIMO) systems.

AoU-Based Local Update and User Scheduling for Semi-Asynchronous Online Federated Learning in Wireless Networks

AoU-Based Local Update and User Scheduling for Semi-Asynchronous Online Federated Learning in Wireless Networks

Hierarchical Federated Learning in Wireless Networks: Pruning Tackles Bandwidth Scarcity and System Heterogeneity

Hierarchical Federated Learning in Wireless Networks: Pruning Tackles Bandwidth Scarcity and System Heterogeneity

Hierarchically Federated Learning in Wireless Networks: D2D Consensus and Inter-Cell Aggregation

Hierarchically Federated Learning in Wireless Networks: D2D Consensus and Inter-Cell Aggregation

Efficiency-Boosting Federated Learning in Wireless Networks: A Long-Term Perspective

Federated learning (FL) can train a global model from clients' local dataset, which can make full use of the computing resources of clients and performs more extensive and efficient machine learning applications on clients with protecting user information requirements. Many existing works have focused on optimizing FL accuracy within the resource constrained in each individual round. However there are few works comprehensively consider the optimization for latency, accuracy and energy consumption from a long-term perspective in wireless federated learning. Inspired by this, in this paper, we investigate FL in wireless networks where client selection and bandwidth allocation are two crucial factors for improving the FL accuracy as well as reducing latency and energy consumption. We formulate the optimization problem as a mixed-integer problem, which minimizes the cost function over finite rounds while satisfying energy budget constraints of each client from a long-term perspective. To address this optimization problem, we propose the Per-round Energy Drift Plus Cost (PEDPC) algorithm in an online manner, including two parts: client selection and bandwidth allocation, which can be addressed by Increasing Time-Maximum Client Selection (ITMCS) algorithm and Barrier Method, respectively. Finally, the performance of the PEDPC algorithm is verified by extensive simulations in terms of latency, accuracy and energy consumption in IID and NON-IID data distributions.

Ensemble Distillation Based Adaptive Quantization for Supporting Federated Learning in Wireless Networks

Federated learning (FL) has become a promising technique for developing intelligent wireless networks. In traditional FL paradigms, local models are usually required to be homogeneous for aggregation. However, due to heterogeneous models coming with wireless sysTem heterogeneity, it is preferable for user equipments (UEs) to undertake appropriate amount of computing and/or data transmission work based on sysTem constraints. Meanwhile, considerable communication costs are incurred by model training, when a large number of UEs participate in FL and/or the transmitted models are large. Therefore, resource-efficient training schemes for heterogeneous models are essential for enabling FL-based intelligent wireless networks. In this paper, we propose an adaptive quantization scheme based on ensemble distillation (AQeD), to facilitate heterogeneous model training. We first partition and group the participating UEs into clusters, where the local models in specific clusters are homogeneous with different quantization levels. Then we propose an augmented loss function by jointly considering ensemble distillation loss, quantization levels and wireless resources constraints. In AQeD, model aggregations are performed at two levels: model aggregation for individual clusters and distillation loss aggregation for cluster ensembles. Numerical results show that the AQeD scheme can significantly reduce communication costs and training time in comparison with some state-of-the-art solutions.

Physical Layer Authentication Based on Integrated Semi-Supervised Learning in Wireless Networks for Dynamic Industrial Scenarios

Physical Layer Authentication Based on Integrated Semi-Supervised Learning in Wireless Networks for Dynamic Industrial Scenarios

WFLTree: A Spanning Tree Construction for Federated Learning in Wireless Networks

Nowadays, more and more federated learning algorithms have been implemented in edge computing, to provide various customized services for mobile users, which has strongly supported the rapid development of edge intelligence. However, most of them are designed relying on the reliable device-to-device communications, which is not a realistic assumption in the wireless environment. This paper considers a realistic aggregation problem for federated learning in a single-hop wireless network, in which the parameters of machine learning models are aggregated from the learning agents to a parameter server via a wireless channel with physical interference constraint. Assuming that all the learning agents and the parameter server are within a distance [Formula: see text] from each other, we show that it is possible to construct a spanning tree to connect all the learning agents to the parameter server for federated learning within [Formula: see text] time steps. After the spanning tree is constructed, it only takes [Formula: see text] time steps to aggregate all the training parameters from the learning agents to the parameter server. Thus, the server can update its machine learning model once according to the aggregated results. Theoretical analyses and numerical simulations are conducted to show the performance of our algorithm.

Guest Editorial Special Section on Distributed Edge Learning in Wireless Networks

Guest Editorial Special Section on Distributed Edge Learning in Wireless Networks

Device Selection and Resource Allocation for Layerwise Federated Learning in Wireless Networks

In this article, we study device selection and resource allocation (DSRA) for layerwise federated learning (FL) in wireless networks. For effective learning, DSRA should be carefully determined considering the characteristics of both layerwise FL and wireless networks. To address this, we propose a DSRA algorithm for layerwise FL, called LAFLAS, that maximizes the total average number of the shallow and entire parameter transmissions over time in FL while guaranteeing the ratio between their numbers. Through experiments, we show that the model trained by our LAFLAS outperforms those by state-of-the-art algorithms, which demonstrates the effectiveness of DSRA by LAFLAS.

FedPCC: Parallelism of Communication and Computation for Federated Learning in Wireless Networks

The advances of both computation and communication technologies facilitate the exploitation of massive data generated by mobile devices. It is attractive to leverage these data and computation resources to train high-performance machine learning (ML) models. In traditional ML methods, all data are uploaded to servers for model training, which incur issues of large overhead and privacy concerns. Federated learning (FL) was proposed to address these issues. In this paper, we propose FedPCC, an efficient approach based on the parallelism of communication and computation among devices for FL in wireless networks. FedPCC considers the difference in communication and computation capabilities between different devices and optimizes the training schedule for devices selected in each round. Instead of dividing an FL training round into separate communication and computation steps, FedPCC protocol arranges devices to download the global model sequentially and then start local model training immediately once finishing downloading, which allows better utilization of communication and computation resources. Specifically, we formulate an optimization problem to minimize the time for each training round and propose an algorithm to solve the problem based on the trade-off of devices’ communication and computation capabilities. FedPCC uses a heuristic algorithm to make slow devices start relatively earlier, which can shorten the time in one training round, thus reducing the entire training time. We conduct extensive experiments to demonstrate that the proposed protocol outperforms the existing protocols under different system settings.

Resource Consumption for Supporting Federated Learning in Wireless Networks

Federated learning (FL) has recently become one of the hottest focuses in wireless edge networks with the ever-increasing computing capability of user equipment (UE). In FL, UEs train local machine learning models and transmit them to an aggregator, where a global model is formed and then sent back to UEs. In wireless networks, local training and model transmission can be unsuccessful due to constrained computing resources, wireless channel impairments, bandwidth limitations, etc., which degrades FL performance in model accuracy and/or training time. Moreover, we need to quantify the benefits and cost of deploying edge intelligence, as model training and transmission consume certain amount of resources. Therefore, it is imperative to deeply understand the relationship between FL performance and multiple-dimensional resources. In this paper, we construct an analytical model to investigate the relationship between the FL model accuracy and consumed resources in FL empowered wireless edge networks. Based on the analytical model, we explicitly quantify the model accuracy, available computing resources and communication resources. Numerical results validate the effectiveness of our theoretical modeling and analysis, and demonstrate the trade-off between the communication and computing resources for achieving a certain model accuracy.

Revisiting Analog Over-the-Air Machine Learning: The Blessing and Curse of Interference

We study a distributed machine learning problem carried out by an edge server and multiple agents in a wireless network. The objective is to minimize a global function that is a sum of the agents&#x2019; local loss functions. And the optimization is conducted by analog over-the-air model training. Specifically, each agent modulates its local gradient onto a set of waveforms and transmits to the edge server simultaneously. From the received analog signal the edge server extracts a noisy aggregated gradient which is distorted by the channel fading and interference, and uses it to update the global model and feedbacks to all the agents for another round of local computing. Since the electromagnetic interference generally exhibits a heavy-tailed intrinsic, we use the <inline-formula><tex-math notation="LaTeX">$\alpha$</tex-math></inline-formula>-stable distribution to model its statistic. In consequence, the global gradient has an infinite variance that hinders the use of conventional techniques for convergence analysis that rely on second-order moments&#x2019; existence. To circumvent this challenge, we take a new route to establish the analysis of convergence rate, as well as generalization error, of the algorithm. We also show that the training algorithm can be run in tandem with the momentum scheme to accelerate the convergence. Our analyses reveal a two-sided effect of the interference on the overall training procedure. On the negative side, heavy tail noise slows down the convergence rate of the model training: the heavier the tail in the distribution of interference, the slower the algorithm converges. On the positive side, heavy tail noise has the potential to increase the generalization power of the trained model: the heavier the tail, the better the model generalizes. This perhaps counterintuitive conclusion implies that the prevailing thinking on interference &#x2013; that it is only detrimental to the edge learning system &#x2013; is outdated and we shall seek new techniques that exploit, rather than simply mitigate, the interference for better machine learning in wireless networks.

Federated Learning With Non-IID Data in Wireless Networks

Federated learning provides a promising paradigm to enable network edge intelligence in the future sixth generation (6G) systems. However, due to the high dynamics of wireless circumstances and user behavior, the collected training data is non-independent and identically distributed (non-IID), which causes severe performance degradation of federated learning. To solve this problem, federated learning with non-IID data in wireless networks is studied in this paper. Firstly, based on the derived upper bound of expected weight divergence, a federated averaging scheme is proposed to reduce the distribution divergence of non-IID data. Secondly, to further harmonize the distribution divergence, data sharing is associated with federated learning in wireless networks, and a joint optimization algorithm is designed to keep a sophisticated balance between the model accuracy and the cost. Finally, the simulation results based on a common-used image data set are provided to evaluate the performance of our proposed schemes, which can achieve significant performance gains with a small price of latency and energy consumption.

Guest Editorial Special Issue on Distributed Learning Over Wireless Edge Networks—Part II

This is Part II of a double-part special issue on distributed learning over wireless edge networks. This two-part special issue features papers dealing with two main research challenges: optimization of wireless network performance for efficient implementation of distributed learning in wireless networks, and distributed learning for solving communication problems and optimizing network performance. The accepted papers in this special issue have been grouped into three topics: 1) network optimization for federated learning (FL), 2) network optimization for other distributed learning methods, and 3) distributed reinforcement learning (RL) for wireless network optimization. In Part I (vol. 39, no. 12, Dec. 2021), the focus is on the first cluster (network optimization for FL). The focus of Part II is on the second and third clusters (network optimization for other distributed learning methods and RL for wireless network optimization). The readers are referred to Part I for an overview paper [A1] by the team of guest editors where a comprehensive study of how distributed learning can be efficiently deployed over wireless edge networks is provided. The contributions made by the papers in Part II are summarized as follows.

Application of Deep Learning in Wireless Networks for Channel Estimation: A Survey

The quality of channel estimation (CE) is critical to the performence of wireless communication systems. In addition to the conventional model based CE methods appeared in the past few decades, deep learning (DL) based CE methods, which is introduced to learn the statistical characteristics of wireless channels, has emerged as a promising methods in recently years. In this survey, different DL models applied to solve CE problems are discussed firstly, then DL-based CE problems in different wireless communication systems are investigated.

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

The next-generation of wireless networks will enable many machine learning (ML) tools and applications to efficiently analyze various types of data collected by edge devices for inference, autonomy, and decision making purposes. However, due to resource constraints, delay limitations, and privacy challenges, edge devices cannot offload their entire collected datasets to a cloud server for centrally training their ML models or inference purposes. To overcome these challenges, distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges, thus reducing the communication overhead and latency as well as improving data privacy. However, deploying distributed learning over wireless networks faces several challenges including the uncertain wireless environment (e.g., dynamic channel and interference), limited wireless resources (e.g., transmit power and radio spectrum), and hardware resources (e.g., computational power). This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks. We present a detailed overview of several emerging distributed learning paradigms, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning. For each learning framework, we first introduce the motivation for deploying it over wireless networks. Then, we present a detailed literature review on the use of communication techniques for its efficient deployment. We then introduce an illustrative example to show how to optimize wireless networks to improve its performance. Finally, we introduce future research opportunities. In a nutshell, this paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks.

Guest Editorial Special Issue on Distributed Learning Over Wireless Edge Networks—Part I

Analyzing massive amounts of data using complex machine learning models requires significant computational resources. The conventional approach to such problems involves centralizing training data and inference processes in the cloud, i.e., in data centers. However, with the proliferation of mobile devices and increasing application of the Internet-of-Things (IoT) paradigm, very large amounts of data are collected at the edges of wireless networks, and due to privacy constraints and limited communication resources, it is undesirable or impractical to upload this data from mobile devices to the cloud for centralized learning. This problem can be solved by distributed learning at the network edge, by which edge devices collaboratively train a shared learning model using real-time mobile data. The avoidance of raw-data uploading not only helps to preserve privacy but may also alleviate network-traffic congestion and minimize latency. With that said, distributed training still requires a substantial amount of information exchange between devices and edge servers over wireless links. In the process, wireless impairments such as noise, interference, and imperfect knowledge of channel states can significantly slow down distributed learning (e.g., convergence speed) and degrades its performance (e.g., learning accuracy). This makes it crucial to optimize wireless network performance so as to support the efficient deployment of distributed learning algorithms. On the other hand, distributed learning algorithms provide a powerful tool-set for solving complex problems in wireless communication and networking. One important framework, called federated learning (FL), enables users to collaboratively learn a shared model while helping to preserve local data privacy. The application of FL can endow edge devices with capabilities of user behavior prediction, user identification, and wireless environment analysis. As another example, distributed reinforcement learning is capable of leveraging distributed computation power and data to solve complex optimization and control problems that arise in various use cases, such as network control, user clustering, resource management, and interference alignment. To cover this paradigm of distributed learning over wireless networks, this two-part Special Issue features papers dealing with two main research challenges: a) optimization of wireless network performance for efficient implementation of distributed learning in wireless networks, and b) distributed learning for solving communication problems and optimizing network performance.