Personalized Meta-Federated Learning for IoT-Enabled Health Monitoring.

In this article, the problem of training federated learning (FL) algorithms over a realistic wireless network is studied. In the considered model, wireless users execute an FL algorithm while training their local FL models using their own data and transmitting the trained local FL models to a base station (BS) that generates a global FL model and sends the model back to the users. Since all training parameters are transmitted over wireless links, the quality of training is affected by wireless factors such as packet errors and the availability of wireless resources. Meanwhile, due to the limited wireless bandwidth, the BS needs to select an appropriate subset of users to execute the FL algorithm so as to build a global FL model accurately. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize an FL loss function that captures the performance of the FL algorithm. To seek the solution, a closed-form expression for the expected convergence rate of the FL algorithm is first derived to quantify the impact of wireless factors on FL. Then, based on the expected convergence rate of the FL algorithm, the optimal transmit power for each user is derived, under a given user selection and uplink resource block (RB) allocation scheme. Finally, the user selection and uplink RB allocation is optimized so as to minimize the FL loss function. Simulation results show that the proposed joint federated learning and communication framework can improve the identification accuracy by up to 1.4%, 3.5% and 4.1%, respectively, compared to: 1) An optimal user selection algorithm with random resource allocation, 2) a standard FL algorithm with random user selection and resource allocation, and 3) a wireless optimization algorithm that minimizes the sum packet error rates of all users while being agnostic to the FL parameters.

In this paper, the problem of training federated learning (FL) algorithms over a realistic wireless network is studied. In particular, in the considered model, wireless users perform an FL algorithm that trains their local FL models using their own data and send the trained local FL models to a base station (BS) that will generate a global FL model and send it back to the users. Since all training parameters are transmitted over wireless links, the quality of the training will be affected by wireless factors such as packet errors and availability of wireless resources. Meanwhile, due to the limited wireless bandwidth, the BS must select an appropriate subset of users to execute the FL learning algorithm so as to build a global FL model accurately. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize an FL loss function that captures the performance of the FL algorithm. To address this problem, a closed-form expression for the expected convergence rate of the FL algorithm is first derived to quantify the impact of wireless factors on FL. Then, based on the expected convergence rate of the FL algorithm, the optimal transmit power for each user is derived, under a given user selection and uplink resource block (RB) allocation scheme. Finally, the user selection and uplink RB allocation is optimized so as to minimize the FL loss function. Simulation results show that the proposed joint federated learning and communication framework can reduce the FL loss function value by up to 10% and 16%, respectively, compared to 1) an optimal user selection algorithm with random resource allocation and 2) a random user selection and resource allocation algorithm.

In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, with the considered model, wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected and transmit their local FL model parameters to the BS at each learning step. Meanwhile, since each user has unique training data samples and the BS must wait to receive all users' local FL models to generate the global FL model, the FL performance and convergence time will be significantly affected by the user selection scheme. In consequence, it is necessary to design an appropriate user selection scheme that enables all users to execute an FL scheme and efficiently train it. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time while optimizing the FL performance. To address this problem, a probabilistic user selection scheme is proposed using which the BS will connect to the users, whose local FL models have large effects on its global FL model, with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission, which enables the BS to include more users' local FL models to generate the global FL model so as to improve the FL convergence speed and performance. Simulation results show that the proposed ANN-based FL scheme can reduce the FL convergence time by up to 53.8%, compared to a standard FL algorithm.

In this paper, the convergence time of federated learning (FL), when deployed over a realistic wireless network, is studied. In particular, a wireless network is considered in which wireless users transmit their local FL models (trained using their locally collected data) to a base station (BS). The BS, acting as a central controller, generates a global FL model using the received local FL models and broadcasts it back to all users. Due to the limited number of resource blocks (RBs) in a wireless network, only a subset of users can be selected to transmit their local FL model parameters to the BS at each learning step. Moreover, since each user has unique training data samples, the BS prefers to include all local user FL models to generate a converged global FL model. Hence, the FL training loss and convergence time will be significantly affected by the user selection scheme. Therefore, it is necessary to design an appropriate user selection scheme that can select the users who can contribute toward improving the FL convergence speed more frequently. This joint learning, wireless resource allocation, and user selection problem is formulated as an optimization problem whose goal is to minimize the FL convergence time and the FL training loss. To solve this problem, a probabilistic user selection scheme is proposed such that the BS is connected to the users whose local FL models have significant effects on the global FL model with high probabilities. Given the user selection policy, the uplink RB allocation can be determined. To further reduce the FL convergence time, artificial neural networks (ANNs) are used to estimate the local FL models of the users that are not allocated any RBs for local FL model transmission at each given learning step, which enables the BS to improve the global model, the FL convergence speed, and the training loss. Simulation results show that the proposed approach can reduce the FL convergence time by up to 56% and improve the accuracy of identifying handwritten digits by up to 3%, compared to a standard FL algorithm.

In federated learning (FL), a server determines a global learning model by aggregating the local learning models of clients, and the determined global model is broadcast to all the clients. However, the global learning model can significantly deteriorate if a Byzantine attacker transmits malicious learning models trained with incorrectly labeled data. We propose a Byzantine-robust FL algorithm that, by employing a consensus confirmation method, can reduce the success probability of Byzantine attacks. After aggregating the local models from clients, the proposed FL server validates the global model candidate by sending the global model candidate to a set of randomly selected FL clients and asking them to perform local validation with their local data. If most of the validation is positive, the global model is confirmed and broadcast to all the clients. We compare the performance of the proposed FL against Byzantine attacks with that of existing FL algorithms analytically and empirically.

Federated Learning (FL), a promising privacy-preserving distributed learning paradigm, has been extensively applied in urban environmental prediction tasks of Mobile Edge Computing (MEC) by training a global machine learning model without data sharing. However, it is hard for the shared global model to be well generalized among local edge servers, due to the statistical data heterogeneity, especially in real-world urban environmental data. Besides, the existing FL approaches may result in excessive communication and computation overhead due to the frequent transmission and aggregation of model parameters between massive edge servers and remote cloud servers. To address the above issues, we propose HPFL-CN, a novel communication-efficient Hierarchical Personalized Federated edge Learning framework via Complex Network feature clustering, aiming to cluster edge servers with similar environmental data distributions and then high-efficiently train personalized models for each cluster via hierarchical architecture. Specifically, HPFL-CN introduces Privacy-preserving Feature Clustering (PFC) to extract privacy-preserving low-dimensional feature representations of each edge server via mapping the environmental data to different complex network domains for clustering similar edge servers accurately. According to the clustering results of PFC, HPFL-CN further introduces an edge-mediator-cloud architecture for hierarchical model aggregation by Effective Hierarchical Scheduling (EHS), in which every mediator coordinates the training of edge servers within each cluster and periodically uploads model to cloud server for global model aggregation. Meanwhile, each mediator server would find a trade-off between cloud and edge models to realize personalization within clusters. Our extensive experiments on real-world datasets demonstrate the effectiveness and generalization of HPFL-CN, which outperforms other state-of-the-art FL methods regarding personalization performance and communication efficiency.

Wireless federated learning (FL) is a collaborative machine learning (ML) framework in which wireless client-devices independently train their ML models and send the locally trained models to the FL server for aggregation. In this paper, we consider the coexistence of privacy-sensitive client-devices and privacy-insensitive yet computing-resource constrained client-devices, and propose an FL framework with a hybrid centralized training and local training. Specifically, the privacy-sensitive client-devices perform local ML model training and send their local models to the FL server. Each privacy-insensitive client-device can have two options, i.e., (i) conducting a local training and then sending its local model to the FL server, and (ii) directly sending its local data to the FL server for the centralized training. The FL server, after collecting the data from the privacy-insensitive client-devices (which choose to upload the local data), conducts a centralized training with the received datasets. The global model is then generated by aggregating (i) the local models uploaded by the client-devices and (ii) the model trained by the FL server centrally. Focusing on this hybrid FL framework, we firstly analyze its convergence feature with respect to the client-devices' selections of local training or centralized training. We then formulate a joint optimization of client-devices' selections of the local training or centralized training, the FL training configuration (i.e., the number of the local iterations and the number of the global iterations), and the bandwidth allocations to the client-devices, with the objective of minimizing the overall latency for reaching the FL convergence. Despite the non-convexity of the joint optimization problem, we identify its layered structure and propose an efficient algorithm to solve it. Numerical results demonstrate the advantage of our proposed FL framework with the hybrid local and centralized training as well as our proposed algorithm, in comparison with several benchmark FL schemes and algorithms.

In this paper, the problem of training federated learning (FL) algorithms over a wireless network with mobile users is studied. In the considered model, several mobile users and a network base station (BS) cooperatively perform an FL algorithm. In particular, the wireless mobile users train their local FL models and send the trained local FL model parameters to the BS. The BS will then integrate the received local FL models to generate a global FL model and send it back to all users. Due to the limited training time at each iteration, the number of users that can transmit their local FL models to the BS will be affected by changes in the users’ locations and wireless channels. In this paper, this joint learning, user selection, and wireless resource allocation problem is formulated as an optimization problem whose goal is to minimize the FL loss function, which captures the FL performance, while meeting the transmission delay requirement. To solve this problem, a closed-form expression for the expected convergence rate of the FL algorithm is first derived to quantify the impact of the users’ mobility and wireless factors on FL. Then, based on the expected FL convergence rate, the user selection and uplink resource allocation is optimized at each FL iteration so as to minimize the FL loss function while satisfying the FL parameter transmission delay requirement. Simulation results show that the proposed approach can reduce the FL loss function value by up to 20% compared to a standard FL algorithm.

Federated learning (FL) offers collaborative machine learning across decentralized devices while safeguarding data privacy. However, data security and privacy remain key concerns. This paper introduces "Secure Federated Learning with a Homomorphic Encryption Model," addressing these challenges by integrating homomorphic encryption into FL. The model starts by initializing a global machine learning model and generating a homomorphic encryption key pair, with the public key shared among FL participants. Using this public key, participants then collect, preprocess, and encrypt their local data. During FL Training Rounds, participants decrypt the global model, compute local updates on encrypted data, encrypt these updates, and securely send them to the aggregator. The aggregator homomorphic ally combines updates without revealing participant data, forwarding the encrypted aggregated update to the global model owner. The Global Model Update ensures the owner decrypts the aggregated update using the private key, updates the global model, encrypts it with the public key, and shares the encrypted global model with FL participants. With optional model evaluation, training can iterate for several rounds or until convergence. This model offers a robust solution to Florida data privacy and security issues, with versatile applications across domains. This paper presents core model components, advantages, and potential domain-specific implementations while making significant strides in addressing FL's data privacy concerns.

Federated Learning (FL), an emerging distributed machine learning framework that enables each client to collaboratively train a global model by sharing local knowledge without disclosing local private data, is vulnerable to backdoor model poisoning attacks. By compromising some users, the attacker manipulates their local training process, and uploads malicious gradient updates to poison the global model, resulting in the poisoned global model behaving abnormally on the sub-tasks specified by the malicious user. Prior research has proposed various strategies to mitigate backdoor attacks. However, existing FL backdoor defense methods affect the fairness of the FL system, while fair FL performance may not be robust. Motivated by these concerns, in this paper, we propose Self-Awareness Revision (SARS), a personalized FL framework designed to resist backdoor attacks and ensure the fairness of the FL system. SARS consists of two key modules: adaptation feature extraction and knowledge mapping. In the adaptation feature extraction module, benign users can adaptively extract clean global knowledge with self-awareness and self-revision of the backdoor knowledge transferred from the global model. Based on the previous module, users can effectively ensure the correct mapping of clean sample features and labels. Through extensive experimental results, SARS can defend against backdoor attacks and improve the fairness of the FL system by comparing several state-of-the-art FL backdoor defenses or fair FL methods, including FedAvg, Ditto, WeakDP, FoolsGold, and FLAME.

Federated learning (FL) is an evolving machine learning technique that allows collaborative model training without sharing the original data among participants. In real-world scenarios, data residing at multiple clients are often heterogeneous in terms of different resolutions, magnifications, scanners, or imaging protocols, and thus challenging for global FL model convergence in collaborative training. Most of the existing FL methods consider data heterogeneity within one domain by assuming same data variation in each client site. In this paper, we consider data heterogeneity in FL with different domains of heterogeneous data by raising the problems of domain-shift, class-imbalance, and missing data. We propose a method, multi-domain FL as a solution to heterogeneous training data from multiple domains by training robust vision transformer model. We use two loss functions, one for correctly predicting class labels and other for encouraging similarity and dissimilarity over latent features, to optimize the global FL model. We perform various experiments using different convolution-based networks and non-convolutional Transformer architectures on multi-domain datasets. We evaluate the proposed approach on benchmark datasets and compare with the existing FL methods. Our results show the superiority of the proposed approach which performs better in term of robust FL global model than the exiting methods.

The recently emerging federated learning (FL) exploits massive data stored at multiple user nodes to train a global optimal learning model without leaking the privacy of user data. However, it is still inadequate to learn the global model safely at the centralized aggregator, which is an essential part for the traditional FL architecture. Specifically, when using FL in radio access networks to enable edge intelligence, it is difficult for a central server, which belongs to a third party, to guarantee its credibility. Moreover, because the central server may cause a single point of failure, its reliability is also difficult to guarantee. Besides, a malicious participating node of FL may send ill parameters for model aggregation. In this article, we develop a blockchain assisted federated learning (BC-FL) framework, with aim to overcome the single point of failure caused by central server. Meanwhile, we propose to use blockchain to implement auditing of individual involved nodes to ensure the reliability of learning process. To avoid privacy leakage during the audit process to the greatest extent, we design a matching audit mechanism to realize efficient random matching audit process. A cryptocurrency free delegated byzantine fault tolerant (CF-DBFT) consensus mechanism is also designed to realize the low-latency distributed consensus of all nodes in the FL proces. We apply the proposed BC-FL framework to resolve the computing resource allocation problem at the edger servers in MEC network. Simulation results demonstrate the effectiveness and performance superiority of the proposed BC-FL framework. Compared with legacy FL algorithm, the serving time of MEC servers and utilization of computing resource are increased by 35% and 48% respectively under our proposed BC-FL algorithm.

In federated learning (FL), clients may have diverse objectives, and merging all clients' knowledge into one global model will cause negative transfer to local performance. Thus, clustered FL is proposed to group similar clients into clusters and maintain several global models. In the literature, centralized clustered FL algorithms require the assumption of the number of clusters and hence are not effective enough to explore the latent relationships among clients. In this paper, without assuming the number of clusters, we propose a peer-to-peer (P2P) FL algorithm named PANM. In PANM, clients communicate with peers to adaptively form an effective clustered topology. Specifically, we present two novel metrics for measuring client similarity and a two-stage neighbor matching algorithm based Monte Carlo method and Expectation Maximization under the Gaussian Mixture Model assumption. We have conducted theoretical analyses of PANM on the probability of neighbor estimation and the error gap to the clustered optimum. We have also implemented extensive experiments under both synthetic and real-world clustered heterogeneity. Theoretical analysis and empirical experiments show that the proposed algorithm is superior to the P2P FL counterparts, and it achieves better performance than the centralized cluster FL method. PANM is effective even under extremely low communication budgets.

Federated learning: A cutting-edge survey of the latest advancements and applications

This paper intends to optimize the overall implementing process of federated learning (FL) in practical edge computing systems. First, we present a general FL algorithm, namely GenQSGD+, whose parameters include the numbers of global and local iterations, mini-batch size, and step size sequence. Then, we analyze the convergence of GenQSGD+ with arbitrary algorithm parameters. Next, we optimize all the algorithm parameters of GenQSGD+ to minimize the energy cost under the constraints on the time cost, convergence error, and step size sequence. The resulting optimization problem is challenging due to its non-convexity and the presence of a dimension-varying vector variable and non-differentiable constraint functions. We transform the complicated problem into a more tractable nonconvex problem using the structural properties of the original problem and propose an iterative algorithm using general inner approximation (GIA) and complementary geometric programming (CGP) to obtain a KKT point. Finally, we numerically demonstrate remarkable gains of optimization-based GenQSGD+ over typical FL algorithms and the advancement of the proposed optimization framework for federated edge learning.