Federated Learning Model Research Articles

A Novel Federated Learning Framework Based on Conditional Generative Adversarial Networks for Privacy Preserving in 6G

With the rapid development of 6G networks, data transmission speed has significantly increased, making data privacy protection issues even more crucial. The federated learning (FL) is a distributed machine learning framework with privacy protection and secure encryption technology, aimed at enabling dispersed participants to collaborate on model training without disclosing private data to other participants. Nonetheless, recent research indicates that the exchange of shared gradients may lead to information disclosure, and thus FL still needs to address privacy concerns. Additionally, FL relies on a large number of diverse training data to forge efficient models, but in reality, the training data available to clients are limited, and data imbalance issues lead to over fitting in existing federated learning models. To alleviate these issues, we introduce a Novel Federated Learning Framework based on Conditional Generative Adversarial Networks (NFL-CGAN). NFL-CGAN divides the local networks of each client into private and public modules. The private module contains an extractor and a discriminator to protect privacy by retaining them locally. Conversely, the public module is shared with the server to aggregate the shared knowledge of clients, thereby improving the performance of each client local network. Comprehensive experimental analyses demonstrate that NFL-CGAN surpasses traditional FL baseline methods in data classification, showcasing its superior efficacy. Moreover, privacy assessments also verified robust and reliable privacy protection capabilities of NFL-CGAN.

Алгоритм и анализ сходимости модели федеративного обучения, использующейся в полете роя дронов

Представлен анализ возможности применения сравнительно новой и хорошо зарекомендовавшей себя методики машинного обучения для распределенной си-стемы сбора и обработки информации – федеративного обучения моделей, использую-щейся для выполнения полета роем дронов, как группы агентов с точки зрения передачи накопленных данных и обучения модели искусственного интеллекта. Описана схема ра-боты федеративного обучения для группы беспилотных летательных аппаратов (БПЛА). Особое внимание уделено понятию конвергенции (сходимости) в машинном обучении. The purpose of this study is to review and analyze the application of a relative-ly new and well–proven machine learning methodology for a distributed information collection and processing system – federated model training used to fly a swarm of drones as a group of agents in terms of transferring accumulated data and training an artificial intelligence model. The scheme of federal training for a group of unmanned aerial vehicles (UAV) is described. Special attention is paid to the concept of convergence in machine learning.

Towards cross-silo federated learning for corporate organizations

Digital media companies rely on machine learning models to target their content toward their audience’s interests. Machine learning models usually rely on the amount and quality of training data. While today, data is abundant, it is typically stored in data silos and cannot be shared between companies or publishers due to data protection and user privacy. Federated Learning (FL) is a distributed machine learning approach that is rapidly gaining popularity and enables collaboratively training machine learning models on a large corpus of decentralized data. Prior research on FL mainly focuses on an FL setup containing millions of clients. For example, a client may be a single user’s mobile device with data. However, we note that, in many scenarios, corporate organizations such as news media companies that have available data from multiple sets of users could also benefit from FL. In this work, we aim to focus on building FL models where multiple corporate organizations like news media companies or banks participate in the training process of FL to collaboratively train federated models. We used federated learning to train models for a set of corporate stakeholders and applied FL for two tasks: a classification task and a ranking task. For the classification task, we designed a tree-based federated random forest algorithm and a neural network-based federated algorithm. For the ranking task, we designed a federated neural ranking model for news article recommendations. Our experimental results demonstrate that corporate companies by participating in FL can achieve benefits in improving the model performance in terms of accuracy for classification tasks and in terms of ranking for recommendation tasks. Furthermore, we designed and developed a simple framework for a small number of stakeholders to train federated models.

A lightweight and personalized edge federated learning model

As a new distributed machine learning paradigm, federated learning has gained increasing attention in the industry and research community. However, federated learning is challenging to implement on edge devices with limited resources and heterogeneous data. This study aims to realize a lightweight and personalized model through pruning and masking with insufficient resources and heterogeneous data. Particularly, the server first downloads the subnetwork to the client according to the mask, and client prunes the subnetwork with the alternating direction method of multipliers (ADMM), so as to remove the unimportant parameters and reduce the cost of training and communication. At the same time, mask is used to mark the pruning condition of the model. Then, the unpruned parts and masks of local models are transmitted to the server for aggregation. The experimental results showed that the accuracy of the proposed model was improved by 9.36%, and the communication cost was reduced by 1.45 times compared with state-of-the-art models. Last but not least, we deploy flower identification models in Android Studio to illustrate the practicality of the proposed method.

Self-Attention Mechanism-Based Federated Learning Model for Cross Context Recommendation System

Self-Attention Mechanism-Based Federated Learning Model for Cross Context Recommendation System

Robustly federated learning model for identifying high-risk patients with postoperative gastric cancer recurrence

The prediction of patient disease risk via computed tomography (CT) images and artificial intelligence techniques shows great potential. However, training a robust artificial intelligence model typically requires large-scale data support. In practice, the collection of medical data faces obstacles related to privacy protection. Therefore, the present study aims to establish a robust federated learning model to overcome the data island problem and identify high-risk patients with postoperative gastric cancer recurrence in a multicentre, cross-institution setting, thereby enabling robust treatment with significant value. In the present study, we collect data from four independent medical institutions for experimentation. The robust federated learning model algorithm yields area under the receiver operating characteristic curve (AUC) values of 0.710, 0.798, 0.809, and 0.869 across four data centres. Additionally, the effectiveness of the algorithm is evaluated, and both adaptive and common features are identified through analysis.

Federated learning model for credit card fraud detection with data balancing techniques

In recent years, credit card transaction fraud has resulted in massive losses for both consumers and banks. Subsequently, both cardholders and banks need a strong fraud detection system to reduce cardholder losses. Credit card fraud detection (CCFD) is an important method of fraud prevention. However, there are many challenges in developing an ideal fraud detection system for banks. First off, due to data security and privacy concerns, various banks and other financial institutions are typically not permitted to exchange their transaction datasets. These issues make traditional systems find it difficult to learn and detect fraud depictions. Therefore, this paper proposes federated learning for CCFD over different frameworks (TensorFlow federated, PyTorch). Second, there is a significant imbalance in credit card transactions across all banks, with a small percentage of fraudulent transactions outweighing the majority of valid ones. In order to demonstrate the urgent need for a comprehensive investigation of class imbalance management techniques to develop a powerful model to identify fraudulent transactions, the dataset must be balanced. In order to address the issue of class imbalance, this study also seeks to give a comparative analysis of several individual and hybrid resampling techniques. In several experimental studies, the effectiveness of various resampling techniques in combination with classification approaches has been compared. In this study, it is found that the hybrid resampling methods perform well for machine learning classification models compared to deep learning classification models. The experimental results show that the best accuracy for the Random Forest (RF); Logistic Regression; K-Nearest Neighbors (KNN); Decision Tree (DT), and Gaussian Naive Bayes (NB) classifiers are 99,99%; 94,61%; 99.96%; 99,98%, and 91,47%, respectively. The comparative results show that the RF outperforms with high performance parameters (accuracy, recall, precision and f score) better than NB; RF; DT and KNN. RF achieve the minimum loss values with all resampling techniques, and the results, when utilizing the proposed models on the entire skewed dataset, achieved preferable outcomes to the unbalanced dataset. Furthermore, the PyTorch framework achieves higher prediction accuracy for the federated learning model than the TensorFlow federated framework but with more computational time.

On differential privacy for federated learning in wireless systems with multiple base stations

AbstractIn this work, we consider a federated learning model in a wireless system with multiple base stations and inter‐cell interference. We apply a differentially private scheme to transmit information from users to their corresponding base station during the learning phase. We show the convergence behavior of the learning process by deriving an upper bound on its optimality gap. Furthermore, we define an optimization problem to reduce this upper bound and the total privacy leakage. To find the locally optimal solutions of this problem, we first propose an algorithm that schedules the resource blocks and users. We then extend this scheme to reduce the total privacy leakage by optimizing the differential privacy artificial noise. We apply the solutions of these two procedures as parameters of a federated learning system where each user is equipped with a classifier and communication cells have mostly fewer resource blocks than numbers of users. The simulation results show that our proposed scheduler improves the average accuracy of the predictions compared with a random scheduler. In particular, the results show an improvement of over 6%. Furthermore, its extended version with noise optimizer significantly reduces the amount of privacy leakage.

FedICU: a federated learning model for reducing the medication prescription errors in intensive care units

FedICU: a federated learning model for reducing the medication prescription errors in intensive care units

Proactive auto-scaling technique for web applications in container-based edge computing using federated learning model

Edge computing has emerged as an attractive alternative to traditional cloud computing by utilizing processing, network, and storage resources close to end devices, such as Internet of Things (IoT) sensors. Edge computing is still in its infancy, and resource provisioning and service scheduling remain research concerns. Kubernetes is a container orchestration tool in distributed environments. Proactive auto-scaling techniques in Kubernetes improve utilization by allocating resources based on future workload prediction. However, prediction models run on central cloud nodes, necessitating data transfer between edge and cloud nodes, which increases latency and response time. We present FedAvg-BiGRU, a proactive auto-scaling method in edge computing based on FedAvg and multi-step prediction by a Bidirectional Gated Recurrent Unit (BiGRU). FedAvg is a technique for training machine learning models in a Federated Learning (FL) model. FL reduces network traffic by exchanging only model updates rather than raw data, relieving learning models of the need to store data on a centralized cloud server. In addition, a technique for determining the number of Kubernetes pods based on the Cool Down Time (CDT) concept has been developed, preventing contradictory scaling actions. To our knowledge, our work is the first to employ FL for proactive auto-scaling in cloud and edge computing. The results demonstrate that the FedAvg-BiGRU method has a slightly higher prediction error than the load centralized processing mode, although this difference is not statistically significant. At the same time, it reduces the amount of data transmission between the edge nodes and the cloud server.

Asynchronous Robust Aggregation Method with Privacy Protection for IoV Federated Learning

Due to the wide connection range and open communication environment of internet of vehicle (IoV) devices, they are susceptible to Byzantine attacks and privacy inference attacks, resulting in security and privacy issues in IoV federated learning. Therefore, there is an urgent need to study IoV federated learning methods with privacy protection. However, the heterogeneity and resource limitations of IoV devices pose significant challenges to the aggregation of federated learning model parameters. Therefore, this paper proposes an asynchronous robust aggregation method with privacy protection for federated learning in IoVs. Firstly, we design an asynchronous grouping robust aggregation algorithm based on delay perception, combines intra-group truth estimation with inter-group delay aggregation, and alleviates the impact of stragglers and Byzantine attackers. Then, we design a communication-efficient and security enhanced aggregation protocol based on homomorphic encryption, to achieve asynchronous group robust aggregation while protecting data privacy and reducing communication overhead. Finally, the simulation results indicate that the proposed scheme could achieve a maximum improvement of 41.6% in model accuracy compared to the baseline, which effectively enhances the training efficiency of the model while providing resistance to Byzantine attacks and privacy inference attacks.

Federated Discriminative Representation Learning for Image Classification.

Acquiring big-size datasets to raise the performance of deep models has become one of the most critical problems in representation learning (RL) techniques, which is the core potential of the emerging paradigm of federated learning (FL). However, most current FL models concentrate on seeking an identical model for isolated clients and thus fail to make full use of the data specificity between clients. To enhance the classification performance of each client, this study introduces the FDRL, a federated discriminative RL model, by partitioning the data features of each client into a global subspace and a local subspace. More specifically, FDRL learns the global representation for federated communication between those isolated clients, which is to capture common features from all protected datasets via model sharing, and local representations for personalization in each client, which is to preserve specific features of clients via model differentiating. Toward this goal, FDRL in each client trains a shared submodel for federated communication and, meanwhile, a not-shared submodel for locality preservation, in which the two models partition client-feature space by maximizing their differences, followed by a linear model fed with combined features for image classification. The proposed model is implemented with neural networks and optimized in an iterative manner between the server of computing the global model and the clients of learning the local classifiers. Thanks to the powerful capability of local feature preservation, FDRL leads to more discriminative data representations than the compared FL models. Experimental results on public datasets demonstrate that our FDRL benefits from the subspace partition and achieves better performance on federated image classification than the state-of-the-art FL models.

FLSys: Toward an Open Ecosystem for Federated Learning Mobile Apps

This article presents the design, implementation, and evaluation of <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FLSys</b> , a mobile-cloud federated learning (FL) system, which can be a key component for an open ecosystem of FL models and apps. FLSys is designed to work on smart phones with mobile sensing data. It balances model performance with resource consumption, tolerates communication failures, and achieves scalability. In FLSys, different DL models with different FL aggregation methods can be trained and accessed concurrently by different apps. Furthermore, FLSys provides advanced privacy preserving mechanisms and a common API for third-party app developers to access FL models. FLSys adopts a modular design and is implemented in Android and AWS cloud. We co-designed FLSys with a human activity recognition (HAR) model. HAR sensing data was collected in the wild from 100+ college students during a 4-month period. We implemented HAR-Wild, a CNN model tailored to mobile devices, with a data augmentation mechanism to mitigate the problem of non-Independent and Identically Distributed data. A sentiment analysis model is also used to demonstrate that FLSys effectively supports concurrent models. This article reports our experience and lessons learned from conducting extensive experiments using simulations, Android/Linux emulations, and Android phones that demonstrate FLSys achieves good model utility and practical system performance.

EuProGigant: A decentralized Federated Learning Approach based on Compute-to-Data and Gaia-X

Machine learning normally requires a considerable amount of data for model training, limiting the field of usage especially for small manufacturing companies due to their lack of machine data. Federated learning provides an opportunity to enlarge the data basis for model training without the need to directly share the machine data with other participants, addressing concerns regarding to privacy, intellectual property and potential reverse engineering of proprietary process information through competitors. Previous research focused mainly on federated learning models, mostly managed and orchestrated by some kind of centralized authority. The presented approach shows a more decentralized, self-sovereign concept of federated learning for the manufacturing industry, expanding its applicability to a broader range of participants. It combines existing solutions for machine learning and Compute-to-Data by Ocean Protocol with the concept of dataspaces as defined by Gaia-X. The methodology is demonstrated through an use case derived from industry demands involving several CNC milling machines.

FLMatchQA: a recursive neural network-based question answering with customized federated learning model.

More sophisticated data access is possible with artificial intelligence (AI) techniques such as question answering (QA), but regulations and privacy concerns have limited their use. Federated learning (FL) deals with these problems, and QA is a viable substitute for AI. The utilization of hierarchical FL systems is examined in this research, along with an ideal method for developing client-specific adapters. The User Modified Hierarchical Federated Learning Model (UMHFLM) selects local models for users' tasks. The article suggests employing recurrent neural network (RNN) as a neural network (NN) technique for learning automatically and categorizing questions based on natural language into the appropriate templates. Together, local and global models are developed, with the worldwide model influencing local models, which are, in turn, combined for personalization. The method is applied in natural language processing pipelines for phrase matching employing template exact match, segmentation, and answer type detection. The (SQuAD-2.0), a DL-based QA method for acquiring knowledge of complicated SPARQL test questions and their accompanying SPARQL queries across the DBpedia dataset, was used to train and assess the model. The SQuAD2.0 datasets evaluate the model, which identifies 38 distinct templates. Considering the top two most likely templates, the RNN model achieves template classification accuracy of 92.8% and 61.8% on the SQuAD2.0 and QALD-7 datasets. A study on data scarcity among participants found that FL Match outperformed BERT significantly. A MAP margin of 2.60% exists between BERT and FL Match at a 100% data ratio and an MRR margin of 7.23% at a 20% data ratio.

Lightweight Federated Transfer Learning for Plant Leaf Disease Detection and Classification across Multiclient Cross-Silo Datasets

Plant leaves and crops play a crucial role as a primary food source globally, making significant contributions to dietary iron intake (9%) and energy consumption (23%) per capita in the Asian region. Bacterial, yeast, and other microbial diseases pose significant challenges to farmers as they detrimentally impact plant health and reduce crop productivity. The manual diagnosis of these diseases poses a considerable challenge, particularly in regions with a scarcity of professionals specializing in leaves and crop protection. Automating leaf disease detection and providing easily accessible decision-support resources are crucial for facilitating efficient leaf protection strategies and mitigating crop damage. Despite multiple classification methods for diagnosing leaf diseases, a secure and accurate approach that fulfills these requirements has not yet been identified. This paper presents an architectural framework called Lightweight Federated Transfer Learning (LFTL) that addresses the challenge of Leaf Disease Detection and Classification (LDDC) while ensuring data privacy limitations are upheld. A dataset consisting of leaf disease images has been compiled, characterized by an imbalance in the distribution of the diseases. The collection includes four conditions: bacterial decay, brown spot, blast, and tungro, corresponding image counts of 1695, 1551, 1711, and 1419, respectively. Following the preprocessing stage, the LFTL framework was tested using both Independent and Identically Distributed (IID) and non-IID datasets. The study commenced with an efficacy evaluation of the Convolutional Neural Network (CNN) and eight TL models in the LDDC. The framework’s performance was evaluated across different circumstances and compared to conventional and federated learning models. The study’s findings revealed that the LFTL framework outperformed traditional distributed deep-learning classifiers, thus demonstrating its efficacy in individual and multiple client scenarios.

FedLPPA: Learning Personalized Prompt and Aggregation for Federated Weakly-supervised Medical Image Segmentation.

Federated learning (FL) effectively mitigates the data silo challenge brought about by policies and privacy concerns, implicitly harnessing more data for deep model training. However, traditional centralized FL models grapple with diverse multi-center data, especially in the face of significant data heterogeneity, notably in medical contexts. In the realm of medical image segmentation, the growing imperative to curtail annotation costs has amplified the importance of weakly-supervised techniques which utilize sparse annotations such as points, scribbles, etc. A pragmatic FL paradigm shall accommodate diverse annotation formats across different sites, which research topic remains under-investigated. In such context, we propose a novel personalized FL framework with learnable prompt and aggregation (FedLPPA) to uniformly leverage heterogeneous weak supervision for medical image segmentation. In FedLPPA, a learnable universal knowledge prompt is maintained, complemented by multiple learnable personalized data distribution prompts and prompts representing the supervision sparsity. Integrated with sample features through a dual-attention mechanism, those prompts empower each local task decoder to adeptly adjust to both the local distribution and the supervision form. Concurrently, a dual-decoder strategy, predicated on prompt similarity, is introduced for enhancing the generation of pseudo-labels in weakly-supervised learning, alleviating overfitting and noise accumulation inherent to local data, while an adaptable aggregation method is employed to customize the task decoder on a parameter-wise basis. Extensive experiments on four distinct medical image segmentation tasks involving different modalities underscore the superiority of FedLPPA, with its efficacy closely parallels that of fully supervised centralized training. Our code and data will be available at https://github.com/llmir/FedLPPA.

FedFAIM: A Model Performance-based Fair Incentive Mechanism for Federated Learning

Federated Learning (FL) has emerged as a privacy-preserving distributed machine learning paradigm. To motivate data owners to contribute towards FL, research on FL incentive mechanisms is gaining great interest. Existing monetary incentive mechanisms generally share the same FL model with all participants regardless of their contributions. Such an assumption can be unfair towards participants who contributed more and promote undesirable free-riding, especially when the final model is of great utility value to participants. In this paper, we propose a Fairness-Aware Incentive Mechanism for federated learning (FedFAIM) to address such problem. It satisfies two types of fairness notion: 1) aggregation fairness, which determines aggregation results according to data quality; 2) reward fairness, which assigns each participant a unique model with performance reflecting his contribution. Aggregation fairness is achieved through efficient gradient aggregation which examines local gradient quality and aggregates them based on data quality. Reward fairness is achieved through an efficient Shapley value-based contribution assessment method and a novel reward allocation method based on reputation and distribution of local and global gradients. We further prove reward fairness is theoretically guaranteed. Extensive experiments show that FedFAIM provides stronger incentives than similar non-monetary FL incentive mechanisms while achieving a high level of fairness.

Massive MIMO for Serving Federated Learning and Non-Federated Learning Users

With its privacy preservation and communication efficiency, federated learning (FL) has emerged as a promising learning framework for beyond 5G wireless networks. It is anticipated that future wireless networks will jointly serve both FL and downlink non-FL user groups in the same time-frequency resource. While in the downlink of each FL iteration, both groups simultaneously receive data from the base station in the same time-frequency resource, the uplink of each FL iteration requires bidirectional communication to support uplink transmission for FL users and downlink transmission for non-FL users. To overcome this challenge, we present half-duplex (HD) and full-duplex (FD) communication schemes to serve both groups. More specifically, we adopt the massive multiple-input multiple-output technology and aim to maximize the minimum effective rate of non-FL users under a quality of service (QoS) latency constraint for FL users. Since the formulated problem is nonconvex, we propose a power control algorithm based on successive convex approximation to find a stationary solution. Numerical results show that the proposed solutions perform significantly better than the considered baselines schemes. Moreover, the FD-based scheme outperforms the HD-based counterpart in scenarios where the self-interference is small or moderate and/or the size of FL model updates is large.

FedACT: an adaptive chained training approach for federated learning in computing power networks

Federated Learning (FL) is a novel distributed machine learning methodology that addresses large-scale parallel computing challenges while safeguarding data security. However, the traditional FL model in communication scenarios, whether for uplink or downlink communications, may give rise to several network problems, such as bandwidth occupation, additional network latency, and bandwidth fragmentation. In this paper, we propose an adaptive chained training approach (FedACT) for FL in computing power networks. First, a Computation-driven Clustering Strategy (CCS) is designed. The server clusters clients by task processing delays to minimize waiting delays at the central server. Second, we propose a Genetic-Algorithm-based Sorting (GAS) method to optimize the order of clients participating in training. Finally, based on the table lookup and forwarding rules of the Segment Routing over IPv6 (SRv6) protocol, the sorting results of GAS are written into the SRv6 packet header, to control the order in which clients participate in model training. We conduct extensive experiments on two datasets of CIFAR-10 and MNIST, and the results demonstrate that the proposed algorithm offers improved accuracy, diminished communication costs, and reduced network delays.