Advances and Open Problems in Federated Learning

Nowadays, people's demand for water is growing as well as in public, commercial, and industrial sectors. However, the limited character for water resources presents a crucial obstacle to satisfying needs for continued human development. The control and provision of potable water are therefore the most vital challenges for the water supply system. It must use water resources in an efficient manner, and fulfill both quality and quantity demands. For this purpose, the current water supply system uses smart infrastructure that gathers, processes, stores, and delivers water from water sources to users. It is done in a very complex environment with ever-increasing demand, and often conflicting services to deliver. In this work, we developed a Federated Learning (FL) model, which is come in the form of a machine learning setting where clients collaboratively train under the orchestration of a central server while keeping the training data decentralized. The FL embodies the principles of focused data collection and minimization, it can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches. Implementing on the central server, the FL will iterate on the gateway's learning models using the federated averaging algorithm. Our approach's capacity is shown by addressing reel instances in a real-world issue in which the proposed method finds much better results and a significant improvement in performance compared to the standard approaches while achieving our goal and suggesting a new interesting direction for research.

The proliferation of Internet of Things (IoT) devices, generating massive amounts of heterogeneous distributed data, has pushed toward edge cloud computing as a promising paradigm to bring cloud capabilities closer to data sources. In many cases of practical interest, centralized Machine Learning (ML) approaches can hardly be employed due to high communication costs, low reliability, legal restrictions, and scalability issues. Therefore, Federated Learning (FL) is emerging as a promising distributed ML approach that enables models to be trained on remote devices using their local data. However, “traditional” FL solutions still present open technical challenges, such as single points of failure and lack of trustworthiness among participants. To address these open challenges, some researchers have started to propose leveraging blockchain technologies. However, the adoption of blockchain for FL at the edge is limited by several factors nowadays, such as long waiting times for transaction confirmation and high energy consumption.In this work, we conduct an original and comprehensive analysis of the key design challenges to address towards an efficient implementation of FL at the edge, and analyze how Distributed Ledger Technologies (DLTs) can be employed to overcome them. Then, we present a novel architecture that enables FL at the edge by leveraging the IOTA Tangle, a next-generation DLT whose data structure is a Directed Acyclic Graph (DAG), and the InterPlanetary File System (IPFS) to store and share partial models. Experimental results demonstrate the feasibility and efficiency of our proposed solution in real-world deployment scenarios.

BACKGROUND The quality of a machine learning model considerably relies on the size of the dataset, the development and widespread application of this method have often been hindered by confidentiality issues, particularly regarding data privacy. Predicting mortality is essential in clinical environments. When a patient is admitted, estimating their likelihood of mortality by the end of their intensive care unit (ICU) stay or within a designated time frame is a way to assess the severity of their condition. This information is crucial in managing treatment planning and resource allocation. However, individual hospitals typically have a limited amount of local data available to create a reliable model. The rise of federated learning as a novel privacy-preserving technology offers the potential for collaboratively creating models in a decentralized manner, eliminating the need to consolidate all datasets in a single location. Nonetheless, there is a scarce of clear and comprehensive evidence that compares the performance of federated learning with that of traditional centralized machine learning approaches, particularly considering healthcare implementation. OBJECTIVE This study aims to review the comparison of performances between federated learning (FL)-based and centralized machine learning (CML) models for mortality prediction in clinical settings. METHODS The electronic database search was conducted for English articles that developed federated-based learning model to predict mortality. Screening, data extraction, and risk of bias assessments were carried out by at least two independent reviewers. Meta-analyses of pooled area under the receiver operating curve (AUROC/AUC) values were examined for FL, CML, and LML. The risk of bias was assessed using critical appraisal and data extraction for systematic reviews of prediction modeling studies (CHARMS) and prediction model risk of bias assessment tool (PROBAST) guidelines RESULTS In total, 9 articles that were heterogeneous in framework design, scenario, and clinical context were included (n = 5 [55.6%] were observed in specific case; n = 3 [33.0%] were in ICU settings; and n = 2 [22.0%] in emergency department, urgent, or trauma center). Cohort datasets were utilized by all included studies. These studies universally indicated that performance of FL model outperforms LML model and closest to the CML model. The pooled AUC for FL and, CML (or LML) performances were 0.81 (95 % CI 0.76–0.85, I2 78.36 %) and 0.82 (95 % CI 0.77–0.86, I2 72.33 %), respectively. All included studies had either a low, high, or unclear risk of bias. CONCLUSIONS This systematic review and meta-analysis demonstrate that federated learning models outperform local machine learning approaches and are comparable to centralized models. However, efficiency may be compromised due to complexity, privacy preservation, and high computation and communication costs. CLINICALTRIAL PROSPERO International Prospective Register of Systematic Reviews CRD42024539245; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=539245

BackgroundThe rise of federated learning (FL) as a novel privacy-preserving technology offers the potential to create models collaboratively in a decentralized manner to address confidentiality issues, particularly regarding data privacy. However, there is a scarcity of clear and comprehensive evidence that compares the performance of FL with that of the established centralized machine learning (CML) in the clinical domain.ObjectiveThis study aimed to review the performance comparisons of FL-based and CML models for mortality prediction in clinical settings.MethodsExperimental studies comparing the performance of FL and CML in predicting mortality were selected. Articles were excluded if they did not compare FL with CML or only compared the effectiveness of different FL baseline models. Two independent reviewers performed the screening, data extraction, and risk of bias assessment. The IEEE Xplore, PubMed, ScienceDirect, and Web of Science databases were searched for articles published up to June 2024. The risk of bias was assessed using CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) and PROBAST (Prediction Model Risk of Bias Assessment Tool). Meta-analyses of the pooled area under the receiver operating curve (AUROC)/area under the curve (AUC) were performed for within-group comparisons (before and after federation).ResultsNine articles with heterogeneous framework design, scenario, and clinical context were included: 4 articles focused on specific case types; 3 articles were conducted in intensive care unit settings; and 2 articles in emergency departments, urgent centers, or trauma centers. Cohort datasets involving 1,412,973 participants were used in all of the included studies. These studies universally indicated that the predictive performance of FL models is comparable to that of CML. The pooled AUC for the FL and CML performances were 0.81 (95% CI 0.76‐0.85; I2=78.36%) and 0.82 (95% CI 0.77‐0.86; I2=72.33%), respectively. The Higgins I2 test indicated high heterogeneity between the included studies (I2≥50%). In total, 4 out of 9 (44%) of the developed models were identified as having a high risk of bias.ConclusionsThis systematic review and meta-analysis demonstrate that FL can achieve similar performance to CML while conquering privacy risks in predicting mortality across various settings. Owing to the small number of studies and a moderate proportion of the high risk of bias, the effect estimates might be imprecise.

The Internet of Things (IoT) will be ripe for the deployment of novel machine learning algorithm for both network and application management. However, given the presence of massively distributed and private datasets, it is challenging to use classical centralized learning algorithms in the IoT. To overcome this challenge, federated learning can be a promising solution that enables on-device machine learning without the need to migrate the private end-user data to a central cloud. In federated learning, only learning model updates are transferred between end-devices and the aggregation server. Although federated learning can offer better privacy preservation than centralized machine learning, it has still privacy concerns. In this paper, first, we present the recent advances of federated learning towards enabling federated learning-powered IoT applications. A set of metrics such as sparsification, robustness, quantization, scalability, security, and privacy, is delineated in order to rigorously evaluate the recent advances. Second, we devise a taxonomy for federated learning over IoT networks. Finally, we present several open research challenges with their possible solutions.

The Industrial Internet of Things (IIoT) applications have been recognized as an advancement of the conventional wireless network that concentrates on incorporating processes and machines specifically for industrial applications. These Industrial applications frequently use centralized machine learning (ML) approaches not only to enhance their functionality but also to evaluate sensor data for a variety of purposes, including digitizing operations in manufacturers, forecasting maintenance requirements in industrial equipment, and detecting anomalies for security monitoring, they may adversely affect overall system performance due to high cost of computing power and privacy concerns, as so much data is stored on a cloud server. Federated Learning (FL) has emerged as a new benchmark for centralized ML methods. It sends models to user devices without transferring private data to third-party or central servers; it is one of the promising solutions to data leakage issues. This work introduces a comprehensive overview of the advancements, challenges, and future directions in FL adoption with edge devices. It covers security threats and mitigation strategies, emphasizing its categories, privacy and concerns, communication overhead obstacles, heterogeneity issues, aggregation techniques, and associated development tools. This review paper delves into FL-related topics, including system platforms, offering a comprehensive overview of best practice systems in real-world FL applications. To ensure security in IIoT applications, reviewing threats and mitigation strategies by integrating FL with state-of-the-art technologies such as blockchain, federated reinforcement learning, and federated meta-learning has been explored. Finally, the recent research is taking place to determine new future directions and opportunities for FL security defense mechanisms has been considered at the end of this review paper.

Federated Learning (FL) is a new approach in machine learning or it can also be called collaborative learning, which is a machine learning method that includes client devices to carry out the training process, so that clients do not need to send training data to the server but directly conduct training on their respective devices. respectively. The models generated from local training will be sent to the server for further global aggregation. Therefore, FL is referred to as machine learning which can maintain the privacy of the data owner, because the data is not submitted to the server and is still stored in each client's device. In this study, a performance comparison will be carried out to prove whether the latest approach which is Federated Learning, can produce the same accuracy performance as the traditional approach, that is Centralized Machine Learning, in the case of Image Classification. The comparison of two approaches would be conducted by using the Open Source Image Classification dataset, namely MNIST. The performance of two approaches would be presented by evaluation that is Accuracy. The result shows that Federated Learning almost overcome the performance of Centralized Learning in the case of Image Classification by provided Accuracy 76%.

Centralized machine learning requires the centralization of data in one server for model training, the data of individuals must be transmitted to the centralized server using its raw form which resulting in serious privacy and security concerns. Federated learning is a decentralization machine learning technique which improves the issues of security and privacy related to traditional machine learning by enabling local model training on devices without sharing raw data with the centralized server. Federated learning includes multiple clients and one central server. Clients perform training on its own data while the server coordinates the overall federated learning process. In federated learning, raw data never leaves its own place, ensuring data confidentiality. Only local model updates, form each client are transmitted to the central server that organizes the learning process. The server performs aggregation on received local model updates. Following the aggregation process, the global model is then updated by the server. The final global model is used then for evaluation. However federated learning improves privacy along with security of centralized machine learning, it is still targeted by attacks through model updates transmitted between clients and server. To improve privacy along with security related to federated learning, privacy preservation techniques are integrated with federated learning. We propose a survey of privacy preservation techniques combined with federated learning to improve privacy and security and achieve a good balance between utility and privacy. Private Aggregation of Teacher Ensembles, Homomorphic Encryption, as well as Secure Multi-Party Computation represent the most popular used privacy preservation techniques with federated learning for malicious behavior detection.

The proliferation of machine learning (ML) applications has lately witnessed a considerable shift to more distributed settings, even reaching hand-held mobile devices; there, contrary to typical Centralized learning (CL) whereby the involved (large amounts of) training data are centrally gathered to train models, the load of training tasks is distributed across a set of capable mobile learners at the expense of their own energy. The idea of Federated learning (FL) has emerged as a privacy-preserving mechanism suggesting that the ML model parameters rather than data, are sent over the network to a central point of aggregation. However, when relaxing the privacy concerns, the debate strongly relates to the available network resources. Interestingly, the sofar theoretical or even experimental comparison of the two approaches overlooks network conditions and remains of low realism. In this work we rely on past measurement studies to introduce a realistic system model that accounts for all involved mobile network conditions such as bandwidth and data availability (af-fecting training accuracy and model aggregation) as well as user mobility patterns (affecting data loss). A dedicated simulation framework we have developed replays rich mobile-traces allowing for a comprehensive comparison of the two ML approaches over a large set of training data shedding light on network-resources utilization, energy efficiency and training convergence. Intuitively, our results suggest that the ratio between the employed raw data and the corresponding ML model shapes the conditions under which FL acts as a network-efficient alternative to CL. Interestingly enough, asymmetry in data availability across users as well as their varying number are shown to hardly affect the FL approach in traffic and energy needs, pointing both to its promising potential and the need for further research.

With the growing concern of the customer data privacy, serving them with personalized engaging experiences through Machine Learning (ML) models built on centralized servers is becoming a challenge. Recent developments such as Federated Learning(FL), which is a privacy preserving ML scheme are gaining much attention. In FL model training happens with data federated across devices and not leaving them to sustain user privacy. This is ensured by making the untrained or partially trained models to reach directly the individual devices and getting locally trained "on-device" using the device owned data, and the server aggregating all the partially trained model learnings to update a global model. Although almost all the model learning schemes in the federated learning setup use gradient descent, there are certain characteristic differences brought about by the non-IID nature of the data availability, that affects the training in comparison to the centralized schemes. In this paper, we discuss the various factors that affect the federated learning training, because of the non-IID distributed nature of the data, as well as the inherent differences in the federating learning approach as against the centralized gradient descent techniques. We empirically demonstrate the effect of number of samples per device and the distribution of labels on federated learning. In addition to the privacy advantage we seek through federated learning, we also study if there is a cost advantage while using federated learning frameworks. The cost includes the cloud infrastructure cost for training and deployment, including communication costs, and download and upload cost(of models). We show that federated learning does have an advantage in cost when the model sizes to be trained are not reasonably large. All in all, we present the need for careful design of model for both performance and cost advantages of FL system adoption across services.

In recent years, mobile devices are equipped with increasingly advanced sensing and computing capabilities. Coupled with advancements in Deep Learning (DL), this opens up countless possibilities for meaningful applications. Traditional cloudbased Machine Learning (ML) approaches require the data to be centralized in a cloud server or data center. However, this results in critical issues related to unacceptable latency and communication inefficiency. To this end, Mobile Edge Computing (MEC) has been proposed to bring intelligence closer to the edge, where data is produced. However, conventional enabling technologies for ML at mobile edge networks still require personal data to be shared with external parties, e.g., edge servers. Recently, in light of increasingly stringent data privacy legislations and growing privacy concerns, the concept of Federated Learning (FL) has been introduced. In FL, end devices use their local data to train an ML model required by the server. The end devices then send the model updates rather than raw data to the server for aggregation. FL can serve as an enabling technology in mobile edge networks since it enables the collaborative training of an ML model and also enables DL for mobile edge network optimization. However, in a large-scale and complex mobile edge network, heterogeneous devices with varying constraints are involved. This raises challenges of communication costs, resource allocation, and privacy and security in the implementation of FL at scale. In this survey, we begin with an introduction to the background and fundamentals of FL. Then, we highlight the aforementioned challenges of FL implementation and review existing solutions. Furthermore, we present the applications of FL for mobile edge network optimization. Finally, we discuss the important challenges and future research directions in FL

The emerging integration of Internet of Things (IoT) and AI has unlocked numerous opportunities for innovation across diverse industries. However, growing privacy concerns and data isolation issues have inhibited this promising advancement. Unfortunately, traditional centralized Machine Learning (ML) methods have demonstrated their limitations in addressing these hurdles. In response to this ever-evolving landscape, Federated Learning (FL) has surfaced as a cutting-edge ML paradigm, enabling collaborative training across decentralized devices. FL allows users to jointly construct AI models without sharing their local raw data, ensuring data privacy, network scalability, and minimal data transfer. One essential aspect of FL revolves around proficient knowledge aggregation within a heterogeneous environment. Yet, the inherent characteristics of FL have amplified the complexity of its practical implementation compared to centralized ML. This survey delves into three prominent clusters of FL research contributions: personalization, optimization, and robustness. The objective is to provide a well-structured and fine-grained classification scheme related to these research areas through a unique methodology for selecting related work. Unlike other survey papers, we employed a hybrid approach that amalgamates bibliometric analysis and systematic scrutinizing to find the most influential work in the literature. Therefore, we examine challenges and contemporary techniques related to heterogeneity, efficiency, security, and privacy. Another valuable asset of this study is its comprehensive coverage of FL aggregation strategies, encompassing architectural features, synchronization methods, and several federation motivations. To further enrich our investigation, we provide practical insights into evaluating novel FL proposals and conduct experiments to assess and compare aggregation methods under IID and non-IID data distributions. Finally, we present a compelling set of research avenues that call for further exploration to open up a treasure of advancement.

Visible light communication (VLC) technology was introduced as a key enabler for the next generation of wireless networks, mainly thanks to its simple and low-cost implementation. However, several challenges prohibit the realization of the full potential of VLC, namely, limited modulation bandwidth, ambient light interference, optical diffuse reflection effects, devices' nonlinearity, and random receiver orientation. On the contrary, centralized machine learning (ML) techniques have demonstrated significant potential in handling different challenges related to wireless communication systems. Specifically, it has been shown that ML algorithms exhibit superior capabilities in handling complicated network tasks, such as channel equalization, estimation and modeling, resources allocation, and opportunistic spectrum access control, to name a few. Nevertheless, concerns pertaining to privacy and communication overhead when sharing raw data of the involved clients with a server constitute major bottlenecks in the implementation of centralized ML techniques. This has led to the emergence of a new distributed ML paradigm, namely federated learning (FL), which can reduce the cost associated with transferring raw data, and preserve privacy by training ML models locally and collaboratively at the clients' side. Hence, it becomes evident that integrating FL into VLC networks can provide ubiquitous and reliable implementation of VLC systems. With this motivation, this is the first in-depth review in the literature on the application of FL in VLC networks. To that end, besides the different architectures and related characteristics of FL, we provide a thorough overview on the main design aspects of FL-based VLC systems. Finally, we also highlight some potential future research directions of FL that are envisioned to substantially enhance the performance and robustness of VLC systems.

The Internet of Things (IoT) is a network of electrical devices that are connected to the Internet wirelessly. This group of devices generates a large amount of data with information about users, which makes the whole system sensitive and prone to malicious attacks eventually. The rapidly growing IoT-connected devices under a centralized ML system could threaten data privacy. The popular centralized machine learning (ML)-assisted approaches are difficult to apply due to their requirement of enormous amounts of data in a central entity. Owing to the growing distribution of data over numerous networks of connected devices, decentralized ML solutions are needed. In this paper, we propose a Federated Learning (FL) method for detecting unwanted intrusions to guarantee the protection of IoT networks. This method ensures privacy and security by federated training of local IoT device data. Local IoT clients share only parameter updates with a central global server, which aggregates them and distributes an improved detection algorithm. After each round of FL training, each of the IoT clients receives an updated model from the global server and trains their local dataset, where IoT devices can keep their own privacy intact while optimizing the overall model. To evaluate the efficiency of the proposed method, we conducted exhaustive experiments on a new dataset named Edge-IIoTset. The performance evaluation demonstrates the reliability and effectiveness of the proposed intrusion detection model by achieving an accuracy (92.49%) close to that offered by the conventional centralized ML models’ accuracy (93.92%) using the FL method.

In this study, to reduce the energy consumption for the federated learning (FL) participation of mobile devices (MDs), we design a novel joint dataset and incentive management mechanism for FL over mobile edge computing (MEC) systems. We formulate a Stackelberg game to model and analyze the behaviors of FL participants, referred to as MDs, and FL service providers, referred to as MECs. In the proposed game, each MEC is the leader, whereas the MDs are followers. As the leader, to maximize its own revenue by considering the trade-off between the cost of providing incentives and the estimated accuracy attained from an FL operation, each MEC provides full incentives to the MDs for the participation of each FL task, as well as the target accuracy level for each MD. The suggested total incentives are allocated over MDs’ proportion to the amount of dataset applied for local training, which indirectly affects the global accuracy of the FL. Based on the suggested incentives, the MDs determine the amount of dataset used for the local training of each FL task to maximize their own payoffs, which is defined as the energy consumed from FL participation and the expected incentives. We study the economic benefits of the joint dataset and incentive management mechanism by analyzing its hierarchical decision-making scheme as a multi-leader multi-follower Stackelberg game. Using backward induction, we prove the existence and uniqueness of the Nash equilibrium among MDs, and then examine the Stackelberg equilibrium by analyzing the leader game. We also discuss extensions of the proposed mechanism where the MDs are unaware of explicit information of other MD profiles, such as the weights of the revenue as a practical concern, which can be redesigned into the Stackelberg Bayesian game. Finally, we reveal that the Stackelberg equilibrium solution maximizes the utility of all MDs and the MECs.