Centered Learning Research Articles

Federated Active Learning for Multicenter Collaborative Disease Diagnosis.

Current computer-aided diagnosis system with deep learning method plays an important role in the field of medical imaging. The collaborative diagnosis of diseases by multiple medical institutions has become a popular trend. However, large scale annotations put heavy burdens on medical experts. Furthermore, the centralized learning system has defects in privacy protection and model generalization. To meet these challenges, we propose two federated active learning methods for multicenter collaborative diagnosis of diseases: the Labeling Efficient Federated Active Learning (LEFAL) and the Training Efficient Federated Active Learning (TEFAL). The proposed LEFAL applies a task-agnostic hybrid sampling strategy considering data uncertainty and diversity simultaneously to improve data efficiency. The proposed TEFAL evaluates the client informativeness with a discriminator to improve client efficiency. On the Hyper-Kvasir dataset for gastrointestinal disease diagnosis, with only 65% of labeled data, the LEFAL achieves 95% performance on the segmentation task with whole labeled data. Moreover, on the CC-CCII dataset for COVID-19 diagnosis, with only 50 iterations, the accuracy and F1-score of TEFAL are 0.90 and 0.95, respectively on the classification task. Extensive experimental results demonstrate that the proposed federated active learning methods outperform state-of-the-art methods on segmentation and classification tasks for multicenter collaborative disease diagnosis.

A Federated Learning System for Histopathology Image Analysis with an Orchestral Stain-Normalization GAN.

Currently, data-driven based machine learning is considered one of the best choices in clinical pathology analysis, and its success is subject to the sufficiency of digitized slides, particularly those with deep annotations. Although centralized training on a large data set may be more reliable and more generalized, the slides to the examination are more often than not collected from many distributed medical institutes. This brings its own challenges, and the most important is the assurance of privacy and security of incoming data samples. In the discipline of histopathology image, the universal stain-variation issue adds to the difficulty of an automatic system as different clinical institutions provide distinct stain styles. To address these two important challenges in AI-based histopathology diagnoses, this work proposes a novel conditional Generative Adversarial Network (GAN) with one orchestration generator and multiple distributed discriminators, to cope with multiple-client based stain-style normalization. Implemented within a Federated Learning (FL) paradigm, this framework well preserves data privacy and security. Additionally, the training consistency and stability of the distributed system are further enhanced by a novel temporal self-distillation regularization scheme. Empirically, on large cohorts of histopathology datasets as a benchmark, the proposed model matches the performance of conventional centralized learning very closely. It also outperforms state-of-the-art stain-style transfer methods on the downstream Federated Learning image classification task, with an accuracy increase of over 20.0% in comparison to the baseline classification model.

Learner vs Teacher Centered Classes: Critical Discourse Analysis of Stakeholders' Stance from Socio-Cognitive Perspective

The research conducted for this study took place in a Pakistani educational institution. It compared the learner-centred (LC) and teacher-centred (TC) methods and investigated the learners' opinions regarding each method. This study employed qualitative methodology. Two classroom discussions were recorded in LC and TC modes to capture data. Participants in the study were given an open-ended questionnaire. Analyses of data were guided by Critical Discourse Analysis (CDA). The application of the socio-cognitive model (Dijk, 2006) and 3D model (Fairclough, 1995) revealed speech and ideology differences between TC and LC. The learners preferred the LC method. This research will encourage other academics to research different instruction methods. This research will also benefit applied linguists, particularly language teachers, to determine the effectiveness of teaching methods.

SUMO Simulations for Federated Learning in Communicating Autonomous Vehicles

In transportation, a vehicle's route is one of the most private information. However, to mutually learn some phenomena in a city, for example, parking lot occupancies, we might have to reveal information about it. In this paper, we focus on assessing the privacy loss in a vehicular federated machine learning system. For the analysis, we used the Monaco SUMO Traffic Scenario (MoST). We also used the simulation inputs as statistical data to calculate privacy loss metrics. Results show that a vehicular federated machine learning system may pose a smaller privacy threat than individual learning, but its performance is lower compared to a centralized learning approach.
 Due to the vast amount of data and processing time, we also describe a method to build a Docker image of SUMO together with a software client-server architecture for SUMO-based learning systems on multiple computers.

Evaluating cooperative-competitive dynamics with deep Q-learning

We model cooperative-competitive social group dynamics with multi-agent environments, specialized in cases with a large number of agents from only a few distinct types. The multi-agent optimization problems are addressed in turn with multi-agent reinforcement learning algorithms to obtain flexible and robust solutions. We analyze the effectiveness of centralized and decentralized algorithms using three variants of deep Q-networks on these cooperative-competitive environments: first, we use the decentralized training independent learning with deep Q-networks, secondly the centralized monotonic value factorizations for deep learning, and lastly the multi-agent variational exploration. We test the algorithms in simulated predator–prey multi-agent environments in two distinct environments: the adversary pursuit and simple tag. The experiments highlight the performance of the different deep Q-learning methods, and we conclude that decentralized training of deep Q-networks accumulates higher episode rewards during training and evaluation in comparison with the selected centralized learning approaches.

Federated Robustness Propagation: Sharing Adversarial Robustness in Heterogeneous Federated Learning.

Federated learning (FL) emerges as a popular distributed learning schema that learns a model from a set of participating users without sharing raw data. One major challenge of FL comes with heterogeneous users, who may have distributionally different (or non-iid) data and varying computation resources. As federated users would use the model for prediction, they often demand the trained model to be robust against malicious attackers at test time. Whereas adversarial training (AT) provides a sound solution for centralized learning, extending its usage for federated users has imposed significant challenges, as many users may have very limited training data and tight computational budgets, to afford the data-hungry and costly AT. In this paper, we study a novel FL strategy: propagating adversarial robustness from rich-resource users that can afford AT, to those with poor resources that cannot afford it, during federated learning. We show that existing FL techniques cannot be effectively integrated with the strategy to propagate robustness among non-iid users and propose an efficient propagation approach by the proper use of batch-normalization. We demonstrate the rationality and effectiveness of our method through extensive experiments. Especially, the proposed method is shown to grant federated models remarkable robustness even when only a small portion of users afford AT during learning. Source code will be released.

Differentially private knowledge transfer for federated learning

Extracting useful knowledge from big data is important for machine learning. When data is privacy-sensitive and cannot be directly collected, federated learning is a promising option that extracts knowledge from decentralized data by learning and exchanging model parameters, rather than raw data. However, model parameters may encode not only non-private knowledge but also private information of local data, thereby transferring knowledge via model parameters is not privacy-secure. Here, we present a knowledge transfer method named PrivateKT, which uses actively selected small public data to transfer high-quality knowledge in federated learning with privacy guarantees. We verify PrivateKT on three different datasets, and results show that PrivateKT can maximally reduce 84% of the performance gap between centralized learning and existing federated learning methods under strict differential privacy restrictions. PrivateKT provides a potential direction to effective and privacy-preserving knowledge transfer in machine intelligent systems.

FedraTrees: A novel computation-communication efficient federated learning framework investigated in smart grids

Smart energy performance monitoring and optimisation at the supplier and consumer levels is essential to realising smart cities. In order to implement a more sustainable energy management plan, it is crucial to conduct a better energy forecast. The next-generation smart meters can also be used to measure, record, and report energy consumption data, which can be used to train machine learning (ML) models for predicting energy needs. However, sharing energy consumption information to perform centralised learning may compromise data privacy and make it vulnerable to misuse, in addition to incurring high transmission overhead on communication resources. This study addresses these issues by utilising federated learning (FL), an emerging technique that performs ML model training at the user/substation level, where data resides. We introduce FedraTrees, a new, lightweight FL framework that benefits from the outstanding features of ensemble learning. Furthermore, we developed a delta-based FL stopping algorithm to monitor FL training and stop it when it does not need to continue. The simulation results demonstrate that FedraTrees outperforms the most popular federated averaging (FedAvg) framework and the baseline Persistence model for providing accurate energy forecasting patterns while taking only 2% of the computation time and 13% of the communication rounds compared to FedAvg, saving considerable amounts of computation and communication resources.

Multi-agent deep reinforcement learning-based optimal energy management for grid-connected multiple energy carrier microgrids

Multi-agent deep reinforcement learning (MA-DRL) method provides a groundbreaking approach to tackling computational problems in power systems, particularly for distributed energy resources that have been widely adopted to advance energy sustainability. This paper presents a novel optimal energy management based on proposed MA-DRL method. This method employs deep neural network to learn strategy based on stacked-denoising auto-encoders and multi-agent deep deterministic policy gradient learning capability. The MA-DRL method is adopted to find the optimal strategy of the optimal energy management problem under the Markov decision process framework. This method aims to coordinate multiple energies and achieve optimal operation over a variety of hourly dispatches while taking into account the distinct properties of electric and thermal energies. The primary challenge of the planning and operation of multiple energy carrier microgrids (MECMs) is determining the optimal interaction between renewable energy resources, energy storage systems, power-to-thermal conversion systems, and upstream power grid in order to improve overall energy utilization efficiency. The presented robust method can adaptively derive the optimal operation for MECMs through centralized learning and decentralized implementation. The optimization problem is employed in this study to concurrently reduce the total emissions and the operating costs while considering engineering design constraints. Finally, to demonstrate the efficiency of the proposed method, it is verified on an integrated modified IEEE 33-bus and 8-node gas systems.

Guest Editorial Special Issue on Collaborative Intelligence for Green Internet of Things in the 6G Era

In the 6G era, with the hyper-connectivity among humans and everything, we are anticipating Internet of Things (IoT) applications in various fields, including smart city, smart factory, smart home, smart grid, e-health, and smart transportation, accompanied by new services with rich experiences, such as truly immersive VR/AR/MR (XR), high-fidelity mobile hologram, and digital twins. However, in order to facilitate these emerging IoT applications, we have to investigate the collaboration among different network entities to achieve joint optimization under heterogeneous communication, caching, and computing resources. Recently, artificial intelligence (AI) based approaches have been attracting great interest in empowering computer systems. Since the centralized learning approaches face many challenges in terms of scalability, some collaborative learning approaches, such as federated learning and multi-agent systems, have been investigated recently to reduce networking overhead and improve learning efficiency. Based on refined AI technologies, collaborative intelligence can achieve better decisions by aggregating knowledge and enabling efficient coordination among multiple agents with a light communication overhead. It is envisioned that the collaborative intelligence is the enabler for collaborative IoT systems.

Application Research on Risk Assessment of Municipal Pipeline Network Based on Random Forest Machine Learning Algorithm

Urban municipal water supply is an important part of underground pipelines, and their scale continues to expand. Due to the continuous improvement in the quality and quantity of data available for pipeline systems in recent years, traditional pipeline network risk assessment cannot cope with the improvement of various monitoring methods. Therefore, this paper proposes a machine learning-based risk assessment method for municipal pipe network operation and maintenance and builds a model example based on the data of a pipeline network base in a park in Suzhou. We optimized the random forest learning model, compared it with other centralized learning methods, and finally evaluated the model’s learning effect. Finally, the risk probability associated with each pipe segment sample was obtained, the risk factors affecting the pipe segment’s failure were determined, and their relevance and importance ranking was established. The results showed that the most influential factors are pipe material, soil properties, service life, and the number of past failures. The random forest algorithm demonstrated better prediction accuracy and robustness on the dataset.

FINE: A Framework for Distributed Learning on Incomplete Observations for Heterogeneous Crowdsensing Networks

Numerous crowdsensing applications have been developed recently in mobile social networks and vehicle networks. How to implement an accurate distributed learning process to estimate parameters of an unknown model in crowdsensing is a significant issue because centralised learning methods produce unreliable data gathering, expensive central servers, and privacy concerns. Due to this, we propose FINE, a distributed learning framework for imperfect data and non-smooth estimation, along with its design, analysis, and assessment. Our design, which is focused on creating a workable framework for learning parameters in crowdsensing networks accurately and efficiently, generalises earlier learning techniques by supporting heterogeneous dimensions of data records observed by various nodes as well as minimization based on non-smooth error functions.In particular, FINE makes use of a distributed dual average technique that efficiently minimises non-smooth error functions and a novel distributed record completion algorithm that enables each node to get the global consensus through effective communication with neighbours. All of these algorithms converge, as shown by our analysis, and the convergence rates are also obtained to support their efficacy. Through experiments on synthetic and actual networks, we assess how well our framework performs

Migrating federated learning to centralized learning with the leverage of unlabeled data

Federated learning carries out cooperative training without local data sharing; the obtained global model performs generally better than independent local models. Benefiting from the free data sharing, federated learning preserves the privacy of local users. However, the performance of the global model might be degraded if diverse clients hold non-IID training data. This is because the different distributions of local data lead to weight divergence of local models. In this paper, we introduce a novel teacher–student framework to alleviate the negative impact of non-IID data. On the one hand, we maintain the advantage of the federated learning on the privacy-preserving, and on the other hand, we take the advantage of the centralized learning on the accuracy. We use unlabeled data and global models as teachers to generate a pseudo-labeled dataset, which can significantly improve the performance of the global model. At the same time, the global model as a teacher provides more accurate pseudo-labels. In addition, we perform a model rollback to mitigate the impact of latent noise labels and data imbalance in the pseudo-labeled dataset. Extensive experiments have verified that our teacher ensemble performs a more robust training. The empirical study verifies that the reliance on the centralized pseudo-labeled data enables the global model almost immune to non-IID data.

Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification

Realizing human-like perception is a challenge in open driving scenarios due to corner cases and visual occlusions. To gather knowledge of rare and occluded instances, federated learning assisted connected autonomous vehicle (FLCAV) has been proposed, which leverages vehicular networks to establish federated deep neural networks (DNNs) from distributed data captured by vehicles and road sensors. Without the need of data aggregation, FLCAV preserves privacy while reducing communication costs compared with conventional centralized learning. However, it is challenging to determine the network resources and road sensor placements for multi-stage training with multi-modal datasets in multi-variant scenarios. This article presents networking and training frameworks for FLCAV perception. Multi-layer graph resource allocation and vehicle-road contrastive sensor placement are proposed to address the network management and sensor deployment problems, respectively. We also develop CarlaFLCAV, a software platform that implements the above system and methods. Experimental results confirm the superiority of the proposed techniques compared with various benchmarks.

A comparison study of centralized and decentralized federated learning approaches utilizing the transformer architecture for estimating remaining useful life

The current prognostics approaches for a network of assets are centralized and reliant on the availability of assets’ sensors, failures, and anomaly data. To address this, the data from similar assets are usually aggregated to make a richer dataset for prognosis. However, if similar assets are located at different enterprises, business owners may not be willing to share their raw data with each other. One solution is decentralized Federated Learning (FL), where local client data and training is preserved on-site rather than being shared with a central server. Since FL theoretically addresses the challenges faced by the traditional centralized learning approaches, its performance needs to be investigated and compared with the centralized methods. The current paper aims to compare the performance of a centralized model with two decentralized FL algorithms to predict the remaining useful life (RUL) of an asset. Two prediction models, a long short-term memory (LSTM) and the Transformer architecture were developed to predict RUL. The comparison has been conducted using NASA C-MAPSS dataset where the results indicated that FedProx performed better than FedAvg generally, and Transformer architecture performed better overall than LSTM across all datasets in the centralized and decentralized scenarios.

Evaluation of Federated Learning in Phishing Email Detection.

The use of artificial intelligence (AI) to detect phishing emails is primarily dependent on large-scale centralized datasets, which has opened it up to a myriad of privacy, trust, and legal issues. Moreover, organizations have been loath to share emails, given the risk of leaking commercially sensitive information. Consequently, it has been difficult to obtain sufficient emails to train a global AI model efficiently. Accordingly, privacy-preserving distributed and collaborative machine learning, particularly federated learning (FL), is a desideratum. As it is already prevalent in the healthcare sector, questions remain regarding the effectiveness and efficacy of FL-based phishing detection within the context of multi-organization collaborations. To the best of our knowledge, the work herein was the first to investigate the use of FL in phishing email detection. This study focused on building upon a deep neural network model, particularly recurrent convolutional neural network (RNN) and bidirectional encoder representations from transformers (BERT), for phishing email detection. We analyzed the FL-entangled learning performance in various settings, including (i) a balanced and asymmetrical data distribution among organizations and (ii) scalability. Our results corroborated the comparable performance statistics of FL in phishing email detection to centralized learning for balanced datasets and low organizational counts. Moreover, we observed a variation in performance when increasing the organizational counts. For a fixed total email dataset, the global RNN-based model had a 1.8% accuracy decrease when the organizational counts were increased from 2 to 10. In contrast, BERT accuracy increased by 0.6% when increasing organizational counts from 2 to 5. However, if we increased the overall email dataset by introducing new organizations in the FL framework, the organizational level performance improved by achieving a faster convergence speed. In addition, FL suffered in its overall global model performance due to highly unstable outputs if the email dataset distribution was highly asymmetric.

Implementing the Undergraduate Simulation Framework for Large Nursing Cohorts: A Program Evaluation

BackgroundThe Undergraduate Simulation Framework was developed to support quality teaching and improve the active learning experiences in undergraduate nursing curricula. MethodTo evaluate the framework, eight educators were invited to share their experiences of utilizing this new model of simulation delivery. ResultsThematic data analysis revealed four themes: timing is everything; learning through observation and reflection; overcoming being overwhelmed; and increased learner centeredness. ConclusionThe Undergraduate Simulation Framework is appropriate to embed in nursing courses and can increase the delivery of simulations to large cohorts without impairing their quality.

Secure Intrusion Detection by Differentially Private Federated Learning for Inter-Vehicle Networks

Along with providing several benefits, the unprecedented growth of connected and automated vehicles brings worries about damaging cyber attacks. Network-based intrusion detection systems (IDSs) using deep learning methods can effectively mitigate the threats by promptly detecting malicious behaviors. However, the centralized learning mode may cause data leverage. Federated learning has emerged as a new distributed machine learning training paradigm to preserve data privacy by allowing clients to train and validate models locally with their data and then send the model parameters to the central server. First, we propose a new framework named DPFL-F2IDS scheme for an edge inter-vehicle network that transmits Basic Safety Messages, which consists of Differentially Private Federated Learning (DPFL) and F2IDS (Framework for IDS). DPFL can defend against the member inference attacks faced by the standard federated learning, but difficulties still exist in making a tradeoff on the utility metrics and privacy metrics. Second, experiments by centralized learning methods were performed on the VeReMi Extension dataset. Third, the performance of federated learning by different numbers of vehicles and different optimizers is evaluated. DPFL by different noise values is also evaluated. Results clarified that the F1-scores reached 0.9915 and 0.9700 by long short-term memory (LSTM)-based intrusion detection for binary classification and multi-classification in the centralized learning mode, respectively. The utility results achieved by federated averaging (FedAvg) with Adabound optimizer were closer to the centralized learning mode than the classical FedAvg algorithms. The optimal values of the noise multipliers were also found without degrading the model quality and preserving privacy.

Hybrid Centralized and Distributed Learning for MEC-Equipped Satellite 6G Networks

For future networks in the 6G, it will be important to maintain a ubiquitous connection, bring processing heavy applications to remote areas, and analyze big amounts of data to efficiently provide services. To achieve such goals, the literature has utilized satellite networks to reach areas far away from the network core, and there has even been research into equipping such satellites with edge cloud servers to provide computation offloading to remote devices. However, analyzing the big data created by these devices is still a problem. One could transfer the data to a central server, but this has a high transmission cost. One could process the data through distributed machine learning, but such a technique is not as efficient as centralized learning. Thus, in this paper, we analyze the learning costs behind centralized and distributed learning and propose a hybrid solution that adaptively uses the advantages of both in a cloud server-equipped satellite network. Our proposal can identify the best learning strategy for each device based on the current scenario. Results show that the proposal is not only efficient in solving machine learning tasks, but it is also dynamic to react to different configurations while maintaining top performance.

Wireless Distributed Learning: A New Hybrid Split and Federated Learning Approach

Cellular-connected unmanned aerial vehicle (UAV) with flexible deployment is foreseen to be a major part of the sixth generation (6G) networks. The UAVs connected to the base station (BS), as aerial users (UEs), could exploit machine learning (ML) algorithms to provide a wide range of advanced applications, like object detection and video tracking. Conventionally, the ML model training is performed at the BS, known as centralized learning (CL), which causes high communication overhead due to the transmission of large datasets, and potential concerns about UE privacy. To address this, distributed learning algorithms, including federated learning (FL) and split learning (SL), were proposed to train the ML models in a distributed manner via only sharing model parameters. FL requires higher computational resource on the UE side than SL, while SL has larger communication overhead when the local dataset is large. To effectively train an ML model considering the diversity of UEs with different computational capabilities and channel conditions, we first propose a novel distributed learning architecture, a hybrid split and federated learning (HSFL) algorithm by reaping the parallel model training mechanism of FL and the model splitting structure of SL. We then provide its convergence analysis under non-independent and identically distributed (non-IID) data with random UE selection scheme. By conducting experiments on training two ML models, Net and AlexNet, in wireless UAV networks, our results demonstrate that the HSFL algorithm achieves higher learning accuracy than FL and less communication overhead than SL under IID and non-IID data, and the learning accuracy of HSFL algorithm increases with the increasing number of the split training UEs. We further propose a Multi-Arm Bandit (MAB) based best channel (BC) and best 2-norm (BN2) (MAB-BC-BN2) UE selection scheme to select the UEs with better wireless channel quality and larger local model updates for model training in each round. Numerical results demonstrate it achieves higher learning accuracy than BC, MAB-BC and MAB-BN2 UE selection scheme under non-IID, Dirichlet-nonIID and Dirichlet-Imbalanced data.