Distributed Learning Scheme Research Articles

Potential field mechanisms and distributed learning for enhancing the navigation of micro/nanorobot in biomedical environments

In complex systems, single micro/nanorobots encounter challenges related to limited loading capacity and navigation, hindering their effective utilization in targeted therapy and drug delivery. To solve these challenges, this paper explores potential field mechanisms as a means to simulate natural collective behavior. This approach aims to enhance the navigation and efficiency of micro/nanorobots in high-demand therapeutic areas. The mechanism enables micro/nanorobots to dynamically adapt to environmental gradients, minimizing off-target effects while maximizing therapeutic efficacy and enhancing robustness through redundancy. Additionally, this study introduces innovative distributed learning and cooperative control strategies. Each micro/nanorobot updates its navigation strategy through local interactions and influences with the dynamic environment. This allows micro/nanorobots to share information and improve their navigation toward therapeutic targets. The simulation results demonstrate that collective behavior and potential field mechanisms can enhance the precision and efficiency of targeted therapy and drug delivery in dynamically changing environments. In conclusion, the proposed approach can improve the limitations of single micro/nanobot, offering new possibilities for the development of advanced therapeutics and drug delivery systems.

Investigation of distributed learning for automated lesion detection in head MR images

In this study, we investigated the application of distributed learning, including federated learning and cyclical weight transfer, in the development of computer-aided detection (CADe) software for (1) cerebral aneurysm detection in magnetic resonance (MR) angiography images and (2) brain metastasis detection in brain contrast-enhanced MR images. We used datasets collected from various institutions, scanner vendors, and magnetic field strengths for each target CADe software. We compared the performance of multiple strategies, including a centralized strategy, in which software development is conducted at a development institution after collecting de-identified data from multiple institutions. Our results showed that the performance of CADe software trained through distributed learning was equal to or better than that trained through the centralized strategy. However, the distributed learning strategies that achieved the highest performance depend on the target CADe software. Hence, distributed learning can become one of the strategies for CADe software development using data collected from multiple institutions.

Proposal and investigation of a distributed learning strategy in Orbital Edge Computing-endowed satellite networks for Earth Observation applications

One of the key enabling solutions to in-orbit extract information from Earth Observation images is given by deep learning techniques. However, the accuracy of these algorithms is strictly related to the availability of large datasets of satellite images for training purposes. Limitations on the available transmission bandwidth in the orbital context may prevent the possibility to downlink all acquired images to a node where centralized training happens. Instead, Federated Learning (FL) could be fruitfully leveraged in this scenario, since it provides for each satellite to train a local model only with its own dataset, and then to share its trained model with a central server, which receives models trained by the different satellites and aggregates them into a new global model being eventually shared with all the satellites, and this repeats until convergence is reached. However, because communication with a node acting as a central parameter server may be still limited by short visibility time, the described process may need a long time because of limited communication windows, negatively impacting the time needed to reach model convergence. For this reason, we propose a communication strategy to support a completely distributed learning technique to train a deep learning model in-orbit, by leveraging the fact that satellites may form a network thanks to the potential availability of Inter-Satellite Links (ISLs) within and between orbital planes. Our proposal is different from a FL approach since we provide for each satellite to receive all the information needed to calculate an updated global model by itself, without leaning on a central parameter server. Numerical results show that distributed learning outperforms FL in number of learning rounds completed in the unit time, allowing for reaching validation accuracy convergence in a shorter time, as it has been verified on a land coverage classification task based on the EuroSAT dataset.

MS-DINO: Masked Self-Supervised Distributed Learning Using Vision Transformer.

Despite promising advancements in deep learning in medical domains, challenges still remain owing to data scarcity, compounded by privacy concerns and data ownership disputes. Recent explorations of distributed-learning paradigms, particularly federated learning, have aimed to mitigate these challenges. However, these approaches are often encumbered by substantial communication and computational overhead, and potential vulnerabilities in privacy safeguards. Therefore, we propose a self-supervised masked sampling distillation technique called MS-DINO, tailored to the vision transformer architecture. This approach removes the need for incessant communication and strengthens privacy using a modified encryption mechanism inherent to the vision transformer while minimizing the computational burden on client-side devices. Rigorous evaluations across various tasks confirmed that our method outperforms existing self-supervised distributed learning strategies and fine-tuned baselines.

Federated Learning for Digital Twin-Based Vehicular Networks: Architecture and Challenges

A digital twin uses a virtual model of the physical system to fulfill the diverse requirements (e.g., latency, reliability, quality of physical experience) for emerging vehicular network applications. Although a twin-based implementation of vehicular networks can offer performance optimization, modeling a digital twin is a significantly challenging task. Federated learning (FL) is a better privacy-preserving, distributed learning scheme that can be used to model twin models. Although FL can offer performance enhancement, it requires careful design. Therefore, in this article, we present an overview of FL for a twinbased vehicular network. A general architecture for FL-enabled digital twins for a vehicular network is presented. Our proposed architecture consists of two spaces, such as twin space and the physical space. A physical space consists of all the physical entities (e.g., cars and edge servers) required for vehicular networks, whereas, the twin space refers to the logical space that is used for the deployment of twins. A twin space can be implemented either using edge servers and cloud servers. We also outline a few use cases of FL for the twin-based vehicular network. Finally, the paper is concluded and an outlook on open challenges is presented.

Nash Equilibrium Seeking Algorithm Design for Distributed Nonsmooth Multicluster Games Over Weight-Balanced Digraphs.

In this article, we study the multicluster games over weight-balanced digraphs, where the cost functions of all players are nonsmooth. Besides, in the problem, not only are the decisions of all players constrained by heterogeneous local constraints but also the decisions of players in the same cluster are constrained by coupling constraints. Due to the nonsmooth cost functions, the coupling constraints, the general local convex constraints, and the weight-balanced digraphs, existing Nash equilibrium seeking algorithms cannot solve our problem. In order to seek the Nash equilibrium of the game, we design a distributed algorithm based on subgradient descent, differential inclusions, and projection operations. In the algorithm, a distributed learning strategy is embedded for the players to estimate the decisions of other players. Moreover, we analyze the asymptotical convergence of the algorithm via set-valued LaSalle invariance principle. Finally, a numerical simulation about electricity market games is presented to illustrate the effectiveness of our result.

A Privacy Preserving System for Movie Recommendations Using Federated Learning

Recommender systems have become ubiquitous in the past years. They solve the tyranny of choice problem faced by many users, and are utilized by many online businesses to drive engagement and sales. Besides other criticisms, like creating filter bubbles within social networks, recommender systems are often reproved for collecting considerable amounts of personal data. However, to personalize recommendations, personal information is fundamentally required. A recent distributed learning scheme called federated learning has made it possible to learn from personal user data without its central collection. Consequently, we present a recommender system for movie recommendations, which provides privacy and thus trustworthiness on multiple levels: First and foremost, it is trained using federated learning and thus, by its very nature, privacy-preserving, while still enabling users to benefit from global insights. Furthermore, a novel federated learning scheme, called FedQ, is employed, which not only addresses the problem of non-i.i.d.-ness and small local datasets, but also prevents input data reconstruction attacks by aggregating client updates early. Finally, to reduce the communication overhead, compression is applied, which significantly compresses the exchanged neural network parametrizations to a fraction of their original size. We conjecture that this may also improve data privacy through its lossy quantization stage.

FedER: Federated Learning through Experience Replay and privacy-preserving data synthesis

In the medical field, multi-center collaborations are often sought to yield more generalizable findings by leveraging the heterogeneity of patient and clinical data. However, recent privacy regulations hinder the possibility to share data, and consequently, to come up with machine learning-based solutions that support diagnosis and prognosis. Federated learning (FL) aims at sidestepping this limitation by bringing AI-based solutions to data owners and only sharing local AI models, or parts thereof, that need then to be aggregated. However, most of the existing federated learning solutions are still at their infancy and show several shortcomings, from the lack of a reliable and effective aggregation scheme able to retain the knowledge learned locally to weak privacy preservation as real data may be reconstructed from model updates. Furthermore, the majority of these approaches, especially those dealing with medical data, relies on a centralized distributed learning strategy that poses robustness, scalability and trust issues. In this paper we present a federated learning strategy, FedER, that, exploiting experience replay and generative adversarial concepts, effectively integrates features from local nodes, providing models able to generalize across multiple datasets while maintaining privacy. FedER is tested on two tasks — tuberculosis and melanoma classification — using multiple datasets in order to simulate realistic non-i.i.d. medical data scenarios. Results show that our approach achieves performance comparable to standard (non-federated) learning and significantly outperforms state-of-the-art federated methods. Remarkably, we also observe that FedER enables any node model to be used as a global federation model. Indeed, the experience replay strategy with privacy-preserving synthetic data allows all node models to converge to reach the same optimum without the need of a single shared model. Code is available at https://github.com/perceivelab/FedER.

Privacy-Preserving Probabilistic Voltage Forecasting in Local Energy Communities

This paper presents a new privacy-preserving framework for the short-term (multi-horizon) probabilistic forecasting of nodal voltages in local energy communities. This task is indeed becoming increasingly important for cost-effectively managing network constraints in the context of the massive integration of distributed energy resources. However, traditional forecasting tasks are carried out centrally, by gathering raw data of end-users in a single database that exposes their private information. To avoid such privacy issues, this work relies on a distributed learning scheme, known as federated learning wherein individuals’ data are kept decentralized. The learning procedure is then augmented with differential privacy, which offers formal guarantees that the trained model cannot be reversed-engineered to infer sensitive local information. Moreover, the problem is framed using cross-series learning, which allows to smoothly integrate any new client joining the community (i.e., cold-start forecasting) without being plagued by data scarcity. Outcomes show that the proposed approach achieves improved performance compared to non-collaborative (locally trained) models, and is able to reach a trade-off between privacy and performance for different architectures of deep learning networks.

FL-Incentivizer: FL-NFT and FL-Tokens for Federated Learning Model Trading and Training

Federated learning (FL) is an on-device distributed learning scheme that does not require training devices to transfer their data to a centralized facility. The goal of federated learning is to learn a global model over several iterations. It is challenging to claim ownership rights and commercialize the global model efficiently and transparently. Additionally, incentives need to be provided to ensure that devices participate in the FL process. In this paper, we propose a smart contract-based framework called FL-Incentivizer, which relies on custom smart contracts to maintain flow governance of the FL process in a transparent and immutable manner. FL-Incentivizer commercializes and tokenizes the global model using FL-NFT (FL Non-Fungible Token) based on the ERC-721 standard. FL-Incentivizer uses ERC-20 compliant FL-Tokens to incentivize devices participating in FL. We present the system design and operational sequence of the FL-Incentivizer. We provide implementation and deployment details, complete smart contract codes, and qualitative evaluation of the FL-Incentivizer. After implementing FL-Incentivizer for a global iteration of a Federated learning task, we showed the FL-NFT on OpenSea and an FL-Token for a learner on MetaMask. FL-NFTs can be traded on markets such as OpenSea like other NFTs. While FL-Tokens can be transferred in the same manner as other ERC-20-based tokens.

A Wireless Virtual Reality-Based Multimedia-Assisted Teaching System Framework under Mobile Edge Computing

In recent years, virtual reality (VR) has gradually entered the daily education and teaching activities from pure scientific research. In the area of assistance teaching, some typical computer softwares still play some important roles. This makes remote teaching activities can just learn voice, yet cannot possess the feeling of realistic existence. Especially in scenario of COVID-19, remote teaching activities with proper perceptibility are in urgent demand. To deal with the current challenge, this paper proposes a wireless VR-based multimedia-assisted teaching system framework under mobile edge computing networks. In this framework, cooperative edge caching and adaptive streaming based on viewport prediction are adopted to jointly improve the quality of experience (QoE) of VR users. First, we investigated the resource management problem of caching and adaptive streaming in this framework. Considering the complexity of the formulated problem, a distributed learning scheme is proposed to solve the problem. The experimental data are verified and the experimental results prove that the studied methods improve the performance of user QoE.

Federated Vehicular Transformers and Their Federations: Privacy-Preserving Computing and Cooperation for Autonomous Driving

Cooperative computing is promising to enhance the performance and safety of autonomous vehicles benefiting from the increase in the amount, diversity as well as scope of data resources. However, effective and privacy-preserving utilization of multi-modal and multi-source data remains an open challenge during the construction of cooperative mechanisms. Recently, Transformers have demonstrated their potential in the unified representation of multi-modal features, which provides a new perspective for effective representation and fusion of diverse inputs of intelligent vehicles. Federated learning proposes a distributed learning scheme and is hopeful to achieve privacy-secure sharing of data resources among different vehicles. Towards privacy-preserving computing and cooperation in autonomous driving, this paper reviews recent progress of Transformers, federated learning as well as cooperative perception, and proposes a hierarchical structure of Transformers for intelligent vehicles which is comprised of Vehicular Transformers, Federated Vehicular Transformers and the Federation of Vehicular Transformers to exploit their potential in privacy-preserving collaboration.

Histone-Net: a multi-paradigm computational framework for histone occupancy and modification prediction

Deep exploration of histone occupancy and covalent post-translational modifications (e.g., acetylation, methylation) is essential to decode gene expression regulation, chromosome packaging, DNA damage, and transcriptional activation. Existing computational approaches are unable to precisely predict histone occupancy and modifications mainly due to the use of sub-optimal statistical representation of histone sequences. For the establishment of an improved histone occupancy and modification landscape for multiple histone markers, the paper in hand presents an end-to-end computational multi-paradigm framework “Histone-Net”. To learn local and global residue context aware sequence representation, Histone-Net generates unsupervised higher order residue embeddings (DNA2Vec) and presents a different application of language modelling, where it encapsulates histone occupancy and modification information while generating higher order residue embeddings (SuperDNA2Vec) in a supervised manner. We perform an intrinsic and extrinsic evaluation of both presented distributed representation learning schemes. A comprehensive empirical evaluation of Histone-Net over ten benchmark histone markers data sets for three different histone sequence analysis tasks indicates that SuperDNA2Vec sequence representation and softmax classifier-based approach outperforms state-of-the-art approach by an average accuracy of 7%. To eliminate the overhead of training separate binary classifiers for all ten histone markers, Histone-Net is evaluated in multi-label classification paradigm, where it produces decent performance for simultaneous prediction of histone occupancy, acetylation, and methylation.

Model-Driven Learning for Generic MIMO Downlink Beamforming With Uplink Channel Information

Accurate downlink channel information is crucial to the beamforming design, but it is difficult to obtain in practice. This paper investigates a deep learning-based optimization approach of the downlink beamforming to maximize the system sum rate, when only the uplink channel information is available. Our main contribution is to propose a model-driven learning technique that exploits the structure of the optimal downlink beamforming to design an effective hybrid learning strategy with the aim to maximize the sum rate performance. This is achieved by jointly considering the learning performance of the downlink channel, the power and the sum rate in the training stage. The proposed approach applies to generic cases in which the uplink channel information is available, but its relation to the downlink channel is unknown and does not require an explicit downlink channel estimation. We further extend the developed technique to massive multiple-input multiple-output scenarios and achieve a distributed learning strategy for multicell systems without an inter-cell signalling overhead. Simulation results verify that our proposed method provides the performance close to the state of the art numerical algorithms with perfect downlink channel information and significantly outperforms existing data-driven methods in terms of the sum rate.

Federated learning empowered mobility-aware proactive content offloading framework for fog radio access networks

Proactive content caching has emerged as a promising solution to cope with exponentially increasing mobile data traffic. The popular user contents can be cached near the network edge for faster retrieval and processing. Current state-of-the-art approaches adopt a centralized model training mechanism that requires high communication and data exchange overheads to predict content popularity. Moreover, these approaches fail to deal with the dynamicity of the environment since they do not take into account the users’ mobility information and are unable to incorporate content offload timings. In this paper, we address these limitations by proposing a novel federated learning-based Mobility and Demand-aware Proactive Content Offloading (MDPCO) framework. MDPCO exploits distributed learning strategies and capitalizes on users’ mobility and demand information for proactive content offloading. Extensive simulations are carried out to validate the efficacy of MDCPO against local and cloud-based models. Our proposed model yields an average performance improvement of 6.7% in comparison to the cloud-based model. Furthermore, with the increase in the number of fog servers, the MDPCO achieves a 9.8% higher data offloading ratio and 1.18% increase in the downlink rates while being more energy-efficient than cloud-based approaches.

A distributed learning strategy improves performance and retention of skills in neonatal resuscitation

Skill retention after neonatal resuscitation training is poor. A distributed learning strategy – where learning is spread over multiple sessions – can improve retention of declarative memory (facts & knowledge). Session timings are critical – maximal retention occurs when a refresher session is scheduled at 10-30% of the time between initial training and test. We hypothesized this also holds true for neonatal resuscitation, a complex skill set requiring both declarative and procedural memory. We conducted a prospective, single-blinded randomized-controlled trial. University of Ottawa residents were recruited to training in neonatal resuscitation, with a high-fidelity simulated pre-test, immediate post-tests, and a retention test at 4 months. After training, they were randomized to either a refresher session at 3 weeks (18% of interval) or at 2 months (50%). Technical and non-technical skills were scored using validated checklists, knowledge with standardized questions. There was no difference between groups prior to the retention test. The early refresher group demonstrated significantly improved technical (mean ± 95% CI: 22.4 ± 1.3 v 18.2 ± 2.5, p = 0.02) and non-technical (31.0 ± 0.9 v 25.6 ± 3.1, p = 0.03) skill scores in the retention post-test compared to the late group. No difference was seen with knowledge scores. We conclude that both technical and non-technical aspects of neonatal resuscitation performance can benefit from an early refresher session. Session timings are critical and should be tailored to the desired length of skill retention. Findings may be generalizable to other interventions that depend on mixed types of memory.

Joint Coding and Scheduling Optimization for Distributed Learning Over Wireless Edge Networks

Unlike theoretical analysis of distributed learning (DL) in the literature, DL over wireless edge networks faces the inherent dynamics/uncertainty of wireless connections and edge nodes, making DL less efficient or even inapplicable under the highly dynamic wireless edge networks. This article addresses these problems by leveraging recent advances in coded computing and the deep dueling neural network architecture. By introducing coded structures/redundancy, a distributed learning task can be completed without waiting for straggling nodes. Unlike conventional coded computing that only optimizes the code structure, coded distributed learning over the wireless edge also requires to optimize the selection/scheduling of wireless edge nodes with heterogeneous connections, computing capability, and straggling effects. However, even neglecting the aforementioned dynamics/uncertainty, the resulting joint optimization of coding and scheduling to minimize the distributed learning time turns out to be NP-hard. To tackle this and to account for the dynamics and uncertainty of wireless connections and edge nodes, we reformulate the problem as a Markov Decision Process and design a novel deep reinforcement learning algorithm that employs the deep dueling neural network architecture to find the jointly optimal coding scheme and the best set of edge nodes for different learning tasks without explicit information about the wireless environment and edge nodes’ straggling parameters. Simulations show that the proposed framework reduces the average learning delay in wireless edge computing up to 66% compared with other DL approaches. The jointly optimal framework in this article is also applicable to any distributed learning scheme with heterogeneous and uncertain computing nodes.

A Distributed Learning Scheme for Variational Quantum Algorithms

A Distributed Learning Scheme for Variational Quantum Algorithms

Federated Learning Under Importance Sampling

Federated learning encapsulates distributed learning strategies that are managed by a central unit. Since it relies on using a selected number of agents at each iteration, and since each agent, in turn, taps into its local data, it is only natural to study optimal sampling policies for selecting agents and their data in federated learning implementations. Usually, only uniform sampling schemes are used. However, in this work, we examine the effect of importance sampling and devise schemes for sampling agents and data non-uniformly guided by a performance measure. We find that in schemes involving sampling without replacement, the performance of the resulting architecture is controlled by two factors related to data variability at each agent, and model variability across agents. We illustrate the theoretical findings with experiments on simulated and real data and show the improvement in performance that results from the proposed strategies.

Predictive UAV Base Station Deployment and Service Offloading With Distributed Edge Learning

In modern networks, edge computing will be responsible for processing and learning from the critical network- and user-generated data, such as wireless link usage, mobility information, application requests, and many others. The presence of Artificial Intelligence-based (AI) applications at the edge of the network will enable the network to predict necessary user behavior and its impact on network infrastructure, such as base station overloading. One of the main strategies for offloading users and base stations is to deploy UAV base stations, or flying base stations, which can dynamically provide service and connectivity. In this article, we introduce a framework for distributed learning over Multi-access Edge Computing (MEC), which manages data applications in a fully distributed setting across edge servers, thus reducing the cost of collecting user information in a centralized server. We couple the proposed distributed learning with a novel similarity metric for user trajectories, which can aggregate neural network models with similar costs as other model aggregation techniques. However, the aggregation technique can achieve much higher accuracy. Furthermore, we apply the proposed distributed learning scheme to manage and deploy flying base stations to areas that experience high demand or poor user connectivity, thus optimizing connectivity in terms of user satisfaction, delay, and network throughput.