Centralized Training Research Articles

Privacy-preserving federated learning for transportation mode prediction based on personal mobility data

• We build a federated learning framework for transportation mode prediction based on personal mobility data, which can benefit the public transportation data analysis and resources management. • Our framework can preserve user’s privacy by keeping all user’s personal mobility data in local without uploading to central nodes. • Our framework can train accurate deep neural network models for transportation mode prediction that is close to the centralized training performance. Personal daily mobility trajectories/traces like Google Location Service integrates many valuable information from individuals and could benefit a lot of application scenarios, such as pandemic control and precaution, product recommendation, customized user profile analysis, traffic management in smart cities, etc. However, utilizing such personal mobility data faces many challenges since users’ private information, such as home/work addresses, can be unintentionally leaked. In this work, we build an FL system for transportation mode prediction based on personal mobility data. Utilizing FL-based training scheme, all user’s data are kept in local without uploading to central nodes, providing high privacy preserving capability. At the same time, we could train accurate DNN models that is close to the centralized training performance. The resulted transportation mode prediction system serves as a prototype on user’s traffic mode classification, which could potentially benefit the transportation data analysis and help make wise decisions to manage public transportation resources.

Communicational and Computational Efficient Federated Domain Adaptation

The emerging paradigm of Federated Learning enables mobile users to collaboratively train a model without disclosing their privacy-sensitive data. Nevertheless, data collected from different mobile users may not be independent and identically distributed. Thus directly applying the trained model to a new mobile user usually leads to performance degradation due to the so-called domain shift. Unsupervised Domain Adaptation is an effective technique to mitigate domain shift and transfer knowledge from labeled source domains to the unlabeled target domain. In this article, we design a Federated Domain Adaptation framework that extends Domain Adaptation with the constraints of Federated Learning to train a model for the target domain and preserve the data privacy of all the source and target domains. As mobile devices usually have limited computation and communication capabilities, we design a set of optimization methods that significantly enhance our framework’s computation and communication efficiency, making it more friendly to resource-constrained edge devices. Evaluation results on three datasets show that our framework has comparable performance with the standard centralized training approach, and the optimization methods can reduce the computation and communication overheads by up to two orders of magnitude.

MASAC-based confrontation game method of UAV clusters

With the improvement in the intelligence level of UAVs and the development of cluster control technology, the intelligent decision-making method of UAV cluster confrontation will become the key technology of UAV combat in the future. The UAV cluster confrontation learning environment is of complex characteristics of high dimension, non-linearity, incomplete information, continuous action space, and so on. Recently, artificial intelligence technology represented by deep learning and reinforcement learning has made a great breakthrough. Deep reinforcement learning has shown a great ability to solve intelligent decision-making problems in complex environments. In this paper, enlightened by the multi-agent centralized training distributed execution framework and the idea of maximum policy entropy, we propose a deep reinforcement learning method of the multi-agent soft actor-critic (MASAC) with incomplete information. A game model for UAV cluster confrontation based on multi-agent deep reinforcement learning is established, and a continuous space multiple UAV combat environment is constructed. Simulation experiments are performed on the asymmetric confrontation of UAV clusters with red and blue teams. The experimental results show that MASAC outperforms the existing popular multi-agent deep reinforcement learning methods, making the game players converge to a higher return equilibrium point of the game. Moreover, the convergence of MASAC is investigated and analyzed extensively, and the results show that MASAC is of good convergence and stability, ensuring the practicability of MASAC in intelligent decision-making regarding UAV cluster confrontation.

Personalized Retrogress-Resilient Federated Learning Toward Imbalanced Medical Data.

Clinically oriented deep learning algorithms, combined with large-scale medical datasets, have significantly promoted computer-aided diagnosis. To address increasing ethical and privacy issues, Federated Learning (FL) adopts a distributed paradigm to collaboratively train models, rather than collecting samples from multiple institutions for centralized training. Despite intensive research on FL, two major challenges are still existing when applying FL in the real-world medical scenarios, including the performance degradation (i.e., retrogress) after each communication and the intractable class imbalance. Thus, in this paper, we propose a novel personalized FL framework to tackle these two problems. For the retrogress problem, we first devise a Progressive Fourier Aggregation (PFA) at the server side to gradually integrate parameters of client models in the frequency domain. Then, at the client side, we design a Deputy-Enhanced Transfer (DET) to smoothly transfer global knowledge to the personalized local model. For the class imbalance problem, we propose the Conjoint Prototype-Aligned (CPA) loss to facilitate the balanced optimization of the FL framework. Considering the inaccessibility of private local data to other participants in FL, the CPA loss calculates the global conjoint objective based on global imbalance, and then adjusts the client-side local training through the prototype-aligned refinement to eliminate the imbalance gap with such a balanced goal. Extensive experiments are performed on real-world dermoscopic and prostate MRI FL datasets. The experimental results demonstrate the advantages of our FL framework in real-world medical scenarios, by outperforming state-of-the-art FL methods with a large margin. The source code is available at https://github.com/CityU-AIM-Group/PRR-Imbalancehttps://github.com/CityU-AIM-Group/PRR-Imbalance.

A Threefold of Lutheran Theological Institutions in the Midst of Theological Education in South Africa: 1960–1993

This article explores the undocumented history of theological institutions belonging to the Lutheran Church in Southern Africa (LCSA) which contributed to South African theological education. The establishment of these institutions was initially managed by the mission societies of the LCSA and later the regional leadership. This development created the necessity for centralised theological training since 1910. Due to the political landscape of South Africa, theological education was adversely affected. The LCSA looked at ways to sustain theological education and as a result of the church’s passion, Lutheran Theological education was established. However, the LCSA worked in collaboration with other Christian denominations to achieve its ecumenical vision and establish a solid foundation of theological education.

Multi-agent deep reinforcement learning for online request scheduling in edge cooperation networks

Edge computing as a complementary paradigm of cloud computing has gained more attention by providing mobile users with diversified services at the network edge. However, the increasingly complex mobile applications put a heavier load on edge networks. It is challenging to provide concurrency requests with high-quality service processing, especially when the edge networks are dynamically changing. To address the above issues, this paper investigates the online concurrent user requests scheduling optimization problem in edge cooperation networks. We model it as an online multi-stage decision-making problem, where requests are divided into a group of independent and logically related sub-tasks. We proposed a centralized training distributed execution based multi-agent deep reinforcement learning technique to realize the implicit cooperation scheduling decision-making policy learning among edge nodes. At the centralized training stage of the proposed mechanism, a value-decomposition-based policy learning technique is adopted to improve the long-term system performance, while at the distributed execution stage, only local environment status information is needed for each edge node to make the request scheduling decision. Extensive experiments are conducted, and simulation results demonstrate that the proposed mechanism outperforms other request scheduling mechanisms in reducing the long-term average system delay and energy consumption while improving the throughput rate of the system.

An adaptive active power rolling dispatch strategy for high proportion of renewable energy based on distributed deep reinforcement learning

In this article, an adaptive active power rolling dispatch strategy based on distributed deep reinforcement learning is proposed to deal with the uncertainty of high-proportioned renewable energy. For each agent, by using recurrent neural network layers and graph attention layers in its network structure, we aim to improve the generalization ability of the multiple agents in active power flow control. Furthermore, a regional graph attention network algorithm, which can effectively help agents aggregate the regional information of their neighborhood, is proposed to improve the information capture ability of agents. We adopt the structure of ‘centralized training, distributed execution’ to help agents improve the effectiveness of proposed methods in dynamic environments. The case studies demonstrate that the proposed algorithm can help multi-agents learn effective active power control strategies. Each agent has a strong generalization ability in terms of time granularity and network topology. We expect that such an approach can improve the practicability and adaptability of distributed AI method on power system control issues.

Multi-agent deep reinforcement learning-based coordination control for grid-aware multi-buildings

In commercial buildings, Heat, Ventilation, and Air Conditioning (HVAC) systems account for about 40–50 % of total electricity usage, contributing to an economic burden on building operators. Furthermore, increasing amounts of distributed generation can bring challenges and opportunities for voltage regulation in low voltage distribution networks. In this paper, we tend to develop intelligent management to save the electricity bills of HVAC systems in multiple multi-zone buildings while relieving the stress of voltage regulation across the network. However, it is challenging to achieve the above aims due to the existence of parameter uncertainties (e.g., electricity load, outdoor temperature, photovoltaic generation, etc.), a sizeable continuous decision space, unknown thermal dynamics model, and distribution network topology, and a non-convex multi-objective function. In this context, a novel model-free multi-agent deep reinforcement learning (MADRL)-based multi-building control algorithm is proposed to achieve building-side and grid-level objectives. The proposed method adopts a centralized training and decentralized execution framework while integrating an attention mechanism to ease training and preserve privacy. This also enables the agent to achieve control purposes based only on local measurements, reducing communication cost. Simulation results based on real-world data verify that the proposed method can achieve real-time physical-model-free control of multi-buildings to tackle fast fluctuations of voltage and temperature caused by the uncertain external factors while being advantageous over other two MADRL methods. Additionally, comparison analysis on untouched datasets illustrates that the proposed method achieves similar results and better computation performance with the perfect physical-model-based approach.

A qualitative exploration of burnout prevention and reduction strategies for general practice registrars.

Burnout interventions for trainee doctors have typically overlooked specialty-specific factors as well as presumed centralised training models. The aim of this study was toexplore stakeholders' views of burnout interventions within Australian general practice training, where both factors are pertinent. Forty-seven trainees, supervisors, medical educators and program coordinators from a regional training organisation participated in interviews and focus groups. Template analysis, informed by grounded theory, was used. Strategies were identified for registrars (eg prioritising replenishing activities), practices (eg providing psychological supports), training organisations (egengaging with trainees) and the medical system (egdestigmatising poor wellbeing). Ineffective strategies (egtokenistic interventions) were also highlighted, albeitto a lesser extent. Stakeholders reiterated that burnout prevention and management require both individual and organisational-level change. Specific strategies for practices (eg consideration of workload issues and supports) and training organisations (eg structural changes to training requirements) are delineated, which, in combination, may enhance intervention efficacy within a decentralised training system.

Deep multiagent reinforcement learning: challenges and directions

This paper surveys the field of deep multiagent reinforcement learning (RL). The combination of deep neural networks with RL has gained increased traction in recent years and is slowly shifting the focus from single-agent to multiagent environments. Dealing with multiple agents is inherently more complex as (a) the future rewards depend on multiple players’ joint actions and (b) the computational complexity increases. We present the most common multiagent problem representations and their main challenges, and identify five research areas that address one or more of these challenges: centralised training and decentralised execution, opponent modelling, communication, efficient coordination, and reward shaping. We find that many computational studies rely on unrealistic assumptions or are not generalisable to other settings; they struggle to overcome the curse of dimensionality or nonstationarity. Approaches from psychology and sociology capture promising relevant behaviours, such as communication and coordination, to help agents achieve better performance in multiagent settings. We suggest that, for multiagent RL to be successful, future research should address these challenges with an interdisciplinary approach to open up new possibilities in multiagent RL.

Cooperative multi-target hunting by unmanned surface vehicles based on multi-agent reinforcement learning

To solve the problem of multi-target hunting by an unmanned surface vehicle (USV) fleet, a hunting algorithm based on multi-agent reinforcement learning is proposed. Firstly, the hunting environment and kinematic model without boundary constraints are built, and the criteria for successful target capture are given. Then, the cooperative hunting problem of a USV fleet is modeled as a decentralized partially observable Markov decision process (Dec-POMDP), and a distributed partially observable multi-target hunting Proximal Policy Optimization (DPOMH-PPO) algorithm applicable to USVs is proposed. In addition, an observation model, a reward function and the action space applicable to multi-target hunting tasks are designed. To deal with the dynamic change of observational feature dimension input by partially observable systems, a feature embedding block is proposed. By combining the two feature compression methods of column-wise max pooling (CMP) and column-wise average-pooling (CAP), observational feature encoding is established. Finally, the centralized training and decentralized execution framework is adopted to complete the training of hunting strategy. Each USV in the fleet shares the same policy and perform actions independently. Simulation experiments have verified the effectiveness of the DPOMH-PPO algorithm in the test scenarios with different numbers of USVs. Moreover, the advantages of the proposed model are comprehensively analyzed from the aspects of algorithm performance, migration effect in task scenarios and self-organization capability after being damaged, the potential deployment and application of DPOMH-PPO in the real environment is verified.

Cyber Threat Intelligence Sharing Scheme Based on Federated Learning for Network Intrusion Detection

The uses of machine learning (ML) technologies in the detection of network attacks have been proven to be effective when designed and evaluated using data samples originating from the same organisational network. However, it has been very challenging to design an ML-based detection system using heterogeneous network data samples originating from different sources and organisations. This is mainly due to privacy concerns and the lack of a universal format of datasets. In this paper, we propose a collaborative cyber threat intelligence sharing scheme to allow multiple organisations to join forces in the design, training, and evaluation of a robust ML-based network intrusion detection system. The threat intelligence sharing scheme utilises two critical aspects for its application; the availability of network data traffic in a common format to allow for the extraction of meaningful patterns across data sources and the adoption of a federated learning mechanism to avoid the necessity of sharing sensitive users’ information between organisations. As a result, each organisation benefits from the intelligence of other organisations while maintaining the privacy of its data internally. In this paper, the framework has been designed and evaluated using two key datasets in a NetFlow format known as NF-UNSW-NB15-v2 and NF-BoT-IoT-v2. In addition, two other common scenarios are considered in the evaluation process; a centralised training method where local data samples are directly shared with other organisations and a localised training method where no threat intelligence is shared. The results demonstrate the efficiency and effectiveness of the proposed framework by designing a universal ML model effectively classifying various benign and intrusive traffic types originating from multiple organisations without the need for inter-organisational data exchange.

Cyber Physical System Based Smart Healthcare System with Federated Deep Learning Architectures with Data Analytics

Data shared between hospitals and patients using mobile and wearable Internet of Medical Things (IoMT) devices raises privacy concerns due to the methods used in training. the development of the Internet of Medical Things (IoMT) and related technologies and the most current advances in these areas The Internet of Medical Things and other recent technological advancements have transformed the traditional healthcare system into a smart one. improvement in computing power and the spread of information have transformed the healthcare system into a high-tech, data-driven operation. On the other hand, mobile and wearable IoMT devices present privacy concerns regarding the data transmitted between hospitals and end users because of the way in which artificial intelligence is trained (AI-centralized). In terms of machine learning (AI-centralized). Devices connected to the IoMT network transmit highly confidential information that could be intercepted by adversaries. Due to the portability of electronic health record data for clinical research made possible by medical cyber-physical systems, the rate at which new scientific discoveries can be made has increased. While AI helps improve medical informatics, the current methods of centralised data training and insecure data storage management risk exposing private medical information to unapproved foreign organisations. New avenues for protecting users' privacy in IoMT without requiring access to their data have been opened by the federated learning (FL) distributive AI paradigm. FL safeguards user privacy by concealing all but gradients during training. DeepFed is a novel Federated Deep Learning approach presented in this research for the purpose of detecting cyber threats to intelligent healthcare CPSs.

Distributed Offloading for Cooperative Intelligent Transportation Under Heterogeneous Networks

With the rapid advancement of the Internet of Vehicles and artificial intelligence (AI) technologies, the cooperative intelligent transportation system (C-ITS) has drawn great attention in recent years. To provide an ultra-reliable, low-latency computation experience of C-ITS, computation offloading is deemed indispensable by working with edge-cloud servers. In this paper, we first investigate a distributed dynamic computation offloading model for multi-access edge computing (MEC) enabled C-ITS under a heterogeneous road network, in which the multiple and heterogeneous computing power sources cooperatively provide computation offloading services for vehicles. Considering the autonomous offloading manner of the vehicles, we formulate the task offloading and computing power allocation as a distributed Stackelberg game, where the MEC servers as the leader to allocate computing resources and manage local energy, and the vehicles as the followers to offload local computation task. Since the observable states in the game is incomplete, the problem of resolving the optimal strategies for each game player is modeled as a partially observable Markov decision process (POMDP) to maximize the long-term cumulative reward. Then we develop a computation offloading algorithm using Stackelberg game-based multi-agent deep deterministic policy gradient (SG-MADDPG), which uses a centralized training and decentralized execution method to learn the optimal computing power allocation and computation offloading policies. Finally, extensive simulations are carried out and show the rationality and effectiveness of the proposed algorithm.

Federated Reinforcement Learning for Decentralized Voltage Control in Distribution Networks

Multi-agent reinforcement learning (MARL) with “centralized training & decentralized execution” framework has been widely investigated to implement decentralized voltage control for distribution networks (DNs). However, a centralized training solution encounters privacy and scalability issues for large-scale DNs with multiple virtual power plants. In this letter, a decomposition & coordination reinforcement learning algorithm is proposed based on a federated framework. This decentralized training algorithm not only enhances scalability and privacy but also has a similar learning convergence with centralized ones.

A deep reinforcement learning based approach for dynamic distributed blocking flowshop scheduling with job insertions

The distributed blocking flowshop scheduling problem (DBFSP) with new job insertions is studied. Rescheduling all remaining jobs after a dynamic event like a new job insertion is unreasonable to an actual distributed blocking flowshop production process. A deep reinforcement learning (DRL) algorithm is proposed to optimise the job selection model, and local modifications are made on the basis of the original scheduling plan when new jobs arrive. The objective is to minimise the total completion time deviation of all products so that all jobs can be finished on time to reduce the cost of storage. First, according to the definitions of the dynamic DBFSP problem, a DRL framework based on multi-agent deep deterministic policy gradient (MADDPG) is proposed. In this framework, a full schedule is generated by the variable neighbourhood descent algorithm before a dynamic event occurs. Meanwhile, all newly added jobs are reordered before the agents make decisions to select the one that needs to be scheduled most urgently. This study defines the observations, actions and reward calculation methods and applies centralised training and distributed execution in MADDPG. Finally, a comprehensive computational experiment is carried out to compare the proposed method with the closely related and well-performing methods. The results indicate that the proposed method can solve the dynamic DBFSP effectively and efficiently.

MIRA: Model-Based Imagined Rollouts Augmentation for Non-Stationarity in Multi-Agent Systems

One of the challenges in multi-agent systems comes from the environmental non-stationarity that policies of all agents are evolving individually over time. Many existing multi-agent reinforcement learning (MARL) methods have been proposed to address this problem. However, these methods rely on a large amount of training data and some of them require agents to intensely communicate, which is often impractical in real-world applications. To better tackle the non-stationarity problem, this article combines model-based reinforcement learning (MBRL) and meta-learning and proposes a method called Model-based Imagined Rollouts Augmentation (MIRA). Based on an environment dynamics model, distributed agents can independently perform multi-agent rollouts with opponent models during exploitation and learn to infer the environmental non-stationarity as a latent variable using the rollouts. Based on the world model and latent-variable inference module, we perform multi-agent soft actor-critic implementation for centralized training and decentralized decision making. Empirical results on the Multi-agent Particle Environment (MPE) have proved that the algorithm has a very considerable improvement in sample efficiency as well as better convergent rewards than state-of-the-art MARL methods, including COMA, MAAC, MADDPG, and VDN.

Synthetic rainfall data generator development through decentralised model training

Recent heavy rainfall-induced flood events, for example in Germany, Australia and USA, have highlighted the relevance of countermeasures in saving human lives and preventing property damage. Newly introduced ML-based flood forecasting methods rely on high-intensity synthetic rainfall events due to the sparsity of their real counterpart. Such synthetic data instances can be produced by precipitation generators trained in an adversarial setting on historical rainfall data. Capturing processes for rainfall data are often highly distributed, with multiple radar stations contributing to a centralised data set. However, data centralisation entails challenges regarding data-stream logistics, data locality, and memory overhead. Distributed Analytics (DA) aims to overcome these challenges through decentralised model training by bringing the algorithm to the data instead of vice versa. In this work, we propose a feasibility study evaluating the applicability of DA on hydrological data. As example of use, we choose the decentralised training of rainfall data generators. We introduce a rainfall generator training procedure relying on Generative Adversarial Networks (GANs) and evaluate two DA algorithms: Federated Learning (FL) and Cyclic Institutional Incremental Learning (CIIL). We compare the resulting training outcomes with the centralised model training (CL) approach and find CIIL performed similarly to CL but less stable, while FL outperformed CL by 7.5%. We conclude that the proven feasibility of FL in our simulated distributed setting lays the groundwork for utilising this approach in realistic environments of grander scale while overcoming potential privacy concerns or logistical challenges in the setting of centralised analytics.

Dynamic power allocation in IIoT based on multi-agent deep reinforcement learning

With the rapidly growing fifth generation (5G) wireless data traffic, the cellular network has gradually become an important mode for the Industrial Internet of Things (IIoT). To give full play to the advantages of cellular network, it is desirable to design the optimal allocation strategy under the condition of limited network resources. In this paper, we consider the problem of allocating downlink power to mobile IIoT devices based on multi-agent deep reinforcement learning (MADRL). To maximize the total system capacity, each base station (BS)-user device (UE) is modeled as a RL agent to learn optimal allocation policy. A centralized training and distributed execution learning framework is proposed, the influence of parameter transmission delay on allocation policy is considered. The designed state space and reward function can adapt to large-scale networks on account of their expandability. Simulation results show the effectiveness and superiority of the proposed method in terms of the system sum rate.

Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

As industry rapidly shifts towards mass personalisation, the need for a decentralised multi-agent system capable of dynamic flexible job shop scheduling (FJSP) is evident. Traditional heuristic and meta-heuristic scheduling methods cannot achieve satisfactory results and have limited application to static environments. Recent Reinforcement Learning (RL) approaches that consider dynamic FJSP, lack flexibility and autonomy as they use a single-agent centralised model, assuming global observability. As such, we propose a Multi-Agent Reinforcement Learning (MARL) system for scheduling dynamically arriving assembly jobs in a robot assembly cell. We applied a Double DQN-based algorithm and proposed a generalised observation, action and reward design for the dynamic FJSP setting. Using a centralised training phase, each agent (i.e., robot) in the assembly cell executes decentralised scheduling decisions based on local observations. Our solution demonstrated improved performance against rule-based heuristic methods, for optimising makespan. We also reported the impact of different observation sizes of each agent on optimisation performance.