Reduce Communication Overhead Research Articles

Fed-MPS: Federated learning with local differential privacy using model parameter selection for resource-constrained CPS

In Cyber-Physical Systems (CPS), distributed learning is essential for efficiently handling complex tasks when sufficient resources are available. However, when resources are limited, traditional distributed learning struggles to complete even simple tasks and presents a risk of privacy leakage. As a promising distributed learning paradigm, federated learning only requires the client to send the trained model to the server instead of private data, thereby preserving the client’s privacy to some extent. However, with the rapid development of artificial intelligence technology, attack methods such as inference attacks still cause privacy leakage for clients participating in federated learning. Moreover, due to its distributed learning nature, federated learning cannot escape the dilemma of model accuracy being constrained by resources. To address the aforementioned problems, this paper proposes a Federated local differential privacy scheme using Model Parameter Selection, named Fed-MPS, for resource-constrained CPS. Specifically, to resolve the issue of limited CPS resources, Fed-MPS adopts an update direction consistency-based parameter selection algorithm in federated learning to extract parameters that enhance model accuracy for subsequent training, thereby improving the final model accuracy and reducing communication overhead. Furthermore, Fed-MPS applies the local differential privacy mechanism to further enhance clients’ privacy. By adding noise only to the chosen parameters, the privacy budget is significantly reduced while ensuring model accuracy. Through privacy analysis, we prove that the proposed Fed-MPS scheme satisfies (ϵ,δ)−DP. Additionally, convergence analysis guarantees that Fed-MPS will converge to the global optimum with a convergence ratio of O(1T2) within T rounds of federated learning. Extensive experiments on prominent benchmark datasets Cifar10, Mnist, and FashionMNIST demonstrate that, compared with baseline schemes, the proposed Fed-MPS provides higher model accuracy for CPS under resource constraints.

Key Backup and Recovery for Resilient DID Environment

This paper delves into the advantages of authentication algorithms employing self-sovereign identity, highlighting a reduced communication overhead and the elimination of single points of failure. However, it acknowledges the vulnerability of digital wallets to real-world issues like loss or theft. To address these challenges, we propose an efficient key backup and recovery protocol based on the FROST threshold signature algorithm. This protocol involves trusted third parties and backup devices, ensuring secure secret key sharing and rapid key recovery. Performance evaluations, including key recovery time, demonstrate the protocol’s efficiency and reliability, bolstering the overall robustness of self-sovereign identity systems.

Smart meter-based energy consumption forecasting for smart cities using adaptive federated learning

Forecasting short-term residential energy consumption is critical in modern decentralized power systems. Deep learning-based prediction methods that can handle the high variability of residential electrical loads have made models more accurate. On the other hand, these methods need a lot of sensitive information about how much people use something gathered centrally to train a forecasting model. This is not good for privacy and scalability. Moreover, models may become less accurate over time due to changing conditions. In this work, we propose a framework for energy consumption forecasting that exploits adaptive learning, federated learning, and edge computing concepts. A central server aggregates numerous long short-term memory (LSTM) models that users at various locations train with their energy consumption data to create a generalized model that uses adaptive learning to detect data drifts and enhance forecasting at the edge layer. Our findings show that adaptive federated learning performs better than centralized learning while preserving privacy, reducing communication overhead, lowering the forecast error rate by 8%, and decreasing the training time by approximately 80%.

CVDMARL: A Communication-Enhanced Value Decomposition Multi-Agent Reinforcement Learning Traffic Signal Control Method

Effective traffic signal control (TSC) plays an important role in reducing vehicle emissions and improving the sustainability of the transportation system. Recently, the feasibility of using multi-agent reinforcement learning technology for TSC has been widely verified. However, the process of mapping road network states onto actions has encountered many challenges, due to the limited communication between agents and the partial observability of the traffic environment. To address this problem, this paper proposes a communication-enhancement value decomposition, multi-agent reinforcement learning TSC method (CVDMARL). The model combines two communication methods: implicit and explicit communication, decouples the complex relationships among the multi-signal agents through the centralized-training and decentralized-execution paradigm, and uses a modified deep network to realize the mining and selective transmission of traffic flow features. We compare and analyze CVDMARL with six different baseline methods based on real datasets. The results show that compared to the optimal method MN_Light, among the baseline methods, CVDMARL’s queue length during peak hours was reduced by 9.12%, the waiting time was reduced by 7.67%, and the convergence algebra was reduced by 7.97%. While enriching the information content, it also reduces communication overhead and has better control effects, providing a new idea for solving the collaborative control problem of multi-signalized intersections.

Load-aware task migration algorithm toward adaptive load balancing in Edge Computing

The rapid advancement of the Internet of Things (IoT) is leading to more and more devices joining the network to interact with information, which requires improving the performance of IoT applications to accommodate more data, faster response times, and more complex tasks. Edge computing, as a new computing paradigm, brings resource contention and load imbalance while reducing communication overhead and task latency. This paper addresses the workload distribution challenge in edge networks, aiming to optimize resource utilization and thereby enhance IoT application performance. To achieve this goal, we present the Load-aware Task Migration (LATM) algorithm. Firstly, we present a load state detection model that captures edge nodes’ workloads and dynamically classifies them according to their resource requirements. We then propose an innovative optimization problem, transforming task migration into a weighted tripartite graph matching problem. This problem leverages the Kuhn–Munkres(KM) task migration algorithm to attain the optimal matching between tasks and nodes. Finally, we assess the algorithm’s performance through experimental simulation. The experimental results underscore the algorithm’s substantial potential in reducing task response times, task execution times, load balance, and enhancing resource utilization.

A multi-objective approach for communication reduction in federated learning under devices heterogeneity constraints

Federated learning is a paradigm that proposes protecting data privacy by sharing local models instead of raw data during each iteration of model training. However, these models can be large, with many parameters, provoking a substantial communication cost and having a notable environmental impact. Reducing communication overhead is paramount but conflictual to maintaining the model’s accuracy. Most research has dealt with the different factors influencing communication reduction separately without addressing their correlations. Moreover, most of them do not consider the heterogeneity of clients’ hardware. Finding the optimal configuration to fulfil all these training aspects can become intractable for classical techniques. This work explores the add-in that multi-objective evolutionary algorithms can provide for solving the communication overhead problem while achieving high accuracy. We do this by 1) realistically modelling and formulating this task as a multi-objective problem by considering the devices’ heterogeneity, 2) including all the communication-triggering aspects, and 3) applying a multi-objective evolutionary algorithm with an intensification operator to solve the problem. A simulated client–server architecture of four devices with four different processing speeds is studied. Both fully connected and convolutional neural network models are investigated with 33,400 and 887,530 weights, respectively. The experiments are performed using the MNIST and Fashion-MNIST datasets. A comparison is made between three approaches using an extensive set of metrics. Results prove that our approach obtains solutions with better accuracy than the full-communication setting and other methods while getting reductions in communications by around 1,000 times in most cases and up to 10,000 times in some cases compared to the maximum communication setting.

Battery-aware federated learning: Challenges and solutions

Smartphone battery life is a pivotal factor in consumers' purchasing decisions. Recent years have witnessed a surge in studies focusing on smartphone energy management, with data-driven energy management systems offering solutions to prolong battery life. Federated Learning (FL) emerges as a promising distributed learning algorithm, enabling wireless devices to upload locally trained models, fostering collaborative learning without exposing sensitive data. This paper explores the FL process, particularly the Federated Averaging (FedAvg) approach, which excels in scenarios with homogeneous data. In the era of burgeoning data generation, traditional cloud computing systems face limitations, driving the adoption of Edge Computing (EC), which processes data closer to its source, enhancing response times. To make FL efficient for e-commerce, resource constraints must be addressed. This involves techniques like local updates and model compression, which reduce communication overhead. However, FL brings challenges related to data distribution heterogeneity and privacy concerns. Solutions like differential privacy, encryption, and access control are discussed. In conclusion, this paper presents an overview of smartphone battery life, data-driven energy management, and the potential of FL, emphasizing its relevance in the age of EC. By addressing resource limitations and privacy issues, FL holds promise for efficient data processing.

Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillation

Federated learning (FL) is a privacy-preserving machine learning paradigm in which the server periodically aggregates local model parameters from cli ents without assembling their private data. Constrained communication and personalization requirements pose severe challenges to FL. Federated distillation (FD) is proposed to simultaneously address the above two problems, which exchanges knowledge between the server and clients, supporting heterogeneous local models while significantly reducing communication overhead. However, most existing FD methods require a proxy dataset, which is often unavailable in reality. A few recent proxy-data-free FD approaches can eliminate the need for additional public data, but suffer from remarkable discrepancy among local knowledge due to client-side model heterogeneity, leading to ambiguous representation on the server and inevitable accuracy degradation. To tackle this issue, we propose a proxy-data-free FD algorithm based on distributed knowledge congruence (FedDKC). FedDKC leverages well-designed refinement strategies to narrow local knowledge differences into an acceptable upper bound, so as to mitigate the negative effects of knowledge incongruence. Specifically, from perspectives of peak probability and Shannon entropy of local knowledge, we design kernel-based knowledge refinement (KKR) and searching-based knowledge refinement (SKR) respectively, and theoretically guarantee that the refined-local knowledge can satisfy an approximately-similar distribution and be regarded as congruent. Extensive experiments conducted on three common datasets demonstrate that our proposed FedDKC significantly outperforms the state-of-the-art on various heterogeneous settings while evidently improving the convergence speed.

Asynchronous Federated Learning: Methods and applications

Federated Learning (FL) is a distributed alternative to traditional machine learning frameworks that computes a global model on a centralized aggregation server according to the parameters of local models, which can address the privacy leakage problem caused by collecting sensitive data from local devices. However, the classic FL methods with synchronous aggregation strategies, in many cases, shall suffer from limitations in resource utilization due to the need to wait for slower devices (stragglers) to aggregate during each training epoch. In addition, the accuracy of the global model can be affected by the uneven distribution of data among unreliable devices in real-world scenarios. Therefore, many Asynchronous Federated Learning (AFL) methods have been developed on many occasions to improve communication efficiency, model performance, privacy, and security. This article elaborates on the existing research on AFL and its applications in many areas. The paper first introduces the concept and development of FL, and then discusses in detail the related work and main research directions of AFL, including dealing with stragglers, staleness, communication efficiency between devices, and privacy and scalability issues. Then, this paper also explores the application of AFL in different fields, especially in the fields of mobile device edge computing, Internet of Things devices, and medical data analysis. Finally, the article gives some outlook on future research directions and believes that it is necessary to design efficient asynchronous optimization algorithms, reduce communication overhead and computing resource usage, and explore new data privacy protection methods.

A Smart Contract-based Blockchain Solution in IoT Networks

The emergence and growing use of advanced technologies has opened up new possibilities for addressing the security challenges of resource-constrained IoT net- works. As IoT devices exchange sensitive data, secure key management is essential for IoT network security, particularly during the key revocation phase. However, current IoT key management solutions require improvements due to the resource limitations of IoT devices. Despite these limitations, existing key revocation solutions still have several areas for improvement, including high communication overheads. Therefore, a decentralized and efficient solution is necessary to address these issues in IoT networks, with a focus on security. This paper proposes a new solution for key revocation based on Blockchain technology using smart contracts to minimize communication overhead and energy consumption in IoT networks. The paper presents a security and performance analysis to assess its correctness. The results indicate that our proposal outperforms other solutions by having a reduced communication overhead of 93.55%, 91.87%, and 99.75% compared to other solutions during the compromising, leaving, and draining cases, respectively. This demonstrates that our solution is efficient and suitable for IoT networks.

Research on Privacy Protection in Federated Learning Combining Distillation Defense and Blockchain

Traditional federated learning addresses the data security issues arising from the need to centralize client datasets on a central server for model training. However, this approach still poses privacy protection risks. For instance, central servers cannot verify privacy leaks resulting from poisoning attacks by malicious clients. Additionally, adversarial sample attacks can infer specific samples from the original data by testing the local models on client devices. This paper proposes a federated learning privacy protection method combining distillation defense technology with blockchain architecture. The method utilizes distillation defense technology to reduce the sensitivity of client devices participating in federated learning to perturbations and enhance their ability to resist adversarial sample attacks locally. This not only reduces communication overhead and improves learning efficiency but also enhances the model’s generalization ability. Furthermore, the method leverages the “decentralized” nature of blockchain architecture as a trusted record-keeping mechanism to audit information interactions among clients and shared model parameters. This addresses privacy leakage issues resulting from poisoning attacks by some clients during the model construction process. Simulation experiment results demonstrate that the proposed method, compared with traditional federated learning, ensures model convergence, detects malicious clients, and improves the participation level of highly reputable clients. Moreover, by reducing the sensitivity of local clients to perturbations, it enhances their ability to effectively resist adversarial sample attacks.

SOC-SOH Estimation and Balance Control Based on Event-Triggered Distributed Optimal Kalman Consensus Filter

The inconsistency in state-of-charge (SOC) for electric vehicle batteries will cause component damage and lifespan reduction of batteries. Meanwhile, the consistency in the state-of-health (SOH) also negatively influences the consensus of SOC. To ensure the consensuses of SOC and SOH simultaneously, this paper introduces an innovative distributed optimal Kalman consensus filter (KCF) approach to battery management systems. In addition, at the stage where sensors transmit information to each other, a new event-triggering mechanism (ETM) based on dynamic information is proposed to reduce communication overhead effectively. Theoretical analysis verifies the optimality of the algorithm. By numerical simulations, the proposed event-triggered distributed optimal KCF (ET-DOKCF) method can improve the performance of SOC-SOH estimation and save communication resources.

Asynchronous Robust Aggregation Method with Privacy Protection for IoV Federated Learning

Due to the wide connection range and open communication environment of internet of vehicle (IoV) devices, they are susceptible to Byzantine attacks and privacy inference attacks, resulting in security and privacy issues in IoV federated learning. Therefore, there is an urgent need to study IoV federated learning methods with privacy protection. However, the heterogeneity and resource limitations of IoV devices pose significant challenges to the aggregation of federated learning model parameters. Therefore, this paper proposes an asynchronous robust aggregation method with privacy protection for federated learning in IoVs. Firstly, we design an asynchronous grouping robust aggregation algorithm based on delay perception, combines intra-group truth estimation with inter-group delay aggregation, and alleviates the impact of stragglers and Byzantine attackers. Then, we design a communication-efficient and security enhanced aggregation protocol based on homomorphic encryption, to achieve asynchronous group robust aggregation while protecting data privacy and reducing communication overhead. Finally, the simulation results indicate that the proposed scheme could achieve a maximum improvement of 41.6% in model accuracy compared to the baseline, which effectively enhances the training efficiency of the model while providing resistance to Byzantine attacks and privacy inference attacks.

Energy Trading and Optimum Scheduling for Microgrids Using Multiple Agents Based DL Approach

In microgrids, policy decisions about electricity trading and scheduling are of paramount importance. These decisions are influenced by various factors, including the microgrid’s infrastructure, energy production and demand, and the competitiveness of the energy market. However, due to the intricate nature of these elements, determining the optimal strategy for a microgrid can be a challenging task. To address this challenge without the need for complex system modeling, we introduce a novel approach using Multi-Agent Reinforcement Learning (MARL) augmented with an attention mechanism. In this approach, each microgrid is treated as a distinct agent, and through interactions with other agents, they learn how to effectively coordinate the utilization of energy resources and engage in energy trading. To facilitate effective agent training, we employ an attention mechanism that enables agents to selectively focus on pertinent background information. Once trained, each agent can make control decisions based solely on its local knowledge. This not only safeguards the privacy of individual microgrids but also reduces communication overhead, making decentralized control feasible. We implement this approach in MATLAB R2020a to create a simulation environment and assess its performance. Our experimental results indicate that our proposed strategy significantly reduces the operational costs of microgrids in comparison to conventional methods.

Federated Generative-Adversarial-Network-Enabled Channel Estimation

Accurately estimating channel state information is essential for meeting the quality-of-service requirements of modern applications and scenarios. Deep learning techniques have proven effective in acquiring channel conditions with low pilot overhead in massive connectivity scenarios. However, accessing channel data brings new challenges related to transmission overhead, privacy concerns, scalability, heterogeneous network support, and adaptability to dynamic environments. We propose a federated generative-adversarial-network-enabled channel estimator to address these challenges. We refine the coarse least-squares estimation results for their low complexity and fast convergence. To ensure accuracy, we designed a double U-shaped network. The Lipschitz continuous function is applied to discriminators for spectral normalization. We then propose a federated learning framework to utilize the training process. The local generator parameters are updated at the center, reducing communication overhead and privacy concerns. To deal with nonindependent and identically distributed datasets, the discriminators dynamically push away the predictions by dynamic regularization to obtain a more robust aggregated generative model at the center. Furthermore, we propose a motivation scheme that benefits users participating in the training process, encouraging them to join and take advantage of edge/cloud computing capabilities. Numerical results demonstrate that the proposed federated generative adversarial network-enabled channel estimator provides high estimation accuracy and reduces the burden on pilots. The proposed dynamic regularization terms and motivation scheme boost performance efficiently with low communication cost and high participation.

A survey on state-of-the-art experimental simulations for privacy-preserving federated learning in intelligent networking

<abstract> <p>Federated learning (FL) provides a collaborative framework that enables intelligent networking devices to train a shared model without the need to share local data. FL has been applied in communication networks, which offers the dual advantage of preserving user privacy and reducing communication overhead. Networking systems and FL are highly complementary. Networking environments provide critical support for data acquisition, edge computing capabilities, round communication/connectivity, and scalable topologies. In turn, FL can leverage capabilities to achieve learning adaptation, low-latency operation, edge intelligence, personalization, and, notably, privacy preservation. In our review, we gather relevant literature and open-source platforms that point out the feasibility of conducting experiments at the confluence of FL and intelligent networking. Our review is structured around key sections, including the introduction of FL concepts, the background of FL applied in networking, and experimental simulations covering networking for FL and FL for networking. Additionally, we delved into case studies showcasing FL potential in optimizing state-of-the-art network optimization objectives, such as learning performance, quality of service, energy, and cost. We also addressed the challenges and outlined future research directions that provide valuable guidance to researchers and practitioners in this trending field.</p> </abstract>

Multicenter Hierarchical Federated Learning With Fault-Tolerance Mechanisms for Resilient Edge Computing Networks.

In the realm of federated learning (FL), the conventional dual-layered architecture, comprising a central parameter server and peripheral devices, often encounters challenges due to its significant reliance on the central server for communication and security. This dependence becomes particularly problematic in scenarios involving potential malfunctions of devices and servers. While existing device-edge-cloud hierarchical FL (HFL) models alleviate some dependence on central servers and reduce communication overheads, they primarily focus on load balancing within edge computing networks and fall short of achieving complete decentralization and edge-centric model aggregation. Addressing these limitations, we introduce the multicenter HFL (MCHFL) framework. This innovative framework replaces the traditional single central server architecture with a distributed network of robust global aggregation centers located at the edge, inherently enhancing fault tolerance crucial for maintaining operational integrity amidst edge network disruptions. Our comprehensive experiments with the MNIST, FashionMNIST, and CIFAR-10 datasets demonstrate the MCHFL's superior performance. Notably, even under high paralysis ratios of up to 50%, the MCHFL maintains high accuracy levels, with maximum accuracy reductions of only 2.60%, 5.12%, and 16.73% on these datasets, respectively. This performance significantly surpasses the notable accuracy declines observed in traditional single-center models under similar conditions. To the best of our knowledge, the MCHFL is the first edge multicenter FL framework with theoretical underpinnings. Our extensive experimental results across various datasets validate the MCHFL's effectiveness, showcasing its higher accuracy, faster convergence speed, and stronger robustness compared to single-center models, thereby establishing it as a pioneering paradigm in edge multicenter FL.

A Fast Row-Stochastic Decentralized Method for Distributed Optimization Over Directed Graphs

In this paper, we introduce a fast row-stochastic decentralized algorithm, referred to as FRSD, to solve consensus optimization problems over directed communication graphs. The proposed algorithm only utilizes row-stochastic weights, leading to certain practical advantages in broadcast communication settings over those requiring column-stochastic weights. Under the assumption that each node-specific function is smooth and strongly convex, we show that the FRSD iterate sequence converges with a linear rate to the optimal consensus solution. In contrast to the existing methods for directed networks, FRSD enjoys linear convergence without employing a gradient tracking (GT) technique explicitly, rather it implements GT implicitly with the use of a novel momentum term, which leads to a significant reduction in communication and storage overhead for each node when FRSD is implemented for solving high-dimensional problems over small-to-medium scale networks. In the numerical tests, we compare FRSD with other state-of-the-art methods, which use row-stochastic and/or column-stochastic weights.

Bases selection with pseudo-random functions in BB84 scheme

Because the spectrum of services available in modern telecommunication networks is constantly expanding, security has become increasingly important. Simultaneously, in an era of constant progress in mathematics and computing, the security of existing cryptographic solutions becomes questionable. Quantum Key Distribution (QKD) is a promising secret key agreement primitive that enables long-awaited practical Information-Theoretical Secure (ITS) communications. The key generation rate, however, is one of the limitations of its widespread application to secure high throughput data flows. This paper addresses the aforementioned limitation by employing perfectly correlated bases selection defined by the output of Pseudo-Random Functions based on the keyed-Hash Message Authentication Code construction. In theory, the proposed variant of the BB84 scheme is ITS, reduces memory requirements, and reduces communication overhead during the post-processing stage. It can benefit QKD networks as a service by increasing capacity and accommodating users with varying security needs.

Privacy-preserving intelligent content-caching scheme in heterogeneous aerial access networks

Heterogeneous aerial access networks (AANs), comprising a hierarchical model of a low Earth orbit (LEO) satellite constellation and multiple high-altitude platforms (HAPs), provide a radio access medium from the sky to enhance the service experience of ground terminals (GTs) in remote areas. In this scenario, aerial edge caching addresses the challenge of delivering on-demand content requests to GTs. Although existing caching schemes for AANs have improved caching efficiency, they often overlook important concerns related to privacy preservation and overhead. This paper proposes an intelligent, proactive content-caching scheme to maximize cache efficiency while ensuring privacy preservation for GTs. The proposed caching scheme employs hierarchical federated learning (HierFL) that involves GTs, HAPs, and LEO satellites in heterogeneous AANs. In particular, the proposed HierFL-based proactive content-caching (HierFL-PCC) scheme leverages a deep neural network employing multiple linear regression to predict the dynamically changing content popularity of GTs, maximizing the cache efficiency. The HierFL-PCC also reduces the delay and overhead associated with content delivery. Additionally, the proposed HierFL-PCC scheme reduces communication overhead in model training owing to the hierarchical learning architecture. Simulation results demonstrate that the proposed HierFL-PCC scheme exhibits improvement of about 13%, 43%, 178%, and 457% in cache efficiency, and reduces average content delivery delay by approximately 10%, 20%, 31%, and 37%, along with an improvement of around 89% in training overhead, respectively, compared to the cloud-based FAVG, LFU, LRU, and Random caching schemes.