Asynchronous Parallel Incremental Block-Coordinate Descent for Decentralized Machine Learning

Federated learning: A cutting-edge survey of the latest advancements and applications

The rapid proliferation of Internet of Things (IoT) devices has revolutionized various industries by enabling smart grids, smart cities, and other applications that rely on seamless connectivity and real‐time data processing. However, this growth has also introduced significant security challenges due to the scale, heterogeneity, and resource constraints of IoT systems. Traditional intrusion detection systems (IDS) often struggle to address these challenges effectively, as they require centralized data collection and processing, which raises concerns about data privacy, communication overhead, and scalability. To address these issues, this paper investigates the application of federated learning for network intrusion detection in IoT environments. We first evaluate a range of machine learning (ML) and deep learning (DL) models, finding that the random forest model achieves the highest classification accuracy. We then propose a federated learning approach that allows distributed IoT devices to collaboratively train ML models without sharing raw data, thereby preserving privacy and reducing communication costs. Experimental results using the UNSW‐NB15 dataset demonstrate that this approach achieves promising outcomes in the IoT context, with minimal performance degradation compared to centralized learning. Our findings highlight the potential of federated learning as an effective, decentralized solution for network intrusion detection in IoT environments, addressing critical challenges, such as data privacy, heterogeneity, and scalability.

A review of on-device machine learning for IoT: An energy perspective

Internet of Things (IoT) is the major technology of the 4 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> industrial revolution in which various types of devices are connected together to work smartly without the intervention of humans. IoT seems to impart a great impact on our social, economic, and commercial lives. IoT applications are converting from smart home and smart me to the smart cities or smart planet. However, the large number of devices interconnected with each other by multi protocols puts the security of IoT networks on the verge of threats. Making the IoT devices more secure is also not feasible because of their limited computational power. Hence, there is a need for advancement in methods to secure IoT networks. Machine Learning (ML) models have been hot topics in security research in past years. As the IoT devices generate tons of data on a daily basis which can be used to train ML algorithms, it could be a reasonable solution to provide security to IoT systems. In this work, the main goal is to provide a broader survey of research works in the IoT security field regarding ML implementation. We briefly described the security issues in IoT networks and their impact on the privacy of important data. We then shed light on different ML algorithms and models and discussed their advantages, disadvantages, and applications in IoT individually. Moreover, the ML models currently working in IoT networks for security purposes are discussed. We also talked about the limitations of using ML models to secure the IoT networks which could provide new future research directions.

This chapter addresses the problem of collaborative Predictive Modelling via in-network processing of contextual information captured in Internet of Things (IoT) environments. In-network predictive modelling allows the computing and sensing devices to disseminate only their local predictive Machine Learning (ML) models instead of their local contextual data. The data center, which can be an Edge Gate- way or the Cloud, aggregates these local ML predictive models to predict future outcomes. Given that communication between devices in IoT environments and a centralised data center is energy consuming and communication bandwidth demanding, the local ML predictive models in our proposed in-network processing are trained using Swarm Intelligence for disseminating only their parameters within the network. We further investigate whether dissemination overhead of local ML predictive models can be reduced by sending only relevant ML models to the data center. This is achieved since each IoT node adopts the Particle Swarm Optimisation algorithm to locally train ML models and then collaboratively with their network neighbours one representative IoT node fuses the local ML models. We provide comprehensive experiments over Random and Small World network models using linear and non-linear regression ML models to demonstrate the impact on the predictive accuracy and the benefit of communication-aware in-network predictive modelling in IoT environments.

Applications running on the Internet of Things, such as the Wireless Sensor and Actuator Networks (WSANs) platform, generally have different quality of service (QoS) requirements. For urgent events, it is crucial that information be reported to the actuator quickly, and the communication cost is the second factor. However, for interesting events, communication costs, network lifetime and time all become important factors. In most situations, these different requirements cannot be satisfied simultaneously. In this paper, an adaptive communication control based on a differentiated delay (ACCDS) scheme is proposed to resolve this conflict. In an ACCDS, source nodes of events adaptively send various searching actuators routings (SARs) based on the degree of sensitivity to delay while maintaining the network lifetime. For a delay-sensitive event, the source node sends a large number of SARs to actuators to identify and inform the actuators in an extremely short time; thus, action can be taken quickly but at higher communication costs. For delay-insensitive events, the source node sends fewer SARs to reduce communication costs and improve network lifetime. Therefore, an ACCDS can meet the QoS requirements of different events using a differentiated delay framework. Theoretical analysis simulation results indicate that an ACCDS provides delay and communication costs and differentiated services; an ACCDS scheme can reduce the network delay by 11.111%–53.684% for a delay-sensitive event and reduce the communication costs by 5%–22.308% for interesting events, and reduce the network lifetime by about 28.713%.

Anomaly detection is an important security mechanism for the Internet of Things (IoT). Existing works have been focused on developing accurate anomaly detection models. However, due to the resource-constrained nature of IoT networks and the requirement of real-time security operation, cost efficient (regarding computational efficiency and memory-consumption efficiency) approaches for anomaly detection are highly desirable in IoT applications. In this paper, we investigated machine learning (ML) enabled anomaly detection models for the IoT with regard to multi-objective optimization (Pareto optimization) that minimizes the detection error, execution time, and memory consumption simultaneously. Making use of well-known datasets consisting of network traffic traces captured in an IoT environment, we studied a variety of machine learning algorithms through the world-class H2O AI platform. Our experimental results show that the Gradient Boosting Machine, Random Forest, and Deep Learning models are the most accurate and fastest anomaly detection models; the Gradient Boosting Machine and Random Forest are the most accurate and memory-efficient models. These ML models form the Pareto-optimal set of anomaly detection models. Our results can be used by the industry to facilitate their selection of ML models for anomaly detection on various IoT networks based on their security requirements and system constraints.

The rapid spread of Internet of Things (IoT) technologies and the rapidly increasing use of IoT devices offer technological transformation and innovative solutions in many areas from daily life to industrial processes. However, the resource constraints, simple operating systems, non-standard protocols and embedded software of IoT devices make them vulnerable to cyber-attacks. This makes IoT networks risky against malicious attacks and increases the size of security threats. Moreover, the complexity and heterogeneity of IoT networks render traditional security approaches inadequate and increase the need for advanced solutions. In this context, the need for methods for detecting and preventing attacks on IoT networks that are not only reliable and effective, but also understandable by users and security experts has become increasingly critical. This need for network security necessitates the development of strategies that will both secure technical infrastructures and increase the trust of human elements interacting with these infrastructures. In this context, the need for more interpretable, explainable and transparent security approaches is increasing. In particular, machine learning (ML) and deep learning (DL) based intrusion detection systems offer effective solutions to security problems such as anomaly detection and attack classification. The comprehensibility of the decision mechanisms of the models used enables both security experts to manage the systems more effectively and users to have more confidence in the security measures taken. Explainable Artificial Intelligence (XAI) techniques make the decision processes of ML and DL models transparent, allowing to understand how and why attacks are detected. Accordingly, it has become a critical requirement for security systems not only to achieve high accuracy rates, but also to make the decisions taken interpretable. In this study, the effectiveness of artificial intelligence (ML and DL) techniques for the detection and classification of security threats in IoT networks is analysed. In addition, the applications of XAI methods such as SHapley Additive exPlanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME) and Explain Like I'm 5 (ELI5) for IoT security are investigated. It is shown how these methods make the decision processes of ML and DL models used in IoT networks more transparent and provide a better analysis. As a result, this study presents an approach that combines both performance and explainability in IoT security. By demonstrating the effectiveness of XAI-supported ML and DL models, it aims to contribute to future research and innovative security solutions for enhancing security in IoT networks.

Because of increased applications of Internet of Things (IoT) environment in real‐time environment, confidential data gathered by the IoT devices are being communicated to the cloud environment to train the machine learning (ML) models in understanding the patterns that exist in the data. At the same time, the sensitive nature of the IoT data attracts malicious users into hacking efforts. An intrusion detection system (IDS) can be applied to ensure security in the IoT environment. In order to improve security, the ML models can be executed at the data source instead of centralized cloud server. Federated learning (FL) is a recent progression of ML model which enables the ML models to move into the data source rather than moving the data to centralized cloud and thereby resolves cybersecurity problems in the IoT environment. In this view, this study introduces an FL based IDS using bird swarm algorithm based feature selection with classification (FLIDS‐BSAFSC) model in IoT environment. The presented FLIDS‐BSAFSC model undergoes training on multiple aspects of IoT dataset in a decentralized format to classify, detect, and defend against attacks. The proposed FLIDS‐BSAFSC model initially applies min–max normalization technique to pre‐process the IoT data. Besides, BSA based feature selection (BSA‐FS) technique is designed to elect feature subsets. Finally, social group optimization algorithm with kernel extreme learning machine model is employed for identifying various kinds of classes. In the view of FL where the IoT dataset is not distributed to the server carries out profile aggregation competently with the advantage of peer learning. The experimental validation of the FLIDS‐BSAFSC model is tested using benchmark datasets and the results are inspected under several aspects. The experimental values highlighted the better performance of the FLIDS‐BSAFSC model over recent approaches.

The Internet of Things (IoT) has been widely adopted in a range of verticals, e.g., automation, health, energy, and manufacturing. Many of the applications in these sectors, such as self-driving cars and remote surgery, are critical and high stakes applications, calling for advanced machine learning (ML) models for data analytics. Essentially, the training and testing data that are collected by massive IoT devices may contain noise (e.g., abnormal data, incorrect labels, and incomplete information) and adversarial examples. This requires high robustness of ML models to make reliable decisions for IoT applications. The research of robust ML has received tremendous attention from both academia and industry in recent years. This article will investigate the state of the art and representative works of robust ML models that can enable high resilience and reliability of IoT intelligence. Two aspects of robustness will be focused on, i.e., when the training data of ML models contain noises and adversarial examples, which may typically happen in many real-world IoT scenarios. In addition, the reliability of both neural networks and reinforcement learning framework will be investigated. Both of these two ML paradigms have been widely used in handling data in IoT scenarios. The potential research challenges and open issues will be discussed to provide future research directions.

Auction-guided model diffusion for communication-efficient federated learning on non-IID data.

With the increasing popularity of the Internet of Things (IoT) platforms, the cyber security of these platforms is a highly active area of research. One key technology underpinning smart IoT systems is machine learning, which classifies and predicts events from large-scale data in IoT networks. Machine learning is susceptible to cyber attacks, particularly data poisoning attacks that inject false data when training machine learning models. Data poisoning attacks degrade the performances of machine learning models. It is an ongoing research challenge to develop trustworthy machine learning models resilient and sustainable against data poisoning attacks in IoT networks. We studied the effects of data poisoning attacks on machine learning models, including the gradient boosting machine, random forest, naive Bayes, and feed-forward deep learning, to determine the levels to which the models should be trusted and said to be reliable in real-world IoT settings. In the training phase, a label modification function is developed to manipulate legitimate input classes. The function is employed at data poisoning rates of 5%, 10%, 20%, and 30% that allow the comparison of the poisoned models and display their performance degradations. The machine learning models have been evaluated using the ToN_IoT and UNSW NB-15 datasets, as they include a wide variety of recent legitimate and attack vectors. The experimental results revealed that the models’ performances will be degraded, in terms of accuracy and detection rates, if the number of the trained normal observations is not significantly larger than the poisoned data. At the rate of data poisoning of 30% or greater on input data, machine learning performances are significantly degraded.

Massive amounts of data are produced continuously by billions of Internet of Things (IoT) devices and analyzed via Machine Learning (ML) models to serve a wide variety of needs. However, the high communication cost of traditional ML approaches coupled with data privacy issues makes them unpractical for many IoT applications, especially in healthcare where medical records contain sensitive information that can compromise patient privacy. As a result, current research has begun to investigate Federated Learning (FL) as a new paradigm addressing these critical concerns by training ML models without sharing private data. This paper provides a comprehensive study of existing FL algorithms and discusses their applicability in a medical IoT context using publicly available datasets. This is proceeded by integrating such schemes in the FedML framework using real-world medical datasets in both simulated and on-device federated settings to investigate the impact of clients number, communication loss, and data compression on model performance, energy, time, and code footprint. The paper also makes suggestions on open research issues that need to be addressed by the community.

The integration of the Internet of Things (IoT) with Machine Learning (ML) is a transformative advancement that is revolutionizing the way data-driven decision-making occurs across various industries. IoT systems comprise interconnected devices that collect and transmit vast amounts of real-time data from sensors, machines, and appliances. However, merely collecting data is not sufficient; the real value lies in the analysis and interpretation of this data to generate actionable insights. This is where ML comes into play. ML techniques allow systems to learn from the data generated by IoT devices, enabling predictive analysis, automation, and enhanced decision-making processes. This integration of IoT and ML is paving the way for smarter, more efficient systems that can be applied in a wide array of fields such as healthcare, manufacturing, transportation, home automation, and smart cities. For instance, in healthcare, wearable IoT devices track vital health statistics like heart rate and blood pressure, while ML algorithms process these data in real-time to detect anomalies, predict potential health risks, and provide healthcare professionals with alerts for timely interventions. Similarly, in manufacturing, IoT devices collect sensor data from machines, which is analyzed by ML algorithms to predict maintenance needs, preventing costly breakdowns and improving operational efficiency. The sheer scale and complexity of data produced by IoT devices pose significant challenges for traditional data processing methods. ML algorithms are essential for managing and extracting value from this data, as they can handle large datasets, identify patterns, and make predictions in a scalable manner. By utilizing ML models such as deep learning, reinforcement learning, and clustering techniques, IoT systems are capable of adapting to changing environments, learning from their surroundings, and making intelligent decisions without human intervention. This paper will review the various ways ML can be leveraged within IoT systems to provide scalable, intelligent decision-making processes for analyzing the vast amounts of data produced by IoT devices. It will examine key use cases across different sectors where the integration of ML and IoT has shown significant promise. Specific case studies will be highlighted, including healthcare, where ML models enhance the monitoring and prediction of patient health; industrial IoT (IIoT), where predictive maintenance and anomaly detection improve operational efficiency; and smart cities, where ML-optimized IoT systems are used to manage traffic flow, energy consumption, and public services. By exploring these case studies, this paper aims to demonstrate the immense potential of integrating IoT with ML. It will also examine the challenges that arise in implementing such systems, including issues of scalability, data privacy, and security, and discuss potential solutions to these challenges. The paper will conclude with insights into the future of IoT-ML integration and how these technologies can continue to evolve to create even more intelligent, autonomous, and efficient systems across a broad range of industries.

In edge computing (EC), federated learning (FL) has been widely used to cooperatively train machine learning models and protect privacy over the traditional approaches. In order to avoid the possible communication bottleneck and single point failure in parameter server (PS) framework, we mainly focus on decentralized federated learning (DFL) in which each worker disseminates information through peer-to-peer (P2P) communication network. However, the high communication cost and the non-IID local data hinder the efficient training of DFL. In this paper, we propose a mechanism (termed DFL-DF) to achieve efficient communication and improve the performance on non-IID data. Specifically, each worker will exchange data features with neighbors rather than models (or gradients). Different from the traditional fixed topology methods, each worker will communicate with its neighbors selected at each round. Extensive experiment results show the high effectiveness of DFL-DF. Specifically, DFL-DF can reduce communication cost by 29.4%-82.06% and improve test accuracy 2.5%-9.1% under communication cost constraints, compared with the traditional model exchanging methods.