Changing Network Environment Research Articles

Robust and energy-efficient RPL optimization algorithm with scalable deep reinforcement learning for IIoT

The increasing complexity and quantity of the Industrial Internet of Things (IIoT) pose new challenges to the traditional routing protocol for low-power and lossy networks (RPL) in terms of dynamic management, data transmission reliability, and energy efficiency optimization. This paper proposes a scalable deep reinforcement learning (DRL) algorithm with a multi-attention actor double critic model for routing optimization (MADC) to meet the requirements of IIoT for efficient and intelligent routing decisions while improving data transmission reliability and energy efficiency. Specifically, MADC employs the centralized training and decentralized execution (CTDE) learning paradigm to decouple the model’s training and inference tasks, which reduces the difficulty and computational cost of model learning and improves the training efficiency. In addition, a lightweight actor network based on multi-scale convolutional attention mechanism is designed in MADC, which can provide intelligent and real-time decision-making capabilities for resource-constrained nodes with low computational and storage complexities. Moreover, a scalable critic network utilizing multiple attention mechanisms is proposed. It is not only suitable for dynamic and changing network environments but also can more comprehensively and accurately evaluate local observation states, providing more accurate and efficient guidance for model optimization. Furthermore, MADC incorporates a double critic network architecture to mitigate potential overestimation issues during training, thereby ensuring the model’s robustness and reliability. Simulation results demonstrate that MADC outperforms existing RPL optimization algorithms in terms of energy efficiency, data transmission reliability, and adaptability.

A multi-information fusion anomaly detection model based on convolutional neural networks and AutoEncoder

Network traffic anomaly detection, as an effective analysis method for network security, can identify differentiated traffic information and provide secure operation in complex and changing network environments. To avoid information loss caused when handling traffic data while improving the detection performance of traffic feature information, this paper proposes a multi-information fusion model based on a convolutional neural network and AutoEncoder. The model uses a convolutional neural network to extract features directly from the raw traffic data, and a AutoEncoder to encode the statistical features extracted from the raw traffic data, which are used to supplement the information loss due to cropping. These two features are combined to form a new integrated feature for network traffic, which has the load information from the original traffic data and the global information of the original traffic data obtained from the statistical features, thus providing a complete representation of the information contained in the network traffic and improving the detection performance of the model. The experiments show that the classification accuracy of network traffic anomaly detection using this model outperforms that of classical machine learning methods.

An efficient bandwidth-adaptive gradient compression algorithm for distributed training of deep neural networks

In distributed deep learning with data parallelism, communication bottleneck throttles the efficiency of model training. Recent studies adopt versatile gradient compression techniques, with communication sparsification standing out as an effective approach for reducing the number of gradients to be transmitted. However, the deployment of gradient sparsification is adversely influenced by the change of network environment in real systems, and existing methods either neglect bandwidth dynamics during training or experience drastic fluctuation of compression ratios. In this paper, we propose ACE, a novel adaptive gradient compression mechanism with high communication efficiency under bandwidth variation. ACE adapts the sparsification ratio to the average bandwidth in a time window, other than following its dynamics exactly. To accurately compute the compression ratio, we first profile the compression time and model a single iteration time consisting of communication, computation and compression operations. We then present a practical model to fit the needed training rounds till convergence, and formulate an optimization problem to compute the optimal sparsification ratio. We conduct experiments on different DNN models in different network environments and compare various methods in terms of convergence and model quality. The experimental results show that ACE achieves up to 9.39× and 1.28× training speedups over fixed and state-of-the-art adaptive compression methods.

Security Enhancement for Deep Reinforcement Learning-Based Strategy in Energy-Efficient Wireless Sensor Networks.

Energy efficiency and security issues are the main concerns in wireless sensor networks (WSNs) because of limited energy resources and the broadcast nature of wireless communication. Therefore, how to improve the energy efficiency of WSNs while enhancing security performance has attracted widespread attention. In order to solve this problem, this paper proposes a new deep reinforcement learning (DRL)-based strategy, i.e., DeepNR strategy, to enhance the energy efficiency and security performance of WSN. Specifically, the proposed DeepNR strategy approximates the Q-value by designing a deep neural network (DNN) to adaptively learn the state information. It also designs DRL-based multi-level decision-making to learn and optimize the data transmission paths in real time, which eventually achieves accurate prediction and decision-making of the network. To further enhance security performance, the DeepNR strategy includes a defense mechanism that responds to detected attacks in real time to ensure the normal operation of the network. In addition, DeepNR adaptively adjusts its strategy to cope with changing network environments and attack patterns through deep learning models. Experimental results show that the proposed DeepNR outperforms the conventional methods, demonstrating a remarkable 30% improvement in network lifespan, a 25% increase in network data throughput, and a 20% enhancement in security measures.

A Repeated Game-Based Distributed Denial of Service Attacks Mitigation Method for Mining Pools

A Distributed Denial of Service (DDoS) attack is a prevalent issue in the blockchain network layer, causing significant revenue loss for honest mining pools. This paper introduces a novel method, the Repeated Game-based DDoS attack mitigation (RGD), to address this problem. Unlike traditional methods such as game theory and machine learning-based detection, the RGD method can effectively reflect the changes in mining revenue and strategies under different network-strength environments. In particular, we abstract the problem of DDoS mining pool revenue loss into a game revenue model and propose the subgame perfect equilibrium (SPE) approach to solve the optimal payoffs and pool strategies in various network environments. Furthermore, we address the returns of mining pools in an infinitely repeated game environment using the Two-Stage Repeated Game (TSRG) method, where the strategy varies with different network environments. The Matlab experimental simulation results indicate that as the network environment improves, the optimal mining strategies of mining pools are gradually shifting from honest strategies to launching DDoS attacks against each other. The RGD method can effectively represent the impact of changes in the network environment on the mining pool’s strategy selection and optimal revenue. Consequently, with the changing network environment, the optimal revenue of the mining pool only increases by 10% of the revenue loss during a DDoS attack.

An Optimized Hybrid Deep Intrusion Detection Model (HD-IDM) for Enhancing Network Security

Detecting cyber intrusions in network traffic is a tough task for cybersecurity. Current methods struggle with the complexity of understanding patterns in network data. To solve this, we present the Hybrid Deep Learning Intrusion Detection Model (HD-IDM), a new way that combines GRU and LSTM classifiers. GRU is good at catching quick patterns, while LSTM handles long-term ones. HD-IDM blends these models using weighted averaging, boosting accuracy, especially with complex patterns. We tested HD-IDM on four datasets: CSE-CIC-IDS2017, CSE-CIC-IDS2018, NSL KDD, and CIC-DDoS2019. The HD-IDM classifier achieved remarkable performance metrics on all datasets. It attains an outstanding accuracy of 99.91%, showcasing its consistent precision across the dataset. With an impressive precision of 99.62%, it excels in accurately categorizing positive cases, crucial for minimizing false positives. Additionally, maintaining a high recall of 99.43%, it effectively identifies the majority of actual positive cases while minimizing false negatives. The F1-score of 99.52% emphasizes its robustness, making it the top choice for classification tasks requiring precision and reliability. It is particularly good at ROC and precision/recall curves, discriminating normal and harmful network activities. While HD-IDM is promising, it has limits. It needs labeled data and may struggle with new intrusion methods. Future work should find ways to handle unlabeled data and adapt to emerging threats. Also, making HD-IDM work faster for real-time use and dealing with scalability challenges is key for its broader use in changing network environments.

EdAR: An Experience-Driven Multipath Scheduler for Seamless Handoff in Mobile Networks

Multipath TCP (MPTCP) improves the bandwidth utilization in wireless network scenarios, since it can simultaneously utilize multiple interfaces for data transmission. However, with the fast growth of mobile devices and applications, link interruptions caused by handoffs still lead to drastic performance degradation in such scenarios. Typically, a series of packet losses on part of the links will block the transmission of the entire connection when handoff occurs. This paper proposes an Experience-driven Adaptive Redundant packet scheduler (EdAR) for MPTCP, aiming at achieving seamless handoffs in mobile networks. EdAR enables flexibly scheduling redundant packets with an experience-driven learning-based approach in the face of drastic network environment changes for multipath performance enhancement. To enable accurate learning and prediction, both the network environment and the best course of actions are jointly learned via a Deep Reinforcement Learning (DRL) agent, which we design with a hybrid structure to deal with the complexity of system states. Furthermore, both offline and online learning are utilized to allow the agent to adapt to different and changing network environments. Evaluation results show that EdAR outperforms the state-of-the-art MPTCP schedulers in most network scenarios. Specifically in mobile networks with frequent handoffs, EdAR brings 2× improvement in terms of the overall goodput.

Securing a Smart Home with a Transformer-Based IoT Intrusion Detection System

Machine learning (ML)-based Network Intrusion Detection Systems (NIDSs) can classify each network’s flow behavior as benign or malicious by detecting heterogeneous features, including both categorical and numerical features. However, the present ML-based NIDSs are deemed insufficient in terms of their ability to generalize, particularly in changing network environments such as the Internet of Things (IoT)-based smart home. Although IoT devices add so much to home comforts, they also introduce potential risks and vulnerabilities. Recently, many NIDS studies on other IoT scenarios, such as the Internet of Vehicles (IoV) and smart cities, focus on utilizing the telemetry data of IoT devices for IoT intrusion detection. Because when IoT devices are under attack, their abnormal telemetry data values can reflect the anomaly state of those devices. Those telemetry data-based IoT NIDS methods detect intrusion events from a different view, focusing on the attack impact, from the traditional network traffic-based NIDS, which focuses on analyzing attack behavior. The telemetry data-based NIDS is more suitable for IoT devices without built-in security mechanisms. Considering the smart home IoT scenario, which has a smaller scope and a limited number of IoT devices compared to other IoT scenarios, both NIDS views can work independently. This motivated us to propose a novel ML-based NIDS to combine the network traffic-based and telemetry data-based NIDS together. In this paper, we propose a Transformer-based IoT NIDS method to learn the behaviors and effects of attacks from different types of data that are generated in the heterogeneous IoT environment. The proposed method utilizes a self-attention mechanism to learn contextual embeddings for input network features. Based on the contextual embeddings, our method can solve the feature set challenge, including both continuous and categorical features. Our method is the first to utilize both network traffic data and IoT sensors’ telemetry data at the same time for intrusion detection. Experiments reveal the effectiveness of our method on a realistic network traffic intrusion detection dataset named ToN_IoT, with an accuracy of 97.95% for binary classification and 95.78% for multiple classifications on pure network data. With the extra IoT information, the performance of our method has been improved to 98.39% and 97.06%, respectively. A comparative study with existing works shows that our method can achieve state-of-the-art performance on the ToN_IoT dataset.

Artificial intelligence of things at the edge: Scalable and efficient distributed learning for massive scenarios

Federated Learning (FL) is a distributed optimization method in which multiple client nodes collaborate to train a machine learning model without sharing data with a central server. However, communication between numerous clients and the central aggregation server to share model parameters can cause several problems, including latency and network congestion. To address these issues, we propose a scalable communication infrastructure based on Information-Centric Networking built and tested on Apache Kafka®. The proposed architecture consists of a two-tier communication model. In the first layer, client updates are cached at the edge between clients and the server, while in the second layer, the server computes global model updates by aggregating the cached models. The data stored in the intermediate nodes at the edge enables reliable and effective data transmission and solves the problem of intermittent connectivity of mobile nodes. While many local model updates provided by clients can result in a more accurate global model in FL, they can also result in massive data traffic that negatively impacts congestion at the edge. For this reason, we couple a client selection procedure based on a congestion control mechanism at the edge for the given architecture of FL. The proposed algorithm selects a subset of clients based on their resources through a time-based backoff system to account for the time-averaged accuracy of FL while limiting the traffic load. Experiments show that our proposed architecture has an improvement of over 40% over the network-centric based FL architecture, i.e., Flower. The architecture also provides scalability and reliability in the case of mobile nodes. It also improves client resource utilization, avoids overflow, and ensures fairness in client selection. The experiments show that the proposed algorithm leads to the desired client selection patterns and is adaptable to changing network environments.

A Network Intrusion Detection Method Incorporating Bayesian Attack Graph and Incremental Learning Part

For the current stage of complex and changing network environments and correlated and synchronized vulnerability attacks, this study first fuses attack graph technology and Bayesian networks and constructs Bayesian attack graphs toportray the correlation relationships between vulnerabilities and discovering attackers’ intentions. Meanwhile, improving the Bayesian attack graph is difficult because it is difficult to achieve active updates and adapt to the changing network environment and other problems. The study proposed a detection method that integrated the Bayesian attack graph and the XGBoost incremental learning (IL) approach. Experiments showed that the IL model had an accuracy of 0.951, an accuracy of 0.999, a recall of 0.815, an F1 value of 0.898, and an Area Under Curve (AUC) value of 0.907. The prediction ability of this method was better than that of the base model. Bayesian attack graphs fused with IL can detect attacks in the network more efficiently and accurately, so the probability of each node in the network system being attacked can be updated in real time.

Dependent Task-Offloading Strategy Based on Deep Reinforcement Learning in Mobile Edge Computing

In mobile edge computing, there are usually relevant dependencies between different tasks, and traditional algorithms are inefficient in solving dependent task-offloading problems and neglect the impact of the dynamic change of the channel on the offloading strategy. To solve the offloading problem of dependent tasks in a dynamic network environment, this paper establishes the dependent task model as a directed acyclic graph. A Dependent Task-Offloading Strategy (DTOS) based on deep reinforcement learning is proposed with minimizing the weighted sum of delay and energy consumption of network services as the optimization objective. DTOS transforms the dependent task offloading into an optimal policy problem under Markov decision processes. Multiple parallel deep neural networks (DNNs) are used to generate offloading decisions, cache the optimal decisions for each round, and then optimize the DNN parameters using priority experience replay mechanism to extract valuable experiences. DTOS introduces a penalty mechanism to obtain the optimal task-offloading decisions, which is triggered if the service energy consumption or service delay exceeds the threshold. The experimental results show that the algorithm produces better offloading decisions than existing algorithms, can effectively reduce the delay and energy consumption of network services, and can self-adapt to the changing network environment.

ECAH: A New Energy-Aware Coverage Method for Wireless Sensor Networks using Artificial Bee Colony and Harmony Search

Wireless Sensor Networks (WSNs) offer diverse applications in the research and commercial fields, such as military applications, medical science, waste management, home automation, habitat monitoring, and environmental observation. WSNs are generally composed of a large number of low-cost, low-power, and multifunctional sensor nodes that sense, process, and communicate data. These nodes are connected by a wireless medium, allowing them to collect and share data with each other. To achieve network coverage in a WSN, a few to thousands of tiny and low-power sensor nodes should be placed in an interconnected manner. Over the last decade, deploying sensor nodes in a WSN to cover a large area has received much attention. Coverage, regarded as an NP-hard problem, is an essential parameter for WSNs that determines how the deployed sensor nodes handle each point of interest. Various algorithms have been proposed to tackle this problem. However, they often come with a trade-off between energy efficiency and coverage rate. Moreover, the scalability of the algorithms needs to be considered for large-scale networks. This paper proposes a novel energy-aware method combining Artificial Bee Colony (ABC) and Harmony Search (HS) algorithms to address the coverage problem in WSN, called ECAH. The proposed ECAH algorithm has been tested with various network scenarios and compared with other existing algorithms. The results show that ECAH outperforms the existing methods in terms of network lifetime, coverage rate, and energy consumption. Additionally, the proposed algorithm is also more robust and efficient as it can adjust to dynamic network environment changes, making it suitable for various network scenarios.

АЛГОРИТМ РАНЖИРОВАНИЯ УЗЛОВ БЕСПРОВОДНОЙ СЕНСОРНОЙ СЕТИ ПРИ ИХ СЛУЧАЙНОМ ПРОСТРАНСТВЕННОМ РАСПРЕДЕЛЕНИИ

The method of making a decision on the ranking of sensor nodes in a wireless sensor network randomly distributed on a certain plane, which operates in a constantly changing network environment, is considered. As a priori information about the network environment, expertise is used, framed in the form of a matrix of utility relations. The given matrix is filled with integer point estimates. The solution to the ranking of base stations is a calculated a posteriori fuzzy set. A numerical example of solving the problem is considered.

An Adaptive MAC Protocol Based on Time-Domain Interference Alignment for UWANs

Abstract The spatial and temporal uncertainty caused by large propagation delays is a fundamental feature of Underwater Acoustic Networks (UWANs), which seriously affects the performance of the UWANs and also brings challenges to the design of MAC protocols. In this paper, we develop an adaptive MAC protocol based on deep reinforcement learning for UWANs, called ARL-MAC protocol, to intelligently allocate time slots for nodes. Firstly, we design a reward mechanism based on the idea of Time-Domain Interference Alignment (TDIA). We determine the reward according to the combination of the node action and the feedback corresponding to the action. Then, we propose a flexible training mechanism to deal with the ever-changing underwater environment, which improves the fairness of time slot allocation. In addition, we introduce the Deep Recurrent Q-Network (DRQN) algorithm to solve the partially observable information issue. Finally, we evaluate the ARL-MAC protocol with the different number of nodes and changing network environment. Simulation results reveal that the ARL-MAC protocol outperforms other MAC protocols for UWANs in terms of throughput, collision rate and service fairness.

Correlation between Deep Neural Network Hidden Layer and Intrusion Detection Performance in IoT Intrusion Detection System

As the Internet of Things (IoT) continues to grow, a vast amount of data is generated. The IoT environment is quite sensitive to security challenges because personal information may be leaked or sensor data may be manipulated, which could cause accidents. Because traditional intrusion detection system (IDS) studies are often designed to work well on datasets, it is unknown whether they would work well in a changing network environment. In addition, IDSs for protecting IoT environments have been studied, but their performance was verified using datasets unrelated to the IoT, so it is not known whether the performance would be effective in an IoT environment. In this study, we propose an intrusion detection hyperparameter control system (ID-HyConSys) that automates the IDS using proximal policy optimization (PPO) to solve these problems and reliably protect the IoT environment. ID-HyConSys consists of an intrusion detection module consisting of a deep neural network (DNN) feature extractor that extracts efficient features from a changing network environment, a k-means cluster that clusters the extracted data, and a PPO agent that automates the IDS through learning and control. Through experimentation, it was confirmed that the hidden layer configuration, the number of feature extractions by the DNN feature extractor, and the number of clusters in the k-means cluster significantly affected the intrusion detection performance. The PPO directly controls these hyperparameters and determines the optimized value itself. The performance of ID-HyConSys was evaluated using the CICIDS2017 and MQTTset datasets. An F1-score of 0.9707 on CICIDS2017 and an F1-score of 0.9973 on the MQTTset were obtained. Finally, we merged the two datasets and obtained an F1-score of 0.9901. The superiority of the ID-HyConSys proposed in this study was confirmed because ID-HyConSys showed high performance on each dataset and, at the same time, very high performance on complex merged datasets. ID-HyConSys is expected to protect the IoT environment more quickly and safely by automatically learning network changes and adjusting the intrusion detection module accordingly.

A new fair-efficient spectrum allocation scheme for the LTE and WiFi coexistence platform

Long-term evolution (LTE) carrier aggregation with the unlicensed WiFi spectrum band has been pointed out as a good solution to handle the rapidly increasing wireless data traffic amounts. With both LTE and WiFi networks, the investigation of their coexistence is a hot research topic of active interest, primarily driven by industry and academia. In this study, we investigate a new spectrum allocation scheme for the LTE/WiFi coexistence platform. To maximize the system throughput while guaranteeing a relative fairness between heterogeneous networks, we adopt the cooperative game paradigm to design our novel scheme. According to the basic ideas of Proportional Nash bargaining solution (PNBS) and Zhang’s group bargaining solution (ZGBS), we can leverage a mutual consensus to provide a fair-efficient solution for the LTE/WiFi spectrum sharing problem. Based on the online interactive bargaining approach, network agents work together to effectively share the LTE and WiFi spectrum resources. Under dynamically changing network environments, it is an essential approach to find the relevant trade-off between conflicting requirements, i.e., fairness and efficiency. Validated by simulation results, our proposed scheme can provide important insights into the LTE/WiFi system operations. Finally, we forecast further research trends on the basis of our conclusion.

HTTP Adaptive Streaming Framework with Online Reinforcement Learning

Dynamic adaptive streaming over HTTP (DASH) is an effective method for improving video streaming’s quality of experience (QoE). However, the majority of existing schemes rely on heuristic algorithms, and the learning-based schemes that have recently emerged also have a problem in that their performance deteriorates in a specific environment. In this study, we propose an adaptive streaming scheme that applies online reinforcement learning. When QoE degradation is confirmed, the proposed scheme adapts to changes in the client’s environment by upgrading the ABR model while performing video streaming. In order to adapt the adaptive bitrate (ABR) model to a changing network environment while performing video streaming, the neural network model is trained with a state-of-the-art reinforcement learning algorithm. The proposed scheme’s performance was evaluated using simulation-based experiments under various network conditions. The experimental results confirmed that the proposed scheme performed better than the existing schemes.

A public blockchain consensus mechanism for fault-tolerant distributed computing in LEO satellite communications

In LEO (Low Earth Orbit) satellite communication systems, the satellite network is made up of a large number of satellites, the dynamically changing network environment affects the results of distributed computing. In order to improve the fault tolerance rate, a novel public blockchain consensus mechanism that applies a distributed computing architecture in a public network is proposed. Redundant calculation of blockchain ensures the credibility of the results; and the transactions with calculation results of a task are stored distributed in sequence in Directed Acyclic Graphs (DAG). The transactions issued by nodes are connected to form a net. The net can quickly provide node reputation evaluation that does not rely on third parties. Simulations show that our proposed blockchain has the following advantages: 1. The task processing speed of the blockchain can be close to that of the fastest node in the entire blockchain; 2. When the tasks' arrival time intervals and demanded working nodes(WNs) meet certain conditions, the network can tolerate more than 50% of malicious devices; 3. No matter the number of nodes in the blockchain is increased or reduced, the network can keep robustness by adjusting the task's arrival time interval and demanded WNs.

Comparative Analysis of Routing Schemes Based on Machine Learning

Machine learning-based distributed routing algorithms, in contrast to traditional mathematical model-driven distributed routing algorithms, are typically data-driven, allowing them to adapt to dynamically changing network environments and various performance evaluation index optimization requirements. It is quite likely that it will become a key part of the next-generation Internet in the future. However, current intelligent routing research is still in its early stages. This article provides a comprehensive review of the state-of-the-art routing algorithms based on machine learning. First, important research on existing data-driven intelligent routing algorithms is presented with the key concepts and applications of these systems demonstrated. To enable intelligent routing algorithms to be deployed in real scenarios with cheap cost and high reliability, two appropriate training deployment frameworks and intelligent routing algorithm training and deployment strategies are given. Finally, the future development of machine learning-based intelligent routing systems is examined. The opportunities and problems that have been encountered, as well as prospective research directions, are discussed.

Analysis of Network Slicing for Management of 5G Networks Using Machine Learning Techniques

Consumer expectations and demands for quality of service (QoS) from network service providers have risen as a result of the proliferation of devices, applications, and services. An exceptional study is being conducted by network design and optimization experts. But despite this, the constantly changing network environment continues to provide new issues that today’s networks must be dealt with effectively. Increased capacity and coverage are achieved by joining existing networks. Mobility management, according to the researchers, is now being investigated in order to make the previous paradigm more flexible, user-centered, and service-centric. Additionally, 5G networks provide higher availability, extremely high capacity, increased stability, and improved connection, in addition to quicker speeds and less latency. In addition to being able to fulfil stringent application requirements, the network infrastructure must be more dynamic and adaptive than ever before. Network slicing may be able to meet the present stringent application requirements for network design, if done correctly. The current study makes use of sophisticated fuzzy logic to create algorithms for mobility and traffic management that are as flexible as possible while yet maintaining high performance. Ultimately, the purpose of this research is to improve the quality of service provided by current mobility management systems while also optimizing the use of available network resources. Building SDN (Software-Defined Networking) and NFV (Network Function Virtualization) technologies is essential. Network slicing is an architectural framework for 5G networks that is intended to accommodate a variety of different networks. In order to fully meet the needs of various use cases on the network, network slicing is becoming more important due to the increasing demand for data rates, bandwidth capacity, and low latency.