Learning Based Routing Protocol Research Articles

Q Learning-Based Routing Protocol With Accelerating Convergence for Underwater Wireless Sensor Networks

<i>Q</i> Learning-Based Routing Protocol With Accelerating Convergence for Underwater Wireless Sensor Networks

Multi-Agent Reinforcement Learning-Based Routing Protocol for Underwater Wireless Sensor Networks With Value of Information

Multi-Agent Reinforcement Learning-Based Routing Protocol for Underwater Wireless Sensor Networks With Value of Information

3R: A reliable multi agent reinforcement learning based routing protocol for wireless medical sensor networks

Interest in the Wireless Medical Sensor Network (WMSN) is rapidly gaining attention thanks to recent advances in semiconductors and wireless communication. However, by virtue of the sensitive medical applications and the stringent resource constraints, there is a need to develop a routing protocol to fulfill WMSN requirements in terms of delivery reliability, attack resiliency, computational overhead, and energy efficiency. This paper proposes 3R, a reliable multi agent reinforcement learning routing protocol for WMSN. 3R uses a novel resource-conservative Reinforcement Learning (RL) model to reduce the computational overhead, along with two updating methods to speed up the algorithm convergence. The reward function is re-defined as a punishment, combining the proposed trust management system to defend against well-known dropping attacks. Furthermore, an energy model is integrated with the reward function to enhance the network lifetime and balance energy consumption across the network. The proposed energy model only uses local information to avoid the resource burdens and the security concerns of exchanging energy information. Experimental results prove the lightweightness, attacks resiliency and energy efficiency of 3R, making it a potential routing candidate for WMSN.

An Optimized Reinforcement Learning-Based Routing Protocol Using Spider Monkey Optimization Algorithm

In Wireless Sensor Networks, numerous machine-learning algorithms are used in the packet routing strategy. The recommended routing approach is implemented using adaptive reinforcement learning and a spider monkey optimization algorithm. Data packets are sent over the network via optimum route establishment—the above attempts to save energy by changing the Q-value at each instance. Technology enables adaptive mechanisms to demonstrate intelligent behavior in complex and dynamic situations such as Wireless Sensor Networks (WSN). Recently, new routing methods for WSNs have been devised in response to such a notion. Due to user interference and overall network performance, the numerous routing pathways cause frequent channel-switching occurrences. The spider monkey optimization method obtains routing metrics such as bandwidth, end-to-end latency, throughput, overhead, and packet delivery ratio. Compared to other machine learning algorithms, such as RLBR, the performance of WSN is improved and obtained better performance results.

A reinforcement learning based routing protocol for software-defined networking enabled wireless sensor network forest fire detection

Critical event reporting Wireless Sensor Networks (WSNs) applications need vital requirements (extended network lifetime, reliability, real time responsiveness, and scalability) to be met to ensure outstanding efficiency. Previous frameworks only consider few individual requirements, thus ignoring the other equally important ones. Ensuring that an active path is available at all times is crucial for enabling the timely transmission of critical data and maintaining the quality of service required to efficiently support delay-sensitive applications. This paper proposes an application-specific Routing Protocol based on Reinforcement Learning (RL) for Software Defined Network (SDN)-enabled WSN forest fire detection (RPLS). First, we designed a clustering algorithm that delays re-clustering to save energy by keeping the same topology for several rounds. Unlike existing works, this algorithm decreases the cluster radius based not only on the energy parameters but also on the quality of the links. After the network clustering, the power of the SDN controller is used to intelligently define using RL the optimal paths for the sensor nodes and accordingly reduce the load on these constrained nodes. For routing strategy, we formulate an RL-based reward function considering not only the energy efficiency parameters but also the anticipated and post-failure reliability parameters to ensure real time responsiveness and optimize energy consumption. Finally, we conducted comparisons by means of simulations in forest fires detection scenario. Compared to RL-SDWSN, the results show an improvement of 14.064 % in network operational lifetime and 16.41 % in response time.

A review on machine learning based routing protocols for delay tolerant networks

Nowadays, Delay Tolerant Network plays an important role in improving the communication between the network nodes. Applications of Delay Tolerant Network are disaster recovery, vehicular communication, sensor networks, interplanetary networks, and communication in remote and rural areas. Routing is one of the important tasks for enhancing the energy effectiveness of data transmission among the mobile nodes under network congestion and dynamic topology. Machine Learning-based routing algorithms are used for improving network communication in Delay Tolerant Networks. Its objective is to reduce the delay, minimize the overhead, reduce energy consumption, improve throughput, minimize packet loss, and efficient data transmission. This paper presents a comprehensive review of routing algorithms using machine learning for Delay Tolerant Networks.

ML-RPL: Machine Learning-Based Routing Protocol for Wireless Smart Grid Networks

This research explores the potential of Machine Learning (ML) to enhance wireless communication networks, specifically in the context of Wireless Smart Grid Networks (WSGNs). We integrated ML into the well-established Routing Protocol for Low-Power and Lossy Networks (RPL), resulting in an advanced version called ML-RPL. This novel protocol utilizes CatBoost, a Gradient Boosted Decision Trees (GBDT) algorithm, to optimize routing decisions. The ML model, trained on a dataset of routing metrics, predicts the probability of successfully reaching a destination node. Each node in the network uses the model to choose the route with the highest probability of effectively delivering packets. Our performance evaluation, carried out in a realistic scenario and under various traffic loads, reveals that ML-RPL significantly improves the packet delivery ratio and minimizes end-to-end delay, making it a promising solution for more efficient and responsive WSGNs.

Reinforcement Learning-Based Routing Protocols in Vehicular Ad Hoc Networks for Intelligent Transport System (ITS): A Survey

Today, the use of safety solutions in Intelligent Transportation Systems (ITS) is a serious challenge because of novel progress in wireless technologies and the high number of road accidents. Vehicular ad hoc network (VANET) is a momentous element in this system because they can improve safety and efficiency in ITS. In this network, vehicles act as moving nodes and work with other nodes within their communication range. Due to high-dynamic vehicles and their different speeds in this network, links between vehicles are valid for a short time interval. Therefore, routing is a challenging work in these networks. Recently, reinforcement learning (RL) plays a significant role in developing routing algorithms for VANET. In this paper, we review reinforcement learning and its characteristics and study how to use this technique for creating routing protocols in VANETs. We propose a categorization of RL-based routing schemes in these networks. This paper helps researchers to understand how to design RL-based routing algorithms in VANET and improve the existing methods by understanding the challenges and opportunities in this area.

Reinforcement Learning-Based Routing Protocols in Vehicular and Flying Ad Hoc Networks – A Literature Survey

Vehicular and flying ad hoc networks (VANETs and FANETs) are becoming increasingly important with the development of smart cities and intelligent transportation systems (ITSs). The high mobility of nodes in these networks leads to frequent link breaks, which complicates the discovery of optimal route from source to destination and degrades network performance. One way to overcome this problem is to use machine learning (ML) in the routing process, and the most promising among different ML types is reinforcement learning (RL). Although there are several surveys on RL-based routing protocols for VANETs and FANETs, an important issue of integrating RL with well-established modern technologies, such as software-defined networking (SDN) or blockchain, has not been adequately addressed, especially when used in complex ITSs. In this paper, we focus on performing a comprehensive categorisation of RL-based routing protocols for both network types, having in mind their simultaneous use and the inclusion with other technologies. A detailed comparative analysis of protocols is carried out based on different factors that influence the reward function in RL and the consequences they have on network performance. Also, the key advantages and limitations of RL-based routing are discussed in detail.

Reinforcement Learning-Based Routing Protocols in Flying Ad Hoc Networks (FANET): A Review

In recent years, flying ad hoc networks have attracted the attention of many researchers in industry and universities due to easy deployment, proper operational costs, and diverse applications. Designing an efficient routing protocol is challenging due to unique characteristics of these networks such as very fast motion of nodes, frequent changes of topology, and low density. Routing protocols determine how to provide communications between drones in a wireless ad hoc network. Today, reinforcement learning (RL) provides powerful solutions to solve the existing problems in the routing protocols, and designs autonomous, adaptive, and self-learning routing protocols. The main purpose of these routing protocols is to ensure a stable routing solution with low delay and minimum energy consumption. In this paper, the reinforcement learning-based routing methods in FANET are surveyed and studied. Initially, reinforcement learning, the Markov decision process (MDP), and reinforcement learning algorithms are briefly described. Then, flying ad hoc networks, various types of drones, and their applications, are introduced. Furthermore, the routing process and its challenges are briefly explained in FANET. Then, a classification of reinforcement learning-based routing protocols is suggested for the flying ad hoc networks. This classification categorizes routing protocols based on the learning algorithm, the routing algorithm, and the data dissemination process. Finally, we present the existing opportunities and challenges in this field to provide a detailed and accurate view for researchers to be aware of the future research directions in order to improve the existing reinforcement learning-based routing algorithms.

Performance Analysis of Deep Learning-Based Routing Protocol for an Efficient Data Transmission in 5G WSN Communication

For the past few years, huge interest and dramatic development have been shown for the Internet of Things (IoT) based constrained Wireless sensor network (WSN) to achieve efficient resource utilization and better service delivery. IoT requires a better communication network for data transmission between heterogeneous devices and an optimally deployed energy-efficient WSN. The clustering technique applied for WSN node deployment needs to be efficient; therefore, the entire architecture can obtain a better network lifetime. While clustering, the entire network is partitioned into various clusters. Moreover, the cluster head (CH) selection process also needs proper attention for achieving efficient data communication towards the sink node via selected CH and also for increasing the node reachability within the cluster. In this proposed framework, an energy efficient deep belief network (DBN) based routing protocol is developed, which achieves better data transmission through the selected path. Due to this the packet delivery ratio (PDR) gets improved. In this framework, initially, the nodes in the whole network is grouped as clusters using a reinforcement learning (RL) algorithm, which assigns a reward for the nodes that belong to the particular cluster. Then, the CH required for efficient data communication is selected using a Mantaray Foraging Optimization (MRFO) algorithm. The data is transmitted to the sink node via the selected CH using an efficient deep learning approach. At last, the performance of proposed deep network based routing protocol is evaluated using different evaluation metrics they are network lifetime, energy consumption, number of alive nodes, and packet delivery rate. Finally, the evaluated results are compared with few existing algorithms. Among all these algorithms, the proposed DBN routing protocol has achieved better network lifetime.

Q-FANET: Improved Q-learning based routing protocol for FANETs

Flying Ad-Hoc Networks (FANETs) introduce ad-hoc networking into the context of flying nodes, allowing real-time communication between these nodes and ground control stations. Due to the nature of this kind of node, the structure of a FANET is dynamic, changing very often. Since it has applications in military scenarios and other mission-critical systems, an agile and reliable network is essential with robust and efficient routing protocols. Nonetheless, maintaining an acceptable network delay generated by the selection of routes remains a considerable challenge, owing to the nodes’ high mobility. This article addresses this problem by proposing a routing scheme based on an improved Q-Learning algorithm to reduce network delay in scenarios with high-mobility, called Q-FANET. This proposal has its performance evaluated and compared with other state-of-the-art methods using the WSNET simulator. The experiments provide evidence that the Q-FANET presents lower delay, a minor increase in packet delivery ratio, and significant lower jitter compared with other reinforcement learning-based routing protocols.

Fully-Echoed Q-Routing With Simulated Annealing Inference for Flying Adhoc Networks

Current networking protocols deem inefficient in accommodating the two key challenges of Unmanned Aerial Vehicle (UAV) networks, namely the network connectivity loss and energy limitations. One approach to solve these issues is using learning-based routing protocols to make close-to-optimal local decisions by the network nodes, and Q-routing is a bold example of such protocols. However, the performance of the current implementations of Q-routing algorithms is not yet satisfactory, mainly due to the lack of adaptability to continued topology changes. In this paper, we propose a full-echo Q-routing algorithm with a self-adaptive learning rate that utilizes Simulated Annealing (SA) optimization to control the exploration rate of the algorithm through the temperature decline rate, which in turn is regulated by the experienced variation rate of the Q-values. Our results show that our method adapts to the network dynamicity without the need for manual re-initialization at transition points (abrupt network topology changes). Our method exhibits a reduction in the energy consumption ranging from 7% up to 82%, as well as a 2.6 fold gain in successful packet delivery rate, compared to the state of the art Q-routing protocols.

Reinforcement Learning-Based Routing Protocols for Vehicular Ad Hoc Networks: A Comparative Survey

Vehicular-ad hoc networks (VANETs) hold great importance because of their potentials in road safety improvement, traffic monitoring, and in-vehicle infotainment services. Due to high mobility, sparse connectivity, road-side obstacles, and shortage of roadside units, the links between the vehicles are subject to frequent disconnections; consequently, routing is crucial. Recently, to achieve more efficient routing, reinforcement learning (RL)-based routing algorithms have been investigated. RL represents a class of artificial intelligence that implements a learning procedure based on previous experiences and provides a better solution for future operations. RL algorithms are more favorable than other optimization techniques owing to their modest usage of memory and computational resources. Because a VANET deals with passenger safety, any kind of flaw is intolerable in VANET routing. Fortunately, RL-based algorithms have the potentials to optimize the different quality-of-service parameters of VANET routing such as bandwidth, end-to-end delay, throughput, control overhead, and packet delivery ratio. However, to the best of the authors' knowledge, surveys on RL-based routing protocols for VANETs have not been conducted. To fulfill this gap in the literature and to provide future research directions, it is necessary to aggregate the scattered works on this topic. This study presents a comparative investigation of RL-based routing protocols, by considering their working procedure, advantages, disadvantages, and applications. They are qualitatively compared in terms of key features, characteristics, optimization criteria, performance evaluation techniques, and implemented RL techniques. Lastly, open issues and research challenges are discussed to make RL-based VANET routing protocols more efficient in the future.

Reinforcement Learning-Based Routing Protocol for Underwater Wireless Sensor Networks: A Comparative Survey

Underwater wireless sensor networks (UWSNs) have emerged as a promising networking technology owing to their various underwater applications. Many applications require sensed data to be routed to a centralized location. However, the routing of sensor networks in underwater environments presents several challenges in terms of underwater infrastructure, including high energy consumption, narrow bandwidths, and longer propagation delays than other sensor networks. Efficient routing protocols play a vital role in this regard. Recently, reinforcement learning (RL)-based routing algorithms have been investigated by different researchers seeking to exploit the learning procedure via trial-and-error methods of RL. RL algorithms are capable of operating in underwater environments without prior knowledge of the infrastructure. This paper discusses all routing protocols proposed for RL-based UWSNs. The advantages, disadvantages, and suitable application areas are also mentioned. The protocols are compared in terms of the key ideas, RL designs, optimization criteria, and performance-evaluation techniques. Moreover, research challenges and outstanding research issues are also highlighted, to indicate future research directions.

Reinforcement Learning-Based Routing Protocol to Minimize Channel Switching and Interference for Cognitive Radio Networks

In the existing network-layered architectural stack of Cognitive Radio Ad Hoc Network (CRAHN), channel selection is performed at the Medium Access Control (MAC) layer. However, routing is done on the network layer. Due to this limitation, the Secondary/Unlicensed Users (SUs) need to access the channel information from the MAC layer whenever the channel switching event occurred during the data transmission. This issue delayed the channel selection process during the immediate routing decision for the channel switching event to continue the transmission. In this paper, a protocol is proposed to implement the channel selection decisions at the network layer during the routing process. The decision is based on past and expected future routing decisions of Primary Users (PUs). A learning agent operating in the cross-layer mode of the network-layered architectural stack is implemented in the spectrum mobility manager to pass the channel information to the network layer. This information is originated at the MAC layer. The channel selection is performed on the basis of reinforcement learning algorithms such as No-External Regret Learning, Q-Learning, and Learning Automata. This leads to minimizing the channel switching events and user interferences in the Reinforcement Learning- (RL-) based routing protocol. Simulations are conducted using Cognitive Radio Cognitive Network simulator based on Network Simulator (NS-2). The simulation results showed that the proposed routing protocol performed better than all the other comparative routing protocols in terms of number of channel switching events, average data rate, packet collision, packet loss, and end-to-end delay. The proposed routing protocol implies the improved Quality of Service (QoS) of the delay sensitive and real-time networks such as Cellular and Tele Vision (TV) networks.

Power efficient Memetic Optimized and Adjacent Exponentially Distributed Routing in Mobile Ad Hoc Networks

Mobile Ad Hoc Networks (MANETs) are self-organizing, infrastructure less, possessing constrained power, linked through wireless links without any centralized controller. Optimal selection of routes with minimum end-to-end delay and maximum packet delivery ratio was ensured using learning-based routing protocol, however power factor was not analyzed and therefore increasing the routing overhead. This paper employs the Adjacent Exponentially Distributed Route Maintenance mechanism to include an energy awareness feature with mean data packet arrival rate and link breakage rate to the identified route discovery mechanism. The proposed technique considers routes distance and power during route selection and includes routes energy consumption in its calculations. Besides, route discovery as an optimization is formulated employing Adjacent Exponential Distribution to choose a route that optimizes weighted function of route distance and energy. The simulation results show that the Memetic Optimized Adjacent Exponentially Distributed Routing (MO-AEDR) improves the packet delivery ratio with minimum end-to-end delay and routing overhead.

Opportunistic Multipath Routing in Long-Hop Wireless Sensor Networks.

To improve the packet delivery ratio in wireless sensor networks, many approaches such as multipath, opportunistic, and learning-based routing protocols have been proposed. However, the performance of the existing protocols are degraded under long-hop wireless sensor networks because the additional overhead is proportional to the number of hops. To deal with the overhead, we propose an opportunistic multipath routing that forecasts the required number of paths, as well as bifurcation based on opportunistic routing according to the reliability requirement. In the proposed scheme, an intermediate node is able to select a different node for each transmission and to handle path failure adaptively. Through a performance evaluation, we demonstrate that the proposed scheme achieves a higher packet delivery ratio and reduces the energy consumption by at least approximately 33% and up to approximately 65% compared with existing routing protocols, under the condition of an 80% link success ratio in the long-hop sensor network.

Intelligence in wireless network routing through reinforcement learning

One of the critical challenges in mobile wireless network is resource optimised routing of messages without compromising the performance criteria of the network. Routing in wireless networks has been extensively studied and a variety of routing protocols have been proposed. But these protocols experiences problems due to the dynamism in network topology. To address these problems, reinforcement learning approaches are integrated in routing solutions. This review focuses on the impact of reinforcement learning algorithms to achieve intelligence in wireless network routing. We provide contexts and benefits of applying reinforcement learning paradigm and discuss the major techniques applied to optimise routing solutions. A survey of state of the art reinforcement learning-based routing protocols is presented and categorised these protocols according to the learning strategies. We also provide open issues and suggestions for future research in improving routing solutions.

Reinforcement Learning based Routing Protocols in WSNs: A Survey

Advances in the technology along with reduction in processor size, its memory, and wireless antenna size has facilitated the construction of low cost, low powered and multifunctional Sensor nodes which in turn led to high demand for development of Wireless Sensor Networks. A lot of research work has been done regarding the development of routing protocols for WSNs. This paper provides a brief overview of the routing protocols using Reinforcement learning approach for WSNs.