Optimal Transmission Policy Research Articles

Enabling large-scale low-power LoRa data transmission via multiple mobile LoRa gateways

Low-Power Wide Area Networks (LPWANs) enable large-scale Internet of Things (IoT) applications. LoRa is a promising LPWAN technology, but LoRa nodes are battery-powered, so their lifetime depends on energy efficiency. In practice, LoRa nodes have short battery life because long transmission distances to gateways incur high energy costs. With expensive battery replacement, particularly in large networks, it is crucial to reduce LoRa nodes’ energy consumption. Existing low-power transmission techniques focus on static or single mobile gateways, consuming substantial remote node energy unfit for large-scale networks. This paper proposes integrating LoRa with mobility to minimize node energy use by shortening transmission distances. We design the first large-scale mobile LoRa system, MLoRaDrone, using multiple unmanned aerial vehicles (UAVs) equipped with LoRa gateways flying near nodes. First, we present a low-power mechanism for reliable node sensing and UAV communication. Next, we develop a LoRa transmission parameter assignment strategy encompassing a distributed ADMM-based optimal transmission policy and a 32-approximation channel allocation. Moreover, we propose a UAV trajectory scheduling scheme with a 4-approximation minimizing node energy and UAV flight paths. Evaluations on varied scales verify MLoRaDrone effectiveness for different node counts and UAV paths. Compared to baselines, MLoRaDrone reduces 3000-node energy consumption by up to 35.56× within 25 km2.

Analysis of Coded Slotted ALOHA With Energy Harvesting Nodes for Perfect and Imperfect Packet Recovery Scenarios

We analyze the performance of Coded Slotted ALOHA (CSA) protocols in scenarios where users are equipped with limited batteries that are recharged through Energy Harvesting (EH). First, we assume a Perfect Packet Recovery Scenario (PPRS) for which the received packets are decoded with no errors when there is no interference. We introduce Battery Outage Probability (BOP) as an extra performance metric; and, we derive the optimal EH-CSA transmission policies, which offer the maximum attainable traffic load while maintaining an asymptotically negligible Packet Loss Ratio (PLR), under specific rate and BOP constraints. We extend our study to Imperfect Packet Recovery Scenario (IPRS) where impairments at the physical layer, including channel estimation and channel decoding errors, will distort messages being passed through the iterative Successive Interference Cancellation (SIC) process. The distorted messages being passed through the SIC process potentially lead to error propagation. In order to track the error propagation process, we define the concept of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Accumulated Noise plus Interference Power</i> (ANIP), and analytically track the evolution of its probability distribution. We employ our results to evaluate the bit error rates for different transmission policies for the case of IPRS. We also demonstrate the advantages of the optimal transmission policies through numerical examples for both PPRS and IPRS. Our results show that the optimal EH-CSA policies outperform the policies optimized for standard CSA without EH considerations, and the schemes that are optimal for PPRS are not necessarily optimal for the IPRS case. Furthermore, the EH-CSA optimal policies strictly outperform standard CRDSA when the system is required to support higher traffic loads.

Learning-Based IRS-Assisted Secure Transmission for Mine IoTs.

Mine Internet of Things (MIoT) devices in intelligent mines often face substantial signal attenuation due to challenging operating conditions. The openness of wireless communication also makes it susceptible to smart attackers, such as active eavesdroppers. The attackers can disrupt equipment operations, compromise production safety, and exfiltrate sensitive environmental data. To address these challenges, we propose an intelligent reflecting surface (IRS)-assisted secure transmission system for an MIoT device which enhances the security and reliability of wireless communication in challenging mining environments. We develop a joint optimization problem for the IRS phase shifts and transmit power, with the goal of enhancing legitimate transmission while suppressing eavesdropping. To accommodate time-varying channel conditions, we propose a reinforcement learning (RL)-based IRS-assisted secure transmission scheme that enables MIoT device to optimize both the IRS reflecting coefficients and transmit power for optimal transmission policy in dynamic environments. We adopt the deep deterministic policy gradient (DDPG) algorithm to explore the optimal transmission policy in continuous space. This can reduce the discretization error caused by traditional RL methods. The simulation results indicate that our proposed scheme achieves superior system utility compared with both the IRS-free (IF) scheme and the IRS randomly configured (IRC) scheme. These results demonstrate the effectiveness and practical relevance of our contributions, proving that implementing IRS in MIoT wireless communication can enhance safety, security, and efficiency in the mining industry.

Throughput Maximization of Wireless-Powered Communication Network With Mobile Access Points

In order to mitigate the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">double near-far effect</i> , we focus on a mobile wireless-powered communication network (WPCN), where sensor nodes harvest energy from the radio frequency (RF) signal of the mobile energy access point (EAP), and transmit data to the mobile data access point (DAP) by using the harvested energy. Only the sensor nodes with energy larger than a threshold, which is mainly determined by the energy consumption of one transmission, have opportunities to transmit data. Due to the mobility of the EAP and DAP, the distance between the EAP and DAP changes over time. When the DAP moves into the operation region of the EAP, the EAP and DAP could not work simultaneously due to the severe interference, and an energy harvesting probability is employed to denote the probability that the EAP works in this scenario. The purpose of this paper is to identify the optimal transmission policy, i.e., the optimal pairing of the energy consumption of one transmission and the energy harvesting probability, that maximizes the throughput of the WPCN under an energy causality constraint. By analyzing the energy causality constraint, we show that the WPCN could be divided into an energy-sufficient state and an energy-limited state by the pairing of the energy consumption of one transmission and the energy harvesting probability. Since the energy consumption of one transmission and the energy harvesting probability are jointly intertwined with the energy causality constraint, making the joint optimization problem intractable, we divide the throughput maximization problem into two layers. In the inner problem, we investigate the optimal energy consumption of one transmission with a given energy harvesting probability. In the outer problem, we derive the optimal energy harvesting probability based on the obtained optimal energy consumption of one transmission. According to the aforementioned investigations, we propose a two-layer algorithm to obtain the specific solution. Numerical results are conducted to validate the theoretical results and the efficiency of the proposed two-layer algorithm.

Deep Reinforcement Learning for Latency-Sensitive Communication With Adaptive Redundant Retransmissions

This paper studies packet repetition strategies over erasure channels with memory and a long feedback delay. The problem is initially formulated as a communications problem where a source wishes to transmit one message packet to a destination while minimizing both the delay and the number of transmissions. At each time instant, the sender is provided a delayed acknowledgement feedback about past attempts, and must decide whether to attempt a new transmission or not. This problem is then re-formulated as an episodic reinforcement learning problem, where an agent attempts to learn the optimal transmission policy, provided delayed feedback about past transmission attempts. The agent is helped by a channel estimator, which attempts to capture the channel memory and use that to predict probabilities of erasures in a future window. This channel estimator is also data-driven and learns the channel model without any <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a priori</i> channel knowledge. The paper presents a lower bound on the achievable trade-off between delay and number of transmissions for any channel modeled as a Markov process. Experimental results show that the combination of the proposed channel estimator and the agent can noticeably outperform naive strategies for channels with memory, and achieves results close to the lower bound.

Optimal transmission scheduling for remote state estimation in CPSs with energy harvesting two-hop relay networks

This paper considers the optimal transmission power scheduling for accurate remote state estimation in cyber–physical systems (CPSs). The plant is modeled as a discrete-time stochastic linear system with sensor measurements transmitted to the remote estimator over an intelligent relay. A dynamic cooperative team is formulated in which the sensor and the relay sharing the same cost function are equipped with corresponding energy harvesters, and need to decide on the transmission power levels to minimize the average error covariance at the remote estimator. Since the nodes make simultaneous decisions to achieve a common goal, the sensor and the relay do not have access to the transmission power that will be selected by another node in advance, and hence they cannot build a value estimation of the cost function directly at each time point, which poses a challenge to determine the optimal transmission policies of both sides. To overcome this challenge, we provide a local-action value function and prove that both nodes select the transmission power with respect to the local-action value function without any loss of performance. In addition, a Markov decision process (MDP) and a multi-agent reinforcement learning algorithm are presented to obtain the optimal transmission power schedule over an infinite time horizon. Finally, simulation examples are provided to corroborate and illustrate the theoretical results.

Coding for Secrecy in Remote State Estimation With an Adversary

We study the problem of remote state estimation in the presence of an eavesdropper. A sensor transmits state information over a packet-dropping link to a legitimate user. This information is randomly overheard by an eavesdropper. To reduce information leakage to the eavesdropper, previous studies have shown that by encoding the estimate with the acknowledgments (Acks), perfect secrecy can be achieved. However, this strategy greatly relies on the accuracy of the Acks and may easily fail if the Acks are compromised by cyberattacks. In this article, we tackle this issue by proposing to switch between sending an encoded state and sending the plain state to stay resilient against fake Acks. Specifically, we assume the Acks to be randomly attacked and derive recursive expressions for the minimum-mean-squared error estimates and error covariance matrices at the legitimate user and at the eavesdropper. Based upon this, we propose a transmission policy that depends on the probability of synchronization. We formulate a partially observable Markov decision process to model the evolution of the synchronization status and derive associated optimal transmission policies. Numerical examples are provided to verify the theoretical results.

Deep reinforcement learning based transmission policy enforcement and multi-hop routing in QoS aware LoRa IoT networks

The LoRa wireless connectivity has become a de facto technology for intelligent critical infrastructures such as transport systems. Achieving high Quality of Service (QoS) in cooperative systems remains a challenging task in LoRa. However, high QoS can be achieved via optimizing the transmission policy parameters such as spreading factor, bandwidth, code rate and carrier frequency. Yet existing approaches have not optimized the complete LoRa parameters. Furthermore, the star of stars topology used by LoRa causes more energy consumption and a low packet reception ratio. Motivated by this, this paper presents transmission policy enforcement and multi-hop routing for QoS-aware LoRa networks (MQ-LoRa). A hybrid cluster root rotated tree topology is constructed in which gateways follow a tree topology and Internet of Things (IoT) nodes follow a cluster topology. A ‘membrane’ inspired form the cell tissues which form clusters to sharing the correct information. The membrane inspired clustering algorithm is developed to form clusters and an optimal header node is selected using the influence score. Data QoS ranking is implemented for IoT nodes where priority and non-priority information is identified by the new field of LoRa frame structure (QRank). The optimal transmission policy enforcement uses fast deep reinforcement learning called Soft Actor Critic (SAC) that utilizes the environmental parameters including QRank, signal quality and signal-to-interference-plus-noise-ratio. The transmission policy is optimized with respect to the spreading factor, code rate, bandwidth and carrier frequency. Then, a concurrent optimization multi-hop routing algorithm that uses mayfly and shuffled shepherd optimization to rank routes based on the fitness criteria. Finally, a weighted duty cycle is implemented using a multi-weighted sum model to reduce resource wastage and information loss in LoRa IoT networks. Performance evaluation is implemented using a NS3.26 LoRaWAN module. The performance is examined for various metrics such as packet reception ratio, packet rejection ratio, energy consumption, delay and throughput. Experimental results prove that the proposed MQ-LoRa outperforms the well-known LoRa methods.

Learning nodes: machine learning-based energy and data management strategy

The efficient use of resources in wireless communications has always been a major issue. In the Internet of Things (IoT), the energy resource becomes more critical. The transmission policy with the aid of a coordinator is not a viable solution in an IoT network, since a node should report its state to the coordinator for scheduling and it causes serious signaling overhead. Machine learning algorithms can provide the optimal distributed transmission mechanism with little overhead. A node can learn by itself by utilizing the machine learning algorithm and make the optimal transmission decision on its own. In this paper, we propose a novel learning Medium Access Control (MAC) protocol with learning nodes. Nodes learn the optimal transmission policy, i.e., minimizing the data and energy queue levels, using the Q-learning algorithm. The performance evaluation shows that the proposed scheme enhances the queue states and throughput.

Designing GA based effective transmission strategies for intra-WBAN communication

Wireless Body Area Networks enable a new trend of proactive health care exploiting battery-powered wearable and/or implantable sensor nodes. The transmission of biomedical signals consumes most of the scarce energy resources in a WBAN. State-of-the-art transmission mechanisms either focus on energy efficiency or timely data delivery. However, an optimal balance between the two subject to various ambient conditions is a necessity. Accordingly, in this paper, an optimal data transmission policy is formulated at the node level, considering node lifetime, node heating, and reliable data delivery for various ambient conditions. The proposed model is divided into two phases. In the pre-deployment phase, each node is stochastically modeled following the Markov Decision Process. The formulation involves a reward function that attempts a trade-off between energy consumption and reliable data delivery. Then, a genetic algorithm is applied to obtain a set of optimal transmission decisions for each node based on the reward function for a range of ambient conditions. Finally, in the deployment phase, the results of the previous phase are mapped to an algorithm to be executed at WBAN nodes. The work is experimentally evaluated via simulation and compared with the state-of-the-art approaches. The performance is also observed based on the real-life dataset. Results demonstrate that around 85–90% of data packets could be delivered to the sink with minimal energy so that the nodes hardly get heated.

Energy Harvesting Irregular Repetition ALOHA With Replica Concatenation

In this paper, we consider an asynchronous random access scheme called irregular repetition ALOHA (IRA) as a generalization of contention resolution ALOHA (CRA) with varying repetitions. We present an asymptotic performance analysis of CRA and IRA on the collision channel for regular and irregular repetition rates. We also propose an improvement by merging the clean parts of packet replicas in partial collisions, and extend our analysis to this scenario as well. Specific designs of repetition distributions based on the new analysis show that the optimized solutions of irregular repetition slotted ALOHA (IRSA) perform well in both IRA and the enhanced scheme, and they considerably outperform the regular repetition distributions. We also introduce energy harvesting (EH) to both schemes as a practical and sustainable adaptation, where users are able to harvest energy and store it in their finite-capacity batteries. We model the battery state by a discrete-time Markov chain and derive an optimal transmission policy to maximize the asymptotic performance of the system. We provide comprehensive numerical results for both practical and asymptotic scenarios to verify the validity of the proposed analyses, and illustrate the benefits of the proposed systems.

Distortion minimisation of harvested electrical energy over wireless medium

In this work, two point-to-point energy communication links were considered. The first link transmits energy from different energy harvesting sources to a fusion centre with a unity channel gain while the second involves the transmission of energy from a point (the base station) to a mobile station under a known fading profile. The objective of the first link is to minimise the inherent distortion in the transmission from different energy harvesting sources to a fusion centre while the second link maximises the throughput of the harvested energy to the mobile station. In both cases, the problem was formulated in a convex optimisation framework and the optimal power transmission policy was derived. Lastly, the impacts of correlation in the optimal power transmission policy and processing energy cost in the power transmission policy for the maximisation of throughput were presented. We conclude that efficient harvest of electrical energy and transmission over wireless medium can be used to supply electrical energy to mobile devices.

Age-Optimal Transmission Policy With HARQ for Freshness-Critical Vehicular Status Updates in Space–Air–Ground-Integrated Networks

In this article, we investigate the freshness of the vehicular status updates in space&#x2013;air&#x2013;ground-integrated networks (SAGINs), where the status updates are generated by sampling a fixed-rate dynamic Markov process and delivered to the monitor over an unreliable channel instantaneously. The Age of Information (AoI) is adopted to capture the timeliness of the status updates. Two hybrid automatic repeat request (HARQ) schemes, namely, classical HARQ scheme and incremental redundancy HARQ (IR-HARQ) scheme, are taken into consideration to combat the errors occurred in the transmission. In this setting, once an update is not decoded successfully, one should carefully decide how to schedule the updates for optimizing the AoI. Especially, differential encoding scheme is introduced in the considered system to exploit the temporal correlations of the source. By differential encoding, each update can be actual or differential, based on the differential encoding level. To minimize the long-term average age, we formulate a Markov decision process (MDP), and prove that the optimal transmission policies for classical HARQ scheme and IR-HARQ scheme behave differently in threshold structures. Furthermore, we jointly optimize the codeword length, differential encoding level, and retransmission times to minimize the AoI. The performance comparison shows the advantages of the IR-HARQ scheme over the classical HARQ scheme from the age perspective.

Finding optimal transmission strategy for intra‐WBAN communications

This Letter proposes a stochastic model of the wearable sensor nodes of a wireless body area network (WBAN) based on Markov decision process (MDP). The proposed technique tackles the problem of maximising the likelihood of data transmission while minimising the computational cost and energy. Suitable transmission strategies subject to the energy level of the nodes and channel conditions are needed to be investigated to strike a balance between network lifetime and packet delivery ratio. MDP formulation of the problem is solved offline to find out the optimal transmission strategies for the representative input conditions. The results of this offline computation approach can directly map to optimal transmission policies for intra-WBAN communications. The experimental results demonstrate the robustness of the proposed scheme in handling both favourable and unfavourable operating conditions.

The Age of Incorrect Information: A New Performance Metric for Status Updates

In this paper, we introduce a new performance metric in the framework of status updates that we will refer to as the Age of Incorrect Information (AoII). This new metric deals with the shortcomings of both the Age of Information (AoI) and the conventional error penalty functions as it neatly extends the notion of fresh updates to that of fresh “informative” updates. The word informative in this context refers to updates that bring new and correct information to the monitor side. After properly motivating the new metric, and with the aim of minimizing its average, we formulate a Markov Decision Process (MDP) in a transmitter-receiver pair scenario where packets are sent over an unreliable channel. We show that a simple “always update” policy minimizes the aforementioned average penalty along with the average age and prediction error. We then tackle the general, and more realistic case, where the transmitter cannot surpass a specific power budget. The problem is formulated as a Constrained Markov Decision Process (CMDP) for which we provide a Lagrangian approach to solve. After characterizing the optimal transmission policy of the Lagrangian problem, we provide a rigorous mathematical proof to showcase that a mixture of two Lagrange policies is optimal for the CMDP in question. Equipped with this, we provide a low complexity algorithm that finds the AoII-optimal operating point of the system in the constrained scenario. Lastly, simulation results are laid out to showcase the performance of the proposed policy and highlight the differences with the AoI framework.

Reinforcement learning–enabled efficient data gathering in underground wireless sensor networks

Wireless underground sensor networks (WUSNs) consist of sensors that are buried in and communicate through soil medium, while the channel quality of WUSNs is greatly impacted by the underground environment, such as soil moisture and composition. Due to the precipitation and harsh weather, the underground environments change frequently, which make wireless communication in WUSNs much complicated than that in terrestrial over-the-air wireless sensor networks. To achieve reliable and energy-efficient data gathering in dynamic WUSNs, this article proposes an optimal transmission policy, where path loss of sensory data transmission, energy constraint, and network load balancing are the factors to be considered. Specifically, we capture the effect of underground environments on wireless communications, and evaluate path probability, energy consumption, and load balancing factor with respect to reliability and efficiency of transmission paths. The transmission topology can be reduced to a multi-objective and multi-constrained optimization problem and solved through an improved maximum flow minimum cost algorithm. By using reinforcement learning, we derive an adaptive transmission policy for underground sensors to efficiently use their energy and avoid transmitting sensory data in unreliable paths under a dynamic environment. Through simulations and comparison upon publicly available real data, our technique achieves more reliable wireless communication with significant reduction of packet loss, and enables more energy-efficient data gathering than other techniques, especially when soil moisture varies frequently.

Physical-Layer Security in the Finite Blocklength Regime Over Fading Channels

This paper studies physical-layer secure transmissions from a transmitter to a legitimate receiver against an eavesdropper over slow fading channels, taking into account the impact of finite blocklength secrecy coding. A comprehensive analysis and optimization framework is established to investigate secrecy throughput for both single- and multi-antenna transmitter scenarios. Both adaptive and non-adaptive design schemes are devised, in which the secrecy throughput is maximized by exploiting the instantaneous and statistical channel state information of the legitimate receiver, respectively. Specifically, optimal transmission policy, blocklength, and code rates are jointly designed to maximize the secrecy throughput. Additionally, null-space artificial noise is employed to improve the secrecy throughput for the multi-antenna setup with the optimal power allocation derived. Various important insights are developed. In particular, 1) increasing blocklength benefits both reliability and secrecy under the proposed transmission policy; 2) secrecy throughput monotonically increases with blocklength; 3) secrecy throughput initially increases but then decreases as secrecy rate increases, and the optimal secrecy rate maximizing the secrecy throughput should be carefully chosen in order to strike a good balance between rate and decoding correctness. Numerical results are eventually presented to verify theoretical findings.

A Multi-Objective Model Checking for Transmission Policy Optimization in Hybrid Powered Small Cell Networks

In the fifth generation (5G) communication system, maximizing energy utilization is one of the key challenges especially with limited green energy sources. In particular, it is important to determine the optimal transmission policy for the hybrid model with grid and green energy in small cell networks. The optimal transmission policy should minimize both grid power consumption and wastage of green harvested energy at the same time; while satisfying the quality of service (QoS) requirements such as minimum packet drop probability. In contrast to existing policies with a single objective for the hybrid powered small cell networks, the proposed transmission policy adopts a multi-objective model checking for Markov decision process (MOMC-MDP) with a linear temporal logic (LTL) scheme. The MOMC-MDP firstly determines all feasible decisions for packet transmissions in every time slot, and then chooses one of the best feasible decisions to obtain the optimal transmission policy. Three dimension Markov decision process (3D-MDP) that includes packet buffer, battery capacity, and channel state, are used with the age of information (AoI) of the packet arrivals to improve the overall performance of the proposed MOMC-MDP scheme. Numerical results validate that the proposed MOMC-MDP with AoI can provide higher green energy utilization compared to the conventional MDP-based schemes.

Optimal Online Transmission Schedule for Remote State Estimation over a Hidden Markovian Channel

This paper investigates the optimal transmission scheduling problem in remote state estimation systems over an unreliable wireless channel where the channel state evolves as a Markov chain. However, due to inaccurate observations of the channel state, the wireless channel is modeled as a hidden Markov chain. We propose a prediction algorithm based on the Viterbi algorithm to estimate the channel state. To save the wireless sensor’s energy, we consider scheduling the transmission of sensor transmissions while balancing between estimation performance and sensor energy expenditure. By jointly considering performance and energy, we formulate the scheduling problem as a Markov decision process. We prove the existence of the optimal transmission policy and derive a threshold structure of the optimal strategy. Finally, the performance of the proposed method is evaluated through simulations.

Optimal Selective Transmission Policy for Energy-Harvesting Wireless Sensors via Monotone Neural Networks

We investigate the optimal transmission policy for an energy-harvesting wireless sensor node. The node must decide whether an arrived packet should be transmitted or dropped, based on the packet’s priority, wireless channel gain, and the energy status of the node. The problem is formulated under the Markov decision process (MDP) framework. For such a problem, the conventional method to get the optimal policy is by using a state value function, which is three-dimensional in the considered problem, leading to high complexity. Fortunately, to reduce complexity, we derive an equivalent solution for the optimal policy via a one-dimensional after-state value function. We show that the after-state value function is differentiable and nondecreasing. We also discover a threshold structure of the optimal policy that is derived by the after-state value function. Furthermore, to approximate the after-state value function, we propose a learning algorithm to train a three-layer monotone neural network. The trained network thus finds a near-optimal selective transmission policy of the node. Finally, through simulation, we demonstrate the learning efficiency of the algorithm and the performance of the learned policy.