Automatic Repeat Request Protocol Research Articles

Short-packet communication for MIMO NOMA relay networks: BLER and AoI analysis

This work considers a multiple-input multiple-output (MIMO) non-orthogonal multiple access (NOMA) dual-hop relay system in which the size of signal packets is finite and the propagation environment follows the Nakagami-m distribution. Based on an approximation of the Gaussian Q-function, Riemann integral, and incomplete Gamma function, we derive the closed-form expressions of the block error rate (BLER), age of information (AoI), and latency of the proposed system in both exact and asymptotic cases. On the other hand, the automatic repeat request (ARQ) protocol and acknowledgment (ACK) are applied to confirm the accuracy of the receiving data packet process. Moreover, the maximal-ratio transmission (MRT), and maximal-ratio combining (MRC) are used to improve the BLER, AoI, and latency of the proposed system. The accuracy of mathematical analysis frameworks is confirmed by simulation results. The calculation results show that the considered system satisfies the requirement of ultra-reliable and low-latency communications (URLLC). The results also represent that the MIMO-NOMA system outperforms the MIMO-OMA system in terms of BLER performance. The results also provide insight into the optimizations of the power allocation coefficient and the number of transmission bits that minimize the BLER and AoI of the considered system.

Adaptive Automatic Repeat Request (AdARQ) Protocol to Improve the Throughput Characteristic of the Time-Varying Wireless Channel

Adaptive Automatic Repeat Request (AdARQ) Protocol to Improve the Throughput Characteristic of the Time-Varying Wireless Channel

Investigating Imperfect Cloning for Extending Quantum Communication Capabilities.

Quantum computing allows the implementation of powerful algorithms with enormous computing capabilities and promises a secure quantum Internet. Despite the advantages brought by quantum communication, certain communication paradigms are impossible or cannot be completely implemented due to the no-cloning theorem. Qubit retransmission for reliable communications and point-to-multipoint quantum communication (QP2MP) are among them. In this paper, we investigate whether a Universal Quantum Copying Machine (UQCM) generating imperfect copies of qubits can help. Specifically, we propose the Quantum Automatic Repeat Request (QARQ) protocol, which is based on its classical variant, as well as to perform QP2MP communication using imperfect clones. Note that the availability of these protocols might foster the development of new distributed quantum computing applications. As current quantum devices are noisy and they decohere qubits, we analyze these two protocols under the presence of various sources of noise. Three major quantum technologies are studied for these protocols: direct transmission (DT), teleportation (TP), and telecloning (TC). The Nitrogen-Vacancy (NV) center platform is used to create simulation models. Results show that TC outperforms TP and DT in terms of fidelity in both QARQ and QP2MP, although it is the most complex one in terms of quantum cost. A numerical study shows that the QARQ protocol significantly improves qubit recovery and that creating more clones does not always improve qubit recovery.

Age of Information for Short-Packet Relay Communications in Cognitive-Radio-Based Internet of Things With Outdated Channel State Information

It is crucial to consider time-critical communications in cognitive-radio-based Internet of Things (CR-IoT), which increasingly rely on the exchange of short-packet information to efficiently monitor and control. Therefore, the age of information (AoI) is exploited as a timeliness metric in this paper to investigate the freshness of received information at the remote server with dual-hop short-packet communications in underlay CR-IoT. Since short-packet communications are sensitive to time delay, the influence of outdated factors in the channel state information (CSI) cannot be ignored, so we develop the system under the condition of outdated CSI with Nakagami-m fading channels. Considering the impact of outdated CSI and packets error, we investigate the performance of dual-hop short-packet communications under the classical automatic repeat-request (ARQ) protocol and truncated ARQ (TARQ) protocol, and obtain the approximated expressions of the average AoI and the average peak AoI (PAoI), respectively. Accordingly, the blocklength and the number of retransmissions can be optimized by Fibonacci and iterative algorithms to minimize the average PAoI under two protocols, respectively. Simulation results substantiate the influence of each parameter on the system performance differently under two different protocols. Moreover, the AoI of the dual-hop is also better than that of the single-hop due to the interference threshold of the primary user.

Intelligent Reflecting Surface-Aided Full-Duplex Covert Communications: Information Freshness Optimization

This work investigates the covert information freshness in intelligent reflecting surface (IRS)-aided communications, where a public full-duplex user (Alice) and a private full-duplex user (Bob) exchange information in the presence of a watchful warden (Willie). In particular, with the help of Alice’s undisguised signal transmission, Bob can establish covert communications such that his transmission can be shielded from Willie. Considering both the non-retransmission protocol and the automatic repeat-request (ARQ) protocol for Bob’s transmission, we study the resource allocation design. By exploiting the channel statistics, the joint design of active beamforming at Alice and Bob, the passive beamforming at the IRS, and the packet length of the confidential data packet is formulated as a nonconvex optimization problem which minimizes the age of information (AoI) at Alice for the two considered protocols taking into account the quality of service in terms of the maximum tolerable AoI at Bob and communication covertness. To circumvent the non-convexity of the design problem, we propose alternating optimization algorithms to find effective solutions. Numerical results demonstrate the superiority of our proposed optimization algorithms over various benchmarks and unveil the decrease of the optimized packet length with the improved covert channel quality.

Age and Energy Analysis for $L$th Best Relay Enabled Cooperative Status Update Systems With Short Packet Communications

In the Internet of Things monitoring systems, the fresh information delivery and efficient energy usage are two significant design concerns. However, they generally cannot be simultaneously achieved. In this paper, we investigate the fundamental tradeoff between the age of information (AoI) and energy consumption (EC) for a generalized <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell$</tex-math></inline-formula> th best relay selection policy in two-hop cooperative status update systems with automatic repeat request (ARQ), where the source delivers the sensed status update to the destination through multiple decode-and-forward relays. Among these relays, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell$</tex-math></inline-formula> th best relay (i.e., the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell$</tex-math></inline-formula> -th earliest one that correctly decodes the update packet) will be selected as a forwarder to relay the data to the sink. When a decoding failure occurs, we can either retransmit the old packet to save sensing EC or sense and transmit a new one to reduce AoI. Accordingly, the classical ARQ and truncated ARQ protocols are employed in the second hop. By considering short packet-based transmissions, the average AoI and average EC expressions are derived, and the weighted combination of them is minimized to attain the tradeoff. Numerical results demonstrate that an optimal packet size that can trade off age and energy always exists. Moreover, the best relay does not always minimize the weighted combination, and the truncated ARQ is superior to the classical one. As a promising candidate, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell$</tex-math></inline-formula> th best relay selection can be implemented when either the system energy resource is limited or the best relay is unavailable.

Reliable and energy-efficient UAV-assisted air-to-ground transmission: Design, modeling and analysis

In this paper, a novel reliable and energy-efficient transmission scheme is proposed for unmanned aerial vehicle (UAV) assisted air-to-ground (A2G) communication. The scheme is tailored to handle burst errors in transmission caused by network collision, switching or interference. Application Layer Forward Error Correction (AL-FEC) mechanism is proposed to enhance the reliability of the A2G communication. By avoiding retransmissions, energy efficiency can be guaranteed. Based on Gilbert-Elliott channel model, we perform a detailed mathematical analysis on the performance of the proposed AL-FEC scheme. The packet delivery ratio and energy efficiency are studied and validated by comprehensive simulations. An application layer Automatic Repeat reQuest (AL-ARQ) protocol based on selective retransmission is used as a baseline scheme and compared with the proposed scheme. It is shown that the proposed AL-FEC scheme can significantly reduce the energy consumption and ensure the same or even higher application packet delivery ratio compared with the baseline scheme. In addition, the proposed scheme is especially preferable when the channel quality is poor. These specific findings can provide guidance for practical applications relying on UAV-assisted A2G communication, which require reliable and energy-efficient UAV-to-ground transmission in harsh wireless conditions.

Energy and MCS Optimization in HARQ Protocol for Ultrareliable Regime With Maximized Throughput

In this article, we propose a cross-layer approach to maximize the throughput efficiency of the hybrid automatic repeat request (HARQ) protocol while guaranteeing ultrareliable communication (URC). The study combines adaptive modulation and coding and power control, at the physical layer, with automatic repeat request (ARQ)/HARQ at the data-link layer. The proposed optimization approach, based on genetic algorithms, allows a judicious allocation of the available energy budget and a suitable modulation and coding scheme (MCS) choice while keeping the same energy budget. For a given signal-to-noise ratio (SNR), a vector of energy and an MCS are modeled as one individual. The individuals with the highest throughput efficiency are selected to be used in generating the next population through crossover of the survivals and mutation. The individual with the highest throughput efficiency in the last population presents the solution offering the optimized parameters used in the packet transmission. We compare our solution to an existing one that exploits Lagrange multipliers. The results show that the proposed genetic algorithm based solution ensures URC with packet erasure probability less than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$10^{-5}$</tex-math></inline-formula> from SNR values of 7 and 13 dB in static and fading channels, respectively.

Second-Order Statistics for IRS-Assisted Multiuser Vehicular Network With Co-Channel Interference

In this paper, we consider a downlink multi-user vehicular communication network empowered by an intelligent reflecting surface (IRS) with the base station (BS) employing the orthogonal frequency division multiple access (OFDMA) scheme. The performance of the considered system is evaluated in terms of second-order statistics (SOS) in the presence of multiple co-channel interferers with varying powers and speeds. In particular, we develop the analytical expressions of level crossing rate (LCR) and average outage duration (AOD) of the received signal envelope at a desired vehicle by considering a generalized fading model as - distribution. The impact of various system and channel parameters such as number of reflecting elements, amplitude of reflection coefficients, number of interferers, Doppler frequencies of desired and interfering vehicles, and different channel fading environments has been shown on the LCR and AOD performances. Moreover, we derive an expression for asymptotic LCR under high transmit power conditions. It has been shown that the asymptotic LCR is independent of the threshold and transmit power. In addition, we consider a stop and wait automatic repeat request (SW-ARQ) protocol for reliable data packet transmission and derive the expressions for packet error rate (PER) and throughput by utilizing the finite-state Markov channel (FSMC) model. The optimal packet length is also determined which maximizes the throughput under SW-ARQ protocol. The impact of various system parameters is also shown on the PER and throughput performances.

Short Packet-Based Status Updates in Cognitive Internet of Things: Analysis and Optimization

In the cellular Internet of Things (IoT), some machine type communication devices (MTCDs) are used to sample and send status information of monitored targets to help base station (BS) perform monitoring or controlling tasks. However, due to limited spectrum resources, the MTCDs often cannot occupy separate spectrum resources. This paper studies a cognitive IoT where the MTCDs exploit the spectrum resource vacated by cellular users to send status packets to the BS. The timeliness of status updates is characterized by the recently proposed metric, age of information (AoI). In the IoT, the packets sent by the MTCDs are often short, which means it is necessary to analyze the status updates under short packet communication theory (SPC). In this case, we adopt the automatic repeat request protocol to retransmit the erroneous packets, and the preemption policy is exploited to improve the status update performance. We first carefully analyze the evolution of AoI in the cognitive IoT, and adopt a recursive method to obtain the closed-form expression of the average peak AoI. Then, we analyze age-energy tradeoff, and propose a scheme of jointly optimizing transmit power, packet length, and spectrum sharing to reduce the average peak AoI of the network. The simulation results verify the correctness of the theoretical analysis, and show that the proposed scheme outperforms the benchmark schemes.

Free-Ride Feedback and Superposition Retransmission Over LDPC Coded Links

In this paper, we propose a new transmission scheme for the scenario where two nodes attempt to exchange messages and the conventional low-density parity-check (LDPC) codes are implemented for error correction. In the proposed scheme, the ACK/NACK feedback information is transmitted along with the payload data by free-ride codes, while the re-transmitted codewords are superimposed (XORed) on the current codewords, both of which cost neither extra bandwidth nor transmission power. Firstly, we present a syndrome channel model and derive its capacity (referred to as accessible capacity) with a lower bound, implying that the reliable transmission of extra bits (feedback information) is possible. Then, the performance of the extra bits is analyzed by the dependency testing (DT) bound for the syndrome channels. Moreover, motivated by the DT bound, we present a low-complexity DT-like decoder for the free-ride codes. For the superposition retransmission, we present a practical implementation, where those unsuccessfully decoded codewords are sparsely interleaved and superimposed onto the current codewords. In addition, the presented transmission scheme is combined with the conventional hybrid automatic repeat request (HARQ) protocol, resulting in a throughput-enhanced conjunction HARQ scheme. Numerical results show that the word error rate (WER) of the LDPC codes can be significantly reduced by using the presented transmission scheme, but without any extra bandwidth or transmission power. They also show that the presented conjunction HARQ schemes can achieve a throughput improvement up to 80% over fading channels in comparison with the original 5G HARQ scheme.

Age-Optimal Hybrid Temporal-Spatial Generalized Deduplication and ARQ for Satellite-Integrated Internet of Things

In a typical Satellite-integrated Internet of Things (SIoT), the limited transmission rate of a sensor causes a stale in data freshness due to unavoidable time waiting for transmission. Moreover, due to the high bit error rate (BER) of the satellite-to-ground link, data freshness will be further exacerbated by frequent retransmissions. Generally, this issue is partially solved using a powerful compression scheme that can reduce the data volume. However, conventional compression schemes will necessitate a significant amount of time to accumulate constant data to a certain quantity, posing a difficult challenge. Therefore, this study proposes an age-optimal hybrid temporal-spatial generalized deduplication and automatic repeat request (HARQ-GD) protocol for the high-sampling data collection in SIoT, considering data compression, and transmission collaboratively. A novel Age of Information (AoI) metric is developed for timeliness evaluation over a two-hop end-to-end link of SIoT, which is optimized to design the proposed HARQ-GD protocol by considering the temporal and spatial correlations of sampled data with specific encoding/decoding algorithms and packet formats. The simulation results indicate that the proposed HARQ-GD protocol performs better performance than typical generalized deduplication (GD) and hybrid automatic repeat request with chase combing (HARQ-CC) schemes in reducing AoI, because of its fewer transmission times and higher compression rate.

On the Data Freshness for Industrial Internet of Things With Mobile-Edge Computing

This article studies the freshness of information with the aid of Age of Information (AoI) in the Industrial Internet of Things (IIoT), which plays a vital role to ensure quality and timely delivery of data services. To reduce the AoI, we leverage mobile-edge computing (MEC) to partially offload information to the mobile edge server. Aiming to cope with the packet error in the setting of short packet communication (SPC) in IIoT, we consider the standard automatic repeat request (ARQ) protocol with two policies, i.e., either retransmitting an out-of-date packet (RO) or transmitting a freshest packet (TF), when a packet error occurs. We derive the closed form of average AoI under these two policies, respectively, and then formulate the average AoI minimization problem by jointly optimizing the short packet blocklength and MEC offloading ratio. Due to the nonconvexity nature of the problem, we tackle it by employing block coordinate descent (BCD) and successive convex approximation (SCA) methods and then prove their convergence. Our extensive numerical results show that the optimal average AoI yielded by our proposed approach is almost identical to that from the high-complexity exhaustive search method, and has significant improvement over the benchmark methods. From the AoI perspective, it is revealed that the optimal strategy tends to offload all information to mobile edge server when the computing capacity of local device is less than a threshold. Furthermore, it is found that the RO policy is suitable for the relatively small bandwidth and large local computing capability scenario, whilst the TF policy is better for the large bandwidth and small local computing capability case.

Low computational complexity joint iterative detection and decoding without ARQ in massive MIMO systems with UAVs

Unmanned aerial vehicles (UAVs) have a wide range of military and commercial applications. UAVs play an important role in 6G networks due to their low cost and flexible positioning. UAVs can play different roles in the network and act as an air base station, as a relay, or as users in cellular networks. One of the major challenges in these systems is to increase lifespan with low energy consumption. The automatic repeat request (ARQ) protocol is used to increase throughput and obtain dependable communications for multi-input multiple-input multiple-output (MIMO) systems, even in the presence of severe propagation conditions. On the other hand, (UAV)-enabled communication system provides flexibility and reliability compared to conventional ones which have been considered in the new generation of wireless communications. Thus, using ARQ protocols in this system faces serious challenges such as increasing delay and operational implementation complexities in the receiver. Therefore, to meet the ARQ challenge in the uplink of massive MIMO systems, we have investigated utilizing joint iterative detection and decoding methods with reduced computational complexity have been proposed in a UAV-enabled communication system in the condition that users are equipped with two antennas to provide a solution to reduce energy consumption. In this structure, system performance improves as the computational complexity increases. Therefore, we look for methods that reduce complexity while maintaining system performance at an acceptable level. In this study, we seek to provide a solution that can establish a reasonable compromise between system performance and computational complexity. This aim is achieved by establishing a connection between the components of soft joint detection and decoding, linear detection with approximation methods, and sorting. The first part of the receiver is sorting the users before detection, then, in the next module, a turbo-process-based low-complexity MIMO iterative detection and decoding (IDD) algorithm, with minimum mean square error (MMSE) detector and soft channel decoder, is used. For solving the challenge of computational complexity, utilizing of approximation-based detection method is proposed. This structure works in such a way that after sorting the users, in the first iteration, a certain number of users are decoded using a hard decoding scheme after the soft detection and decoding. Therefore, these data are subtracted from the soft feedback information (prior data) to the detector, as inter user interference (IUI) and the Joint IDD module continue the iterative cycle until all users’ data are decoded.

Throughput Analysis for TCP Over the FSO-Based Satellite-Assisted Internet of Vehicles

Free-space optics (FSO)-based, low-earth orbit (LEO) satellite-assisted Internet of vehicles (IoVs) has recently attracted research efforts worldwide. For the majority of Internet applications, transmission control protocol (TCP) is one of the most widely deployed protocols. Nevertheless, a complete understanding of the TCP performance over FSO-based satellite-assisted IoVs networks is still lacking in the literature. This is a highly challenging issue, as a consequence of the non-trivial interdependencies between transmission losses caused by the dynamics/uncertainty of atmospheric last-mile channels and the buffer overflows on the Internet, as well as their joint impact on the TCP congestion control mechanisms. This paper offers a comprehensive analytical framework that allows assessing the TCP throughput performance under the joint impact of congestion losses occurred in the Internet and transmission errors at the last-mile link from the satellite-to-vehicle. Internet access for emerging unmanned aerial vehicles (UAVs) is considered while the analysis can be also applied to other kinds of vehicles, which, in fact, require simpler channel models. The link-layer incremental redundancy hybrid automatic repeat request (IR-HARQ) protocol is employed at satellite-to-UAV channels to further improve the TCP throughput performance. Numerical results provide non-trivial findings and insights for the TCP performance in satellite/FSO-based IoVs, in which optimal UAV’s parameters are chosen to maximize the TCP throughput.

Mixed Dual-Hop IRS-Assisted FSO-RF Communication System With H-ARQ Protocols

Intelligent reflecting surface (IRS) is an emerging key technology for the fifth-generation (5G) and beyond wireless communication systems to provide more robust and reliable communication links. In this letter, we propose a mixed dual-hop free-space optical (FSO)-radio frequency (RF) communication system that serves the end user via a decode-and-forward (DF) relay employing hybrid automatic repeat request (H-ARQ) protocols on both hops. Novel closed-form expressions of the probability density function (PDF) and cumulative density function (CDF) of the equivalent end-to-end signal-to-noise ratio (SNR) are computed for the considered system. Utilizing the obtained statistics, we derive the outage probability (OP) and packet error rate (PER) of the proposed system by considering generalized detection techniques on the source-to-relay (S-R) link with H-ARQ protocol and IRS having phase error. We obtain useful insights into the system performance through the asymptotic analysis which aids to compute the diversity gain. The derived analytical results are validated using Monte Carlo simulation.

Semi-blind channel estimation based on modified CMA and unitary scrambling for massive MIMO systems

Pilot contamination is one of the main impairments in multi-cell massive Multiple-Input Multiple-Output systems. In order to improve the channel estimation in this context, we propose to use a semi-blind channel estimator based on the constant modulus algorithm (CMA). We consider an enhanced version of the CMA namely the Modified CMA which modifies the cost function of the CMA algorithm to the sum of cost functions for real and imaginary parts. Due to pilot contamination, the channel estimator may estimate the channel of a contaminating user instead of that of the user of interest (the user for which the Base Station wants to estimate the channel and then the data). To avoid this, we propose to scramble the users sequences before transmission. We consider different methods to perform unitary scrambling based on rotating the transmitted symbols (one Dimensional (1-D) scrambling) and using unitary matrices (two-Dimensional (2-D) scrambling). At the base station, the received sequence of the user of interest is descrambled leading to a better convergence of the channel estimator. We also consider the case where the Automatic Repeat reQuest protocol is used. In this case, using scrambling leads to a significant gain in terms of BLock Error Rate due to the change of the contaminating users data from one transmission to another induced by scrambling.

Performance Modeling Analysis of D-MSMR-CARQ with Relay Selection in Wireless Sensor Networks

Reliable and efficient real-time transmission is an important and challenging issue for wireless sensor networks (WSNs). Truncated retransmission times and relay selection can effectively reduce transmission delay and improve system throughput. A new direct multisource multirelay cooperative automatic repeat request (D-MSMR-CARQ) protocol based on truncation with two relay selection methods in WSNs is analytically analyzed in this paper. Firstly, based on two different relay selection methods under the maximum ratio combining (MRC), the discrete time Markov chain (DTMC) model of D-MSMR-CARQ protocol and state space is established. Secondly, for each D-MSMR-CARQ protocol based on different relay selection method, we obtain the closed-form expressions of the system average transmission delay and the expressions of the system throughput through state transition probabilities. Finally, numerical results reveal that the first relay selection method outperforms the second relay selection method on the average transmission delay performance for the proposed protocol. More specifically, the delay performance of the proposed protocol can be improved by 13% compared with the nondirect-link protocol when the channel environment is the same; the proposed protocol improves the throughput performance by 47% compared with the nondirect protocol when the channel environment is harsh under the same simulation parameters. Furthermore, the optimal number of source nodes and relay nodes is determined.

Secure IoT Communications Using HARQ-Based Beamforming for MISOSE Channels

In this article, we consider physical-layer security (PLS) of the Internet-of-Things (IoT) communications based on hybrid automatic repeat request (HARQ) protocols in multiple-input single-output single-eavesdropper (MISOSE) channels. Owing to the reciprocity of wireless channels, a negative acknowledge frame can be used to obtain the channel state information (CSI) from a source with multiple antennas to an IoT destination device, and this CSI of a legitimate link is used for beamforming for the next transmission round (TR). Therefore, a higher secrecy rate can be expected from retransmissions with beamforming based on the CSI as compared to the first TR without CSI. To evaluate the secrecy throughput under time-invariant and time-varying channels, connection and secrecy outage probabilities (OPs) are derived. By exploiting the characteristics of the OPs, we suggest an approximated secrecy throughput. We then consider the optimization of the approximated secrecy throughput considering the rate parameters for legitimate and eavesdropping links. With an alternating optimization approach, an algorithm to find convergent system parameters, i.e., the rates for legitimate and eavesdropping links, is proposed. Moreover, an asymptotic analysis with the large number of transmit antennas indicates an interesting behavior of the secrecy rate with regard to the number of TRs. The analytical results are verified by rigorous numerical simulations and they can serve as a useful guideline to those involved in the design of secure IoT communication systems with HARQ in MISOSE channels.

A Reinforcement Learning Approach to Age of Information in Multi-User Networks With HARQ

Scheduling the transmission of time-sensitive information from a source node to multiple users over error-prone communication channels is studied with the goal of minimizing the long-term average age of information (AoI) at the users. A long-term average resource constraint is imposed on the source, which limits the average number of transmissions. The source can transmit only to a single user at each time slot, and after each transmission, it receives an instantaneous ACK/NACK feedback from the intended receiver, and decides when and to which user to transmit the next update. Assuming the channel statistics are known, the optimal scheduling policy is studied for both the standard automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols. Then, a reinforcement learning (RL) approach is introduced to find a near-optimal policy, which does not assume any a priori information on the random processes governing the channel states. Different RL methods including average-cost SARSA with linear function approximation (LFA), upper confidence reinforcement learning (UCRL2), and deep Q-network (DQN) are applied and compared through numerical simulations.