ACK Signal Research Articles

Deep Recurrent Reinforcement Learning-Based Distributed Dynamic Spectrum Access in Multichannel Wireless Networks With Imperfect Feedback

This paper investigates Dynamic Spectrum Access (DSA) paradigm with imperfect feedback for multiuser wireless network. Each user selects an orthogonal channel in particular time slot to transmit packet with a certain transmission probability. In next time slot, the user who has transmitted a packet receives an ACK signal based on local observation. Bearing in mind the dynamic nature of wireless networks, it is appealing to develop a blended strategy to perform effective DSA. This paper aims to design a distributed Deep Reinforcement Learning (DRL) based scheme with an objective of maximizing network utility. In conventional DRL framework, it is assumed that the feedback received (ACK packet) is always correct but in wireless networks, it may be lost or corrupted due to noise. Furthermore, it is challenging to promise that the multiple agents will cooperate to make coherent decisions in order to accomplish the same objective, particularly under imperfect feedback. To tackle these challenges, this work proposes (i) Deep Recurrent Reinforcement Learning network with integrated GRU layer to optimize the network utility function and (ii) a feedback recovery mechanism using complete and incomplete replay buffers. Extensive simulations corroborate the success of proposed scheme in complex multiuser scenario and exhibit robustness against the detrimental effects of the imperfect feedback.

Deep Reinforcement Learning for Dynamic Spectrum Sensing and Aggregation in Multi-Channel Wireless Networks

In this paper, the problem of dynamic spectrum sensing and aggregation is investigated in a wireless network containing N correlated channels, where these channels are occupied or vacant following an unknown joint 2-state Markov model. At each time slot, a single cognitive user with certain bandwidth requirement either stays idle or selects a segment comprising C (C < N) contiguous channels to sense. Then, the vacant channels in the selected segment will be aggregated for satisfying the user requirement. The user receives a binary feedback signal indicating whether the transmission is successful or not (i.e., ACK signal) after each transmission, and makes next decision based on the sensing channel states. Here, we aim to find a policy that can maximize the number of successful transmissions without interrupting the primary users (PUs). The problem can be considered as a partially observable Markov decision process (POMDP) due to without full observation of system environment. We implement a Deep Q-Network (DQN) to address the challenge of unknown system dynamics and computational expenses. The performance of DQN, Q-Learning, and the Improvident Policy with known system dynamics is evaluated through simulations. The simulation results show that DQN can achieve near-optimal performance among different system scenarios only based on partial observations and ACK signals.

Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access

We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into $K$ orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive, in general, due to the large-state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to the spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. The experimental results demonstrate the strong performance of the algorithm.

Grant-Free Rateless SCMA for Cellular Internet of Things Networks

For supporting the massive connectivities and better adapting the high dynamical channel states of the cellular Internet of things networks, in this paper we proposed a new grant-free rateless sparse code multiple access (RSCMA) protocol. In the proposed grant-free RSCMA, users switch from the inactive mode to the active mode as soon as they receive data packages, and start to transmit the data without the request-grant process between the base station (BS) and users. In each transmission, the active users firstly choose a certain number of the modulated SCMA symbols according to the pre-assigned access control function (ACF), and then transmit the linearly superimposed SCMA symbols to the BS over the available resource elements. At the BS, owing to the sparsity of RSCMA, message passing algorithm which allows multi-user detection with near-optimal performance can be utilized to decode the messages. The transmission will continue until the user successfully receives an ACK signal sent back from the BS. Theoretical analysis in terms of achievable constellation constrained throughput is provided to show the maximal sum rate of RSCMA and to optimize the ACF design under the trade-off between the complexity and throughput performance. Both theoretical and simulation results show that the RSCMA can achieve higher throughput than the existing rateless multiple access schemes, such as hybrid automatic repeat request SCMA, LT code coded SCMA, and rateless multiple access.

A Novel Phase Rotation Scheme on the Constellations for the E-UTRA Uplink ACK/NACK Signals

Hybrid automatic repeat request (HARQ) is employed for the Evolved Universal Terrestrial Radio Access (E-UTRA) downlink. The ACK/NACK signals from each user equipment (UE) are multiplexed by code division multiple access (CDMA) and transmitted via a physical uplink control channel (PUCCH). The ACK/NACK signals are code spread by the cyclic shift (CS) sequences made from zero auto-correlation (ZAC) sequences; however, the orthogonality of these sequences is not guaranteed depending on the propagation channels; moreover, the amount of inter-code interference (ICI) depends on the delay spread of the channel and the transmitting timing control error of each UE. In the conventional PUCCH structure, ICI between two ACK signals does not degrade their detection performance, whereas ICI between an ACK signal and a NACK signal degrades the detection performance. This causes a serious gap between the detection performances of ACK and NACK signals, because generally in a PUCCH, there are more ACK signals than NACK signals. In this paper, we propose a novel phase rotation scheme on the constellations of ACK/NACK signals that can resolve this issue, and the simulation evaluation results confirm the benefits of the proposed phase rotation scheme.