Distributed Dynamic Spectrum Access Research Articles

Deep‐reinforcement learning for fair distributed dynamic spectrum access in priority buffered heterogeneous wireless networks

In this paper, distributed heterogeneous wireless networks are studied and a spectrum access scheme that fairly allocates channels among users based on their request levels is investigated. To be closer to what happens in practice, users' request levels to have different priorities are also considered, i.e. some packets are more important for the user to be transmitted than the others. Due to the distributed nature of the problem and since users are not able to coordinate with each other before transmission, a scheme based on reinforcement learning is proposed in which the network state is estimated based on the previous successful transmissions of packets in the network. The reinforcement learning method's adaptive nature lets our method work in heterogeneous settings where there exist users with other medium access control protocols. The performance of the proposed scheme is evaluated in different network settings, and it is shown that it tries to implement a fair spectrum access policy (considering the packets' priorities) while maximising the total throughput of the network.

Distributive Dynamic Spectrum Access Through Deep Reinforcement Learning: A Reservoir Computing-Based Approach

Dynamic spectrum access (DSA) is regarded as an effective and efficient technology to share radio spectrum among different networks. As a secondary user (SU), a DSA device will face two critical problems: avoiding causing harmful interference to primary users (PUs), and conducting effective interference coordination with other secondary users. These two problems become even more challenging for a distributed DSA network where there is no centralized controllers for SUs. In this paper, we investigate communication strategies of a distributive DSA network under the presence of spectrum sensing errors. To be specific, we apply the powerful machine learning tool, deep reinforcement learning (DRL), for SUs to learn "appropriate" spectrum access strategies in a distributed fashion assuming NO knowledge of the underlying system statistics. Furthermore, a special type of recurrent neural network (RNN), called the reservoir computing (RC), is utilized to realize DRL by taking advantage of the underlying temporal correlation of the DSA network. Using the introduced machine learning-based strategy, SUs could make spectrum access decisions distributedly relying only on their own current and past spectrum sensing outcomes. Through extensive experiments, our results suggest that the RC-based spectrum access strategy can help the SU to significantly reduce the chances of collision with PUs and other SUs. We also show that our scheme outperforms the myopic method which assumes the knowledge of system statistics, and converges faster than the Q-learning method when the number of channels is large.

Distributed Cognitive Radio Spectrum Access with Imperfect Sensing Using CTMC

Cognitive radio (CR) has become an efficient approach for the utilization of scarce spectrum by enabling spectrum access in an opportunistic manner. With increasing wireless network service providers, users, and applications, it is indispensable to optimize usage of even parts of the spectrum that are licensed. Dynamic spectrum access (DSA) is an approach which facilitates the opportunistic use of licensed spectra, when they are idle. In this paper, a primary prioritized distributed DSA algorithm using continuous-time Markov chain (CTMC) model is proposed. In a distributed scheme, each secondary user needs to be aware of the statistics—arrival and service rates—of the other secondary users to optimize the throughput while maintaining fairness. A heuristics-based approach is formulated making use of the estimated spectrum idle probability and the interference each user experiences to iteratively update user statistics. This framework is also extended to incorporate the effects of imperfect spectrum sensing in the form of misdetection and false alarms. Simulation results show that the proposed algorithm attains an overall throughput that is better than CSMA when the primary user spectrum utilization is around 45%. The degradation in throughput caused by imperfect sensing over perfect sensing is also analyzed.

Coordinating Secondary-User Behaviors for Inelastic Traffic Reward Maximization in Large-Scale \osa Networks

We develop efficient coordination techniques that support inelastic traffic in large-scale distributed dynamic spectrum access (DSA) networks. By means of any learning algorithm, the proposed techniques enable DSA users to locate and exploit spectrum opportunities effectively, thereby increasing their achieved throughput (or “rewards” to be more general). Basically, learning algorithms allow DSA users to learn by interacting with the environment, and use their acquired knowledge to select the proper actions that maximize their own objectives, thereby “hopefully” maximizing their long-term cumulative received reward. However, when DSA users' objectives are not carefully coordinated, learning algorithms can lead to poor overall system performance, resulting in lesser per-user average achieved rewards. In this paper, we derive efficient objective functions that DSA users can aim to maximize, and that by doing so, users' collective behavior also leads to good overall system performance, thus maximizing each user's long-term cumulative received rewards. We show that the proposed techniques are: (i) efficient by enabling users to achieve high rewards, (ii) scalable by performing well in systems with a small as well as a large number of users, (iii) learnable by allowing users to reach up high rewards very quickly, and (iv) distributive by being implementable in a decentralized manner.

Maximizing Secondary-User Satisfaction in Large-Scale DSA Systems Through Distributed Team Cooperation

We develop resource and service management techniques to support secondary users (SUs) with QoS requirements in large-scale distributed dynamic spectrum access (DSA) systems. The proposed techniques empower SUs' to seek and exploit spectrum opportunities dynamically and effectively, thereby maximizing the SUs' long-term received service satisfaction levels. Our techniques are efficient in terms of optimality, scalability, distributivity, and fairness. First, they enable SUs to achieve high service satisfaction levels by quickly locating and accessing available spectrum opportunities. Second, they are scalable by performing well in systems with small as well as large numbers of SUs. Third, they can be implemented in a decentralized manner by relying on local information only. Finally, they ensure fairness among SUs by allowing them to receive equal amounts of service.