Dynamic Spectrum Access Algorithm Research Articles

Dynamic Spectrum Access for C-V2X via Imitating Indian Buffet Process

In dense traffic cases, the channels among vehicles and infrastructures in cellular vehicle-to-everything (C-V2X) network are likely to be congested. Vehicles can select spectrum resources autonomously by adopting sensing-based semipersistent scheduling scheme, which may result in frequent packet collisions due to the random selection process, especially when spectrum resources are limited. So far, there still lacks a definite stochastic expression to characterize the channel selection process for vehicles in C-V2X. In this paper, we propose a novel deep reinforcement learning (DRL) based dynamic spectrum access (DSA) algorithm for C-V2X via imitating Indian buffet process (IBP), aiming to meet the vehicles’ strict communication requirements on high transmission rate and low collision probability. First, we explore the correlations among historical spectrum access data to acquire the channel availability list. Then, we exploit DRL to achieve distributive DSA among vehicles. Specifically, the channel state prediction for the distributive DSA is facilitated via imitating the classic IBP, and the spectrum access decisions are made by leveraging the deep Q-learning network combined with long short-term memory technique. Finally, comprehensive simulation results are presented to show that the proposed algorithm can improve the transmission rate by 15% and reduce the collision probability by 12% with a false alarm probability of 0.1 compared with other spectrum access methods.

A multi-channel and multi-user dynamic spectrum access algorithm based on deep reinforcement learning in Cognitive Vehicular Networks with sensing error

In this paper, a spectrum access problem is proposed to improve the spectrum access rates of secondary vehicles in Cognitive Vehicular Networks, where the channel capacity mathematic model is established under the conditions of spectrum sensing errors rates and the dynamic occupancy spectrum rates. Meanwhile, an improved Q-learning method is proposed to conform the dynamic communication under the different conditions of the reward functions. In this function, a Deep Q Network method with a modified reward function (IDQN) is proposed to deal with the situation of multi-vehicle in multi-channel. In order to verify the effectiveness of the IDQN method, the Myopic method, the improved Q-learning method, and the traditional DQN method are compared on Python. The simulation results shown that the proposed IDQN method not only outperforms the compared methods in terms of channel utilization and channel capacity but also improves the ability that the vehicle adapts to the dynamic communication environment.

RDRL: A Recurrent Deep Reinforcement Learning Scheme for Dynamic Spectrum Access in Reconfigurable Wireless Networks

Reconfigurable wireless network can flexibly provide efficient spectrum access service and keep stable operation in highly dynamic environment. In this paper, a primary-prioritized recurrent deep reinforcement learning algorithm for dynamic spectrum access based on cognitive radio (CR) technology is proposed. The spectrum Markov state is modeled to capture the evolution behavior to achieve the priority queuing of the primary users and the secondary users. According to the spectrum access strategies of the secondary users under different optimal criteria, we can obtain the best tradeoff benefits of spectrum access fairness and throughput. Furthermore, we proposed a learning-based algorithm for dynamic spectrum access, which allows the secondary users to modify their parameters to select the optimal access policy to maximize network throughput utilization. The Dueling Deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -Network (Dueling DQN) with prioritized experience replay combined with recurrent neural network is used to improve the convergence speed. Extensive experimental results demonstrate that the proposed RDRL scheme outperforms the existing Dueling DQN and DQN schemes in terms of convergence speed and channel throughput.

Performance Analysis of Efficient Spectrum Utilization in Cognitive Radio Networks by Dynamic Spectrum Access and Artificial Neuron Network Algorithms

Efficient spectrum utilization is a prominent issue in cognitive radio networks. Owing to this, power allocation policies are proposed which underlay cognitive radio networks together among all prime nodes, secondary nodes, eavesdropper and secondary sender powered by renewable energy that is harvested from primary sender to acquire improved energy efficiency to enhance transmission rate, throughput, and Spectrum Utilization (SU). As a result, there is a need for combination of Dynamic Spectrum Access (DSA) algorithm, Artificial Neuron Network (ANN) algorithm which will make an allotment of obtainable network assets for various elements challenging for their resources. The prime objective of this paper is to intend a route control based multi-path Quality Of Service (QoS) and to find substitute paths between Secondary User (SU) source and SU destination fulfilling QoS metrics, specifically providing maximal throughput and minimal delay. In order that primary substitute channels along the paths are used completely to reduce data packets loss by using Network Simulator 2 (NS2) software tool.

Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access

We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into $K$ orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive, in general, due to the large-state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to the spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. The experimental results demonstrate the strong performance of the algorithm.

Fast Convergence Cooperative Dynamic Spectrum Access for Cognitive Radio Networks

Cognitive radio and dynamic spectrum access can reform the way that radiofrequency spectrum is accessed. Problems of spectrum scarcity, coexistence, and unreliable wireless communication that affect industrial wireless networks can be addressed. In this paper, a game theoretic dynamic spectrum access algorithm that improves upon on a hedonic coalition formation algorithm for spectrum sensing and access is presented. The modified algorithm is tailored for faster convergence and scalability and makes use of a novel simultaneous multichannel sensing and access technique. Results to demonstrate the performance improvements of the adapted algorithm are presented and the use of different decision rules are investigated revealing that a conservative decision rule for exploiting spectrum opportunities performs better than an aggressive decision rule in most scenarios. The algorithm that was developed could be a key enabler for future cognitive radio networks.

Dynamic Spectrum Access Algorithms Based on Survival Analysis.

In this study, we design and implement two algorithms for dynamic spectrum access that are based on survival analysis. They use a non-parametric estimate of the cumulative hazard function to predict the remaining idle time available for secondary transmission subject to the constraint of a preset probability of successful completion. In addition to theoretical performance analysis of the algorithms, we evaluate them using data collected from a long term evolution band to model primary user activity to demonstrate their effectiveness in real-world scenarios, even at fine time scales. The algorithms are run in different configurations, i.e., they are trained and run on a few combinations of data sets. Our results show that as long as the cumulative hazard functions are fairly similar across datasets, the algorithms can be trained on one dataset and run on that of another without any significant degradation of performance. The algorithms achieve fairly high white space utilization and have a measured probability of interference that is at or below the preset threshold.

New dynamic spectrum access algorithm for TV white space cognitive radio networks

As more user applications emerge for wireless devices, the corresponding amount of traffic is rapidly expanding, with the corollary that ever-greater spectrum capacity is required. Service providers are experiencing deployment blockages due to insufficient bandwidth being available to accommodate such devices. TV White Space (TVWS) represents an opportunity to supplement existing licensed spectrum by exploiting unlicensed resources. TVWS spectrum has materialised from the unused TV channels in the switchover from analogue to digital platforms. The main obstacles to TVWS adoption are reliable detection of primary users (PUs) i.e. TV operators and consumers, allied with specifically, the hidden node problem. This study presents a new Generalised Enhanced Detection Algorithm (GEDA) that exploits the unique way Digital Terrestrial TV (DTT) channels are deployed in different geographical areas. GEDA effectively transforms an energy detector into a feature sensor to achieve significant improvements in detection probability of a DTT PU. Furthermore, by framing a novel margin strategy utilising a keep-out contour, the hidden node issue is resolved and a viable secondary user sensing solution formulated. Experimental results for a cognitive radio TVWS model have formalised both the bandwidth and throughput gains secured by TVWS users with this new paradigm.

Distributed Heuristically Accelerated Q-Learning for Robust Cognitive Spectrum Management in LTE Cellular Systems

In this paper, we propose an algorithm for dynamic spectrum access (DSA) in LTE cellular systems—distributed ICIC accelerated Q-learning (DIAQ). It combines distributed reinforcement learning (RL) and standardized inter-cell interference coordination (ICIC) signalling in the LTE downlink, using the framework of heuristically accelerated RL (HARL). Furthermore, we present a novel Bayesian network based approach to theoretical analysis of RL based DSA. It explains a predicted improvement in the convergence behaviour achieved by DIAQ, compared to classical RL. The scheme is also assessed using large scale simulations of a stadium temporary event network. Compared to a typical heuristic ICIC approach, DIAQ provides significantly better quality of service and supports considerably higher network throughput densities. In addition, DIAQ dramatically improves initial performance, speeds up convergence, and improves steady state performance of a state-of-the-art distributed Q-learning algorithm, confirming the theoretical predictions. Finally, our scheme is designed to comply with the current LTE standards. Therefore, it enables easy implementation of robust distributed machine intelligence for full self-organisation in existing commercial networks.

Single‐state Q‐learning for self‐organised radio resource management in dual‐hop 5G high capacity density networks

ABSTRACTIn this paper, a dual‐hop wireless backhaul and small cell access network has been exploited with effective spectrum sharing to provide 1 Gb/s/km2 ultra‐high capacity density for 5G ultra‐dense network deployments. We develop a single‐state Q‐learning‐based radio resource management algorithm for dynamic spectrum access creating a self‐organised network. It intelligently utilises the instantaneous spectrum observation information from spectrum sensing or a database to learn long‐term optimised decisions based on historical information of the system. The conventional Q‐learning algorithm with state‐action pairs has been simplified to a stateless format and applied in a fully distributed manner on individual data file transmissions, which reduces the complexity of the learning model and improves the applicability of Q‐learning algorithms to self‐organised wireless networks. The results show that not only does the proposed algorithm completely remove the requirement for frequency planning but it also improves the convergence, Quality of Service and system capacity substantially by achieving higher link capacity on both access and backhaul networks. Copyright © 2016 John Wiley &amp; Sons, Ltd.

Dynamic Spectrum Access Mechanism Based on Graphical Evolutionary Game in Radio Network

In order to realize efficient data transmission for multiple bounded rationality users sharing multiple channels in radio network, a dynamic spectrum access mechanism based on graphical evolutionary game is proposed to describe user's dynamic process for distributing transmission rate. The mechanism can reflect the real game relationship among users, thus simplifying the complexity of the game. Meanwhile, a dynamic spectrum access algorithm with smaller complexity and corresponding dynamic equation are designed for the mechanism, converging to Nash equilibrium with faster speed and obtaining higher system throughput. At Nash equilibrium, the reward of individual user is identical on each channel. Theory analyzes and proves that the dynamic equation is globally asymptotically stable, which illustrates that when user deviates because of bounded rationality, it is still able to converge again with faster speed and guarantee better performance with less deviation, and user's deviation only affects its neighboring users, not spreading to the whole network. Simulation comparison verifies the superiority above.

Dynamic Spectrum Access Algorithm Based on Game Theory in Cognitive Radio Networks

As an enabling technology for dynamic spectrum access (DSA), cognitive radio (CR) is widely regarded as one of the most promising technologies for future the fifth generation (5G) wireless communications. Although there have been significant prior researches to combat interference on primary users (PUs), the problem of mitigating mutual interference between secondary users (SUs), -which is tightly coupled with SU's spectrum leasing- is still not understood well. This paper proposes a DSA algorithm based on game theory, which jointly performs spectrum leasing and interference mitigation among SUs. The problem is modeled as an oligopolistic competition using Stackelberg model. We have carefully studied the SU's spectrum utilization behavior with respect to various criteria of the proposed game theoretic model. Simulation results shows that, Compared with Cournot game model, the proposed scheme enables SUs to efficiently utilize the licensed spectrum shared with PUs in a dynamic environment while maximizing the spectrum utilization.

Providing Vehicular Infotainment Service Using VHF/UHF TV Bands via Spatial Spectrum Reuse

In this paper, we identify the potential of the television bands to be used for providing vehicular infotainment service. Specifically, content delivery from the roadside unit to vehicular users is conducted by exploiting the spectrum holes, which is also referred to as spatial spectrum reuse. In this manner, the spectrum utilization efficiency is improved and the digital television service is not interfered. We first give a thorough description on such vehicular content delivery network and indicate the major design issues. After that, we respectively develop user and resource models tailored for our scenario by considering their characteristics and formulate the resource allocation as a system throughput maximization problem based on these models. For practical implementation, we devise a vehicular content delivery protocol incorporating a dynamic spectrum access algorithm and a maximal frame size design method, both of which aim at throughput and spectrum efficiency enhancement. Extensive simulations are conducted and the numerical results show the system performance is improved with our approaches.

An effective dynamic spectrum access algorithm for multi-hop cognitive wireless networks

Spectrum utilization, as a current hot research topic, has been paid extensive attention to in cognitive wireless networks. However, due to the complexities and dynamic properties in wireless networks, how to enhance effectively the spectrum utilization is still the main challenge to encounter at present. In this paper, we propose an effective dynamic spectrum access algorithm to improve the spectrum utilization and reliable communication in the multi-hop cognitive wireless network. Considering the inherent nature of licensed users, we adopt the Pareto distribution to model their behaviors. Accordingly, cognitive users can accurately sense the white spectrum. Different from the general dynamic spectrum access, we exploit the acknowledge information to provide the reliable transportation of data packets in multi-hop cognitive wireless networks. Moreover, to achieve the high spectrum utilization, we use the graph theory to perform the reasonable channel allocations. The time and frequency division multiplexing technologies are adopted to propose our practical strategies of channel allocations and switches. Consequently, we can solve the problems of channels sensing, channel allocations, channel accesses and channel switches. Simulation results indicate that the algorithm proposed can effectively reduce the overhead of channel sensing and switching, and improve the spectrum utilization.

Reconfigurable Antenna and Dynamic Spectrum Access Algorithm: Integration in a Cognitive Radio Platform for Reliable Communications

Wireless communications provide many advantages for industrial applications; however, some problems regarding the reliability of the link must be dealt (e.g. wideband interferers or multipath fading). Cognitive Radio and Dynamic Spectrum Access are two of the technologies that enable the implementation of reliable communications. In order to ensure the reliability of the communication link, it is necessary to carry out a complete change of the transmission band, and not only small in-band changes. Therefore it has to be ensured that the new chosen band is uncorrelated with respect to the previous one, in terms of coherence bandwidth and RF interference. When performing a change in the transmission band the update of some parameters in the digital domain of the cognitive-radio is not enough, but it also requires the adjustment of the analog part, where the antenna element plays an important role. On this basis, this paper presents the integration of a frequency reconfigurable antenna and a Dynamic Spectrum Access algorithm in an FPGA-based cognitive-radio platform for reliable communications. The reconfigurable antenna and the DSA algorithm, together with a commercial adjustable RF front-end, make the system capable of performing both in-band and dual-band channel change in the Industrial, Scientific and Medical (ISM) bands of 868 MHz and 2.45 GHz.

Spectrum Bonding and Aggregation with Guard-Band Awareness in Cognitive Radio Networks

Spectrum access/sharing algorithms for dynamic spectrum access (DSA) networks are often designed without accounting for adjacent-channel interference. In practice, guard bands are needed to prevent such interference. Introducing guard bands naturally constrains the effective use of the spectrum. In this work, we investigate the problem of assigning channels/powers to opportunistic transmissions, while accounting for such a constraint. Specifically, we propose a novel guard-band-aware channel assignment scheme for DSA systems. Our scheme reduces the number of required guard channels for a given transmission by exploiting the benefit of utilizing adjacent channels and considering already reserved guard channels. We analytically formulate the channel access problem as a joint power control and channel assignment optimization problem, with the objective of minimizing the required spectrum resource for a given CR transmission. We show that the optimization problem is a binary linear program (BLP), which is, in general, NP-hard. Accordingly, we present a near-optimal solution based on sequential fixing, where the binary variables are determined iteratively by solving a sequence of linear programs. Based on the proposed channel assignment algorithm, we develop an operational MAC protocol that enables DSA users to dynamically utilize the spectrum. The proposed protocol realizes our channel assignment algorithm in a distributed manner while relying only on information provided by the two communicating users. Simulation results are provided, which verify the effectiveness of our protocol and demonstrate the significant gain achieved through guard-band-aware channel assignment.

Distributed Cognitive Radio Spectrum Access with Imperfect Sensing Using CTMC

Cognitive radio (CR) has become an efficient approach for the utilization of scarce spectrum by enabling spectrum access in an opportunistic manner. With increasing wireless network service providers, users, and applications, it is indispensable to optimize usage of even parts of the spectrum that are licensed. Dynamic spectrum access (DSA) is an approach which facilitates the opportunistic use of licensed spectra, when they are idle. In this paper, a primary prioritized distributed DSA algorithm using continuous-time Markov chain (CTMC) model is proposed. In a distributed scheme, each secondary user needs to be aware of the statistics—arrival and service rates—of the other secondary users to optimize the throughput while maintaining fairness. A heuristics-based approach is formulated making use of the estimated spectrum idle probability and the interference each user experiences to iteratively update user statistics. This framework is also extended to incorporate the effects of imperfect spectrum sensing in the form of misdetection and false alarms. Simulation results show that the proposed algorithm attains an overall throughput that is better than CSMA when the primary user spectrum utilization is around 45%. The degradation in throughput caused by imperfect sensing over perfect sensing is also analyzed.

A novel dynamic spectrum access algorithm for cognitive radio networks

This paper proposes a new dynamic spectrum access (DSA) algorithm for cognitive radio networks. Once initialized, this algorithm works without the need of coordination overhead and hence can be used when no control channel is available. Secondary user (SU) lists and predetermined access control are used in this algorithm. We analyze the probability of no SU collision with primary user and the throughput of our proposed algorithm. Extensive simulations show that our algorithm outperforms the existing DSA algorithm in terms of both the aggregate throughput and the traffic distribution fairness. Furthermore, the validity of our analysis is confirmed by simulation results.

Price-Based Evolutionary Game Algorithm for Dynamic Spectrum Access

Price-Based Evolutionary Game Algorithm for Dynamic Spectrum Access

Optimal, heuristic and Q‐learning based DSA policies for cellular networks with coordinated access band

AbstractDue to the increasing demands for higher data rate applications, also due to the actual spectrum crowd situation, Dynamic Spectrum Access (DSA) turned into an active research topic. In this paper, we analyse DSA in cellular networks context, where a Coordinated Access Band (CAB) is shared between Radio Access Networks (RANs). We propose a Semi‐Markov Decision Process (SMDP) approach to derive the optimal DSA policies in terms of operator reward. In order to overcome the limitations induced by optimal policy implementation, we also propose two simple, though sub‐optimal, DSA algorithms: a Q‐learning (QL) based algorithm and a heuristic algorithm. The achieved reward using the latter is shown to be very close to the optimal case and thus to significantly exceed the reward obtained with Fixed Spectrum Access (FSA). The rewards achieved by using the QL‐based algorithm are shown to exceed those obtained by using FSA. Higher rewards and better spectrum utilisation with DSA optimal and heuristic methods are, however, obtained at the price of a reduced average user throughput. Copyright © 2010 John Wiley &amp; Sons, Ltd.