Dynamic Spectrum Access Networks Research Articles

Federated Multi-Agent Deep Reinforcement Learning (Fed-MADRL) for Dynamic Spectrum Access

Dynamic spectrum access (DSA) has been introduced as a promising technology that allows a secondary system to access the licensed spectrum of the primary system to improve spectrum utilization. In this paper, we introduce Fed-MADRL by incorporating federated learning (FL) and multi-agent deep reinforcement learning (MADRL) to design a collaborative DSA strategy. Our Fed-MADRL scheme employs FL to enable multiple users to collaboratively optimize the system goal without sharing their training data. By keeping all the training data at the user end, FL improves the communication efficiency and strengthens user data privacy. To further reduce the communication overheads, each user only shares quantized information. We provide the convergence analysis to characterize the trade-off between the communication efficiency and the system performance. In particular, we show that the introduced method converges at a rate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1/ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/4</sup> ), where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> is the number of FL iterations. To the best of our knowledge, Fed-MADRL is the first work that utilizes FL in DSA networks under quantized communication. Performance evaluation results show that the introduced Fed-MADRL method outperforms the independent learning method and achieves comparable performance with the centralized MADRL method, which requires much higher communication overheads.

White spaces pattern finding and inference based on machine learning for multi-frequency spectrum footprints

Spectrum surveys performed worldwide demonstrate that the spectrum utilization efficiency is less than 0.25. Therefore, the traditional long-term licensed spectrum allocation by regulators is not sustainable. Although dynamic spectrum access networks allow increasing the efficiency of spectrum utilization, coexistence is still a major problem. In this paper, we investigate the capability of machine learning for estimating the white space availability based on a dataset from a spectrum survey from 170 MHz to 1 GHz. In addition, we present an algorithm for minimizing the effect of hidden nodes on wrongful spectrum allocation and interference. Our optimization algorithm based on supervised machine learning allows increasing the spectrum utilization efficiency with a factor 5 (from 0.09 to 0.47) in the surveyed region. In addition, our algorithm allows decreasing the interference probability caused by the effect of hidden nodes by a factor 6, compared to the traditional distributed allocation of spectrum in Dynamic Spectrum Access networks.

Wireless Network Design Optimization for Computer Teaching with Deep Reinforcement Learning Application

ABSTRACT Computer technology has had a significant impact on the field of education, and its use in classrooms has facilitated the spread of knowledge and has helped students become well-rounded citizens. However, as the number of users accessing wireless networks for computer-based learning continues to grow, the scarcity of spectrum resources has become more apparent, which makes it crucial to find intelligent methods to improve spectrum utilization in wireless networks. Dynamic spectrum access is a critical technology in wireless networks, and it primarily focuses on how users can efficiently access licensed spectrum in a dynamic environment. This technology is a crucial means to tackle the problem of spectrum scarcity and low spectrum utilization. This work proposes a novel approach to address the issue of dynamic channel access optimization in wireless networks and investigates the dynamic resource optimization problem with deep reinforcement learning algorithms. The proposed approach focuses on the optimization of dynamic multi-channel access under multi-user scenarios and considers the collision and interference caused by multiple users accessing a channel simultaneously. Each user selects a channel to access and transmit data, and the network aims to develop a multi-user strategy that maximizes network benefits without requiring online coordination or information exchange between users. What makes this work unique is its utilization of deep reinforcement learning algorithms and a Long Short-Term Memory (LSTM) network that keeps an internal state and combines observations over events. This approach allows the network to utilize the history of processes to estimate the true state, providing valuable insights into how deep reinforcement learning algorithms can be used to optimize dynamic channel access in wireless networks. The work’s contribution lies in demonstrating that this approach is an effective means to solve the dynamic resource optimization problem, enabling the development of a multi-user strategy that maximizes network benefits. This approach is particularly valuable as it does not require online coordination or information exchange between users, which can be challenging in real-world scenarios. The proposed approach presents an important contribution to the field of dynamic spectrum access and wireless network optimization. As the demand for computer-based learning continues to increase, the use of intelligent methods to improve spectrum utilization in wireless networks will become even more critical. The findings of this work could have significant implications for the future of computer-based learning and education, enabling more efficient use of wireless networks and the creation of well-rounded citizens.

Minimum cost spectrum allocation with QoS guarantees in multi-interface multi-hop dynamic spectrum access networks

Spectrum Sensing as a Service (SSaS) is emerging as the enabling business model for efficient spectrum sharing in many recent applications. In such model, sensing infrastructure provides dynamic spectrum access networks, also known as cognitive radio networks, with information about availability or unavailability of given spectrum bands. In return for such information, the service provider imposes money costs on the users of the dynamic spectrum access network or its operator. In this paper, we address the problem of spectrum allocation along a route in multi-interface dynamic spectrum access networks under the SSaS model. The objective is to allocate spectrum channels to interfaces of nodes along the route so that the total sensing cost imposed by SSaS provider is minimized and the quality of links meet a predetermined QoS requirement, specifically a data rate requirement. The problem is formulated as an Integer Linear Program (ILP) to obtain the optimal solution. Given the intractable complexity of ILPs, a sub-optimal algorithm is proposed. The accuracy of the proposed algorithm is validated via extensive experimentation.

Dynamic spectrum access based on deep reinforcement learning for multiple access in cognitive radio

With the increasing shortage of spectrum resources, dynamic spectrum access (DSA) technology is proposed to maximize the spectrum resources utilization. Traditional DSA solutions can no longer meet the requirements of high throughput and low interference in large-scale access scenarios of cognitive radio (CR). Therefore, in this paper, we propose a DSA scheme based on deep reinforcement learning (DRL) combined with multiple access methods to maximize the system throughput. In the DSA network, the access strategy adopted by the secondary user (SU) will directly affect the performance of the entire system, so we introduce DRL to help the SU learn the best access strategy in a dynamic environment. The trained SU can intelligently access the appropriate channel to avoid interference to the primary user (PU) and other SUs. By combining deep Q-network (DQN) into two multiple access methods: Frequency Division Multiple Access (FDMA) and Non-orthogonal Multiple Access (NOMA), DQN-based FDMA scheme and DQN-based NOMA scheme are designed, respectively, which can find the best DSA strategy to avoid collisions with PU or other SU and improve system throughput. Simulation results show that the DQN-based NOMA scheme has better performance than the DQN-based FDMA scheme.

Multi-Agent Reinforcement Learning-Based Distributed Dynamic Spectrum Access

Dynamic spectrum access (DSA) is an effective solution for efficiently utilizing the radio spectrum by sharing it among various networks. Two primary tasks of a DSA controller are: 1) maximizing the quality of service of users in the licensee’s network and 2) avoiding interference in communications towards the incumbent network. These two tasks become quite challenging in a distributed DSA network due to the lack of a centralized controller to regulate the sharing of the radio spectrum between incumbents and licensees. Hence, optimization-driven techniques to design power allocation schemes in such a network often become intractable. Accordingly, in this paper, we present a distributed DSA based communication framework based on multi-agent reinforcement learning (RL), where the multiple cells in the multi-user multiple-input multiple-output (MU-MIMO) licensee network act as agents, and the average signal-to-noise ratio value is the reward. In particular, by considering the physical layer parameters of the DSA network, we analyze various RL algorithms, namely Q-learning, deep Q-network (DQN), deep deterministic policy gradient (DDPG), and twin delayed deep deterministic (TD3), whereby the licensee network learns to obtain the optimal power allocation policies for accessing the spectrum in a distributed fashion without the need for a central DSA controller to manage the interference towards the incumbent. Trade-offs are identified for the considered algorithms with respect to performance, time complexity and scalability of the DSA network.

Performance Analysis of Efficient Spectrum Utilization in Cognitive Radio Networks by Dynamic Spectrum Access and Artificial Neuron Network Algorithms

Efficient spectrum utilization is a prominent issue in cognitive radio networks. Owing to this, power allocation policies are proposed which underlay cognitive radio networks together among all prime nodes, secondary nodes, eavesdropper and secondary sender powered by renewable energy that is harvested from primary sender to acquire improved energy efficiency to enhance transmission rate, throughput, and Spectrum Utilization (SU). As a result, there is a need for combination of Dynamic Spectrum Access (DSA) algorithm, Artificial Neuron Network (ANN) algorithm which will make an allotment of obtainable network assets for various elements challenging for their resources. The prime objective of this paper is to intend a route control based multi-path Quality Of Service (QoS) and to find substitute paths between Secondary User (SU) source and SU destination fulfilling QoS metrics, specifically providing maximal throughput and minimal delay. In order that primary substitute channels along the paths are used completely to reduce data packets loss by using Network Simulator 2 (NS2) software tool.

A Stochastic Spectrum Trading and Resource Allocation Framework for Opportunistic Dynamic Spectrum Access Networks

In this article, the spectrum trading problem between primary users and secondary networks is investigated. The secondary network requests multiple channels with the targeted availability to satisfy its users&#x2019; demands. Due to the uncertainty about the channels availability, stochastic optimization techniques are adopted to find the optimal set of channels for each secondary network for the lowest cost. Two different constraints on the secondary demand are defined. The first one is when the throughput has to be fully satisfied for a certain percentage of time, and the second one is when the expected value of the throughput has to exceed a certain percentage of the requested one. Also, the possibility for channel subleasing among the secondary networks is investigated to reduce the demand shortage. The results show that demanding simultaneous channels increases the cost as it reaches up to 20% higher than if the same resources were requested individually. Also, channels subleasing reduces the demand shortage probability and increases the achieved throughput, especially at low value of requested demand. In this case, the demand satisfaction probability increases by around 30% while the achieved throughput increases up to 40% compared to the scenario where channels subleasing is not allowed.

A Deep Reinforcement Learning Framework for Spectrum Management in Dynamic Spectrum Access

Dynamic spectrum access (DSA) has the great potential to alleviate spectrum shortage and promote network capacity. However, two fundamental technical issues have to be addressed, namely, interference coordination between DSA users and interference suppression for primary users (PUs). These two issues are very challenging since generally there is no powerful infrastructures in DSA networks to support centralized control. As a result, DSA users have to perform spectrum management individually, including spectrum access and power allocation, without accurate channel state information and centralized control. In this article, a novel spectrum management framework is proposed, in which Q-learning, a type of reinforcement learning, is utilized to enable DSA users to carry out effective spectrum management individually and intelligently. For more efficient process, neural networks (NNs) are employed to implement Q-learning processes, so-called deep Q-network (DQN). Furthermore, we also investigate the optimal way to construct DQN considering both the performance of wireless communications and the difficulty of NN training. Finally, extensive simulation studies are conducted to demonstrate the effectiveness of the proposed spectrum management framework.

Adaptive streaming of HD and 360[formula omitted] videos over software defined radios

In this paper we study and implement real-time adaptation schemes for video encoding and channel selection that work in tandem to facilitate HD and 360° video streaming for secondary users in a dynamic spectrum access network. Out-of-band feedbacks on instantaneous pathloss of the signal between the transmitter and the receiver, the received signal strength indicator (RSSI) at the receiver, and the quality of the reconstructed video, are used to continuously determine the best possible encoding parameters. Similarly, the radio transmitter continuously adjusts the channel parameters (i.e., center frequency and channel bandwidth) based on the transmission activities of the primary users who are prioritized on these channels. We consider the physical limitations of the encoder and the channel statistics to determine when to change the encoder parameters and when to switch to a new channel. We propose a multi-level threshold based mechanism to find the optimal number of encoding bit rates. We also propose a threshold based algorithm to find the best available channel between the transmitter–receiver pair. We validate our theoretical propositions on an indoor testbed using universal software radio peripherals (USRPs) and GNU Radio. Live videos captured with a web-cam and a 360° VR camera were encoded with open source H.264 software libraries and streamed using GStreamer. The signal was transmitted over the 915 MHz ISM bands with omni-directional antennas. We used two B210s (from Ettus Research) for the transmitter and the receiver. Another B210 was programmed to sense the energy levels on all the channels in order to detect the presence of primary user transmissions. GNU Radio was used to build the initial flow-graph of all the signal processing modules, both at the transmitter and the receiver. We used Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) to measure the video quality. Experimental results show that (i) the video encoder and the USRP transmitter–receiver pair were able to adapt to the changing RF conditions, (ii) the adaptation schemes yielded better video quality than non-adaptive schemes, and (iii) the USRPs could switch the channels fast enough allowing uninterrupted video streaming even when the primary users preempted the secondary user’s transmission. We also present a simulation study to show that the proposed encoder and channel adaptation algorithms can be scaled to larger networks.

Connectivity optimization in interference-limited random DSA networks

Network connectivity in dynamic spectrum access (DSA) networks has been well studied–most of which are under the unit disk model. However, the disk model does not capture the primary-secondary and secondary-secondary interference; hence signal to interference and noise ratio (SINR) based models are more appropriate. Moreover, in the SINR regime, there is no unique way to characterize connectivity and hence its maximization becomes even more challenging.In this paper, we develop the long eluding network connectivity objective function which we use to build three connectivity optimization techniques each of which targets a particular network setup. The proposed techniques are: i) Fittest deployment density, ii) Fittest receive-only ratio, and iii) Fittest TDMA slotting.To develop the aforementioned objective function, we start by addressing the lack of any relation between deployment density and network connectivity in interference-limited DSA networks. Next, percolation theory in conjunction with the Boolean model are utilized to develop such a relationship between the density of the percolation visible nodes and the network connectivity in interference limited environments. Finally, we use that relation to build the objective function for connectivity maximization along three optimization techniques. Theoretical findings are validated by simulating networks under various scenarios. Results provide a blueprint to establish and maximize connectivity using physical layer parameters (density, coverage radius, etc.) which can be used in conjunction with higher layer techniques. Also, tackling the connectivity problem at the physical layer relieves the other higher layers like the MAC layer from excess signaling and complex protocol designs.

Efficient on-demand spectrum sensing in sensor-aided cognitive radio networks

Spectrum Sensing as a Service (SSaS) has recently emerged as a promising solution to realize dynamic spectrum access networks (aka, cognitive radio networks (CRNs)). This paradigm relies on a dedicated infrastructure wireless sensor network, operated by a spectrum sensing service provider S3P entity, for spectrum monitoring. Wireless users which are interested in knowing the status of a given spectrum band make a request to the S3P to perform that monitoring on its behalf. In this work, we study the problem of optimizing the selection of sensors needed to serve a given request such that the total cost (consumed energy) is minimized. We consider a spatial and temporal scheduling problem, in which, the energy levels at sensors, the maximum transmission range of secondary user (SUs), and the required monitoring time are taken into account. The problem is formulated as an Integer Linear Program (ILP). Then a sub-optimal greedy algorithm is proposed with two variations. Thorough evaluation shows that the proposed algorithm performs very well with respect to the optimal solution. Finally, results illustrate that the first algorithm variation is better in terms of energy cost, and the second variation allows for serving requests with a lower number of nodes.

Cooperative Spectrum Sensing Using Maximum a Posteriori as a Detection Technique for Dynamic Spectrum Access Networks

Over the past few years, dynamic spectrum access has been gaining an increasing attention as a solution to the spectrum scarcity problem. In this paper, a primary user detection technique based on Maximum A Posteriori estimation is proposed for dynamic spectrum access networks. In the proposed technique, a set of secondary users acting as sensing nodes send their individual decisions about the existence of the primary user to a central fusion center. The fusion center uses the received data to form a codeword then, applies the maximum a posteriori estimation rule to make a final decision regarding the presence of the primary user. The proposed technique takes into consideration the accuracy of the local decisions provided by the secondary users when making a final decision. In this paper, we analyze the performance of the proposed scheme and derive closed-form expressions for the upper bounds of the false alarm and misdetection probabilities. The results show that the proposed technique outperforms other combining techniques in terms of its ability to detect the primary user and, accordingly, minimizes the harmful interference to the licensed network. Moreover, the proposed technique achieves better performance at a lower number of reporting secondary users which compensates for the complexity of the maximum a posteriori estimation.

Demand-Based Spectrum Trading for Opportunistic Secondary Networks With Probabilistic Channel Access Mechanism

In this paper, we consider the spectrum trading problem in dynamic spectrum access networks. The trading process takes place between one opportunistic secondary network, which tries to satisfy its Secondary Users (SUs) throughput demand, and multiple Primary Users (PUs). The secondary network coordinator distributes the SUs over the PUs’ channels and assigns a certain access time for every SU to satisfy its demand for the lowest cost paid to the PUs. We propose a secondary probabilistic access mechanism to control the SUs access to the free spectrum. In this mechanism, the coordinator controls the channel access probability of each SU and thus the amount of time shares it receives. To calculate the required access probabilities, we propose a novel probabilistic channel model that captures the interaction between the SUs and the PU using discrete-time Markov chain. We model the spectrum trading under the proposed access mechanisms as optimization problems. The results show the superiority of using variable access probability channel access mechanism over the equal access mechanism in terms of cost and throughput.

Distributive Dynamic Spectrum Access Through Deep Reinforcement Learning: A Reservoir Computing-Based Approach

Dynamic spectrum access (DSA) is regarded as an effective and efficient technology to share radio spectrum among different networks. As a secondary user (SU), a DSA device will face two critical problems: avoiding causing harmful interference to primary users (PUs), and conducting effective interference coordination with other secondary users. These two problems become even more challenging for a distributed DSA network where there is no centralized controllers for SUs. In this paper, we investigate communication strategies of a distributive DSA network under the presence of spectrum sensing errors. To be specific, we apply the powerful machine learning tool, deep reinforcement learning (DRL), for SUs to learn "appropriate" spectrum access strategies in a distributed fashion assuming NO knowledge of the underlying system statistics. Furthermore, a special type of recurrent neural network (RNN), called the reservoir computing (RC), is utilized to realize DRL by taking advantage of the underlying temporal correlation of the DSA network. Using the introduced machine learning-based strategy, SUs could make spectrum access decisions distributedly relying only on their own current and past spectrum sensing outcomes. Through extensive experiments, our results suggest that the RC-based spectrum access strategy can help the SU to significantly reduce the chances of collision with PUs and other SUs. We also show that our scheme outperforms the myopic method which assumes the knowledge of system statistics, and converges faster than the Q-learning method when the number of channels is large.

A multi-states continuous time Markov chain model for secondary spectrum access in dynamic spectrum access networks

Dynamic Spectrum Access (DSA) networks are vulnerable to hackers who normally pretend themselves to be the primary users and called the Primary User Emulation Attack (PUEA). Research communities have already reported a vast use of PUEA in the existing research. Other potential attackers such as greedy users should not be ignored when investigating the dynamic spectrum access networks. In this paper, we propose a multi-states Continuous Time Markov Chain (CTMC) model to describe the behaviour of DSA, analysis of the channel states and discussion on the impacts of normal, normal greedy and greedy malicious users in DSA network. The CTMC model is simulated and the simulation results have been discussed and validated by comparing with the existing models. Finally, it is proved that CTMC model is an improved method to analyse the performance of the DSA networks when PUEA occurs.

A multi-states continuous time Markov chain model for secondary spectrum access in dynamic spectrum access networks

Dynamic Spectrum Access (DSA) networks are vulnerable to hackers who normally pretend themselves to be the primary users and called the Primary User Emulation Attack (PUEA). Research communities have already reported a vast use of PUEA in the existing research. Other potential attackers such as greedy users should not be ignored when investigating the dynamic spectrum access networks. In this paper, we propose a multi-states Continuous Time Markov Chain (CTMC) model to describe the behaviour of DSA, analysis of the channel states and discussion on the impacts of normal, normal greedy and greedy malicious users in DSA network. The CTMC model is simulated and the simulation results have been discussed and validated by comparing with the existing models. Finally, it is proved that CTMC model is an improved method to analyse the performance of the DSA networks when PUEA occurs.

Database-Assisted Spectrum Prediction in 5G Networks and Beyond: A Review and Future Challenges

This article surveys the state of the art in spectrum prediction and learning, summarizes applications, techniques, main metrics, computational complexity, and provides practical examples. We focus on a cellular case study and define required improvements to database-assisted spectrum sharing. The use of history information and predictive spectrum modeling at different time scales provides valuable information to regulators, operators, and users of dynamic spectrum access networks. Prediction enables dynamic spectrum sharing systems to operate proactively, and consequently improves the performance in terms of reducing delays and interference among coexisting systems. Current database-assisted spectrum sharing concepts are in fact too static for many applications. Our numerical results on local-aware predictive spectrum allocation show the advantage of predictive operation in a vehicle-to-everything (V2X) scenario.

IEEE 1900.5.2: Standard Method for Modeling Spectrum Consumption: Introduction and Use Cases

Dynamic spectrum access (DSA) is a critical technology that allows diverse systems to rapidly adjust spectrum assignments and share spectrum with each other. This greatly improves overall spectrum efficiency at a time when demands for spectrum are rapidly increasing. Wireless services that use DSA or spectrum sharing mechanisms require a well-defined means to communicate how each device or system is using spectrum resources in a particular situation or at a specific location. Spectrum consumption models (SCMs) address this requirement by providing a means to capture the spectral, spatial, and temporal characteristics of spectrum usage for any specific wireless transmitter, receiver, system, or collection of systems. The 1900.5 Working Group of the IEEE Dynamic Spectrum Access Networks Standardization Committee (DySPAN-SC) has written the IEEE 1900.5.2 standard, which defines a data model for SCMs and procedures to arbitrate compatibility among combinations of RF devices and/or spectrum-dependent systems that have expressed the boundaries of their spectrum use with SCMs. This article provides an overview of the IEEE 1900.5.2 standard using example use cases to illustrate how SCMs can enable a wide variety of efficient spectrum use interactions. Additionally, the article provides a brief description of an open source software tool to facilitate and promote the use of SCMs.

PreDA: Preference-based double auction for spectrum allocation in heterogeneous DSA networks

In this paper, we propose PreDA– a preference-based truthful double auction for dynamic spectrum access (DSA) networks where multiple heterogeneous spectrum bands are sold by the primary users and bought by the secondary users. Unlike existing double auctions, we not only consider channels’ heterogeneity and multi-bids from buyers, but also consider buyers’ preferences for the channels. We use the signal to interference and noise ratio (SINR) as a metric for the preference; channels with higher SINR are preferred and hence bids are more compared to the bids for channels that offer lower SINRs. In order to maximize the spatial and temporal reuse of channels, we use the concept of virtual grouping of conflict-free buyers. Virtual groups allow us to transform multi-unit bids to single-unit bids. We propose a novel winner determination and pricing mechanism to allocate the unused spectrum bands to the most appropriate buyers. We prove that the proposed double auction achieves the three economic properties of truthfulness, individual rationality, and ex-post budget balance. We validate PreDA through simulation experiments and show its performance in terms of the number of allocated bands, utilization, revenue, and fairness.