Spectrum Access Research Articles

Intelligent Access to Unlicensed Spectrum: A Mean Field Based Deep Reinforcement Learning Approach

As the demand for mobile data traffic continues to grow, offloading data traffic to unlicensed spectrum is a promising approach that can relieve the pressure on cellular systems. Therefore, it is an urgent need to propose an unlicensed spectrum access method to guarantee the harmonious and efficient coexistence between cellular network technologies such as LTE and incumbent users such as WiFi in the unlicensed spectrum. However, existing coexistence schemes such as licensed assisted access (LAA) and LTE-unlicensed (LTE-U) still suffer from inefficient spectrum utilization and unsatisfactory fairness. In the paper, we formulate the optimization problem of the unlicensed spectrum access among multiple small bases (SBSs) as a game, and then solve the Nash Equilibrium (NE) with cooperative and distributed multi-agent deep reinforcement learning (MADRL). Specifically, a two level access framework for the coexistence scenario, which consists of feedback cycle and executive cycle, is first proposed, and then the key elements of MADRL including state, action, reward and Q-network are designed in detail based on the proposed access framework. To overcome the problems of learning divergence and prohibitive computation overhead in the coexistence scenario with multiple SBSs due to the non-stability phenomena, we adopt the mean field technology to solve the NE, which can simplify the process of solving NE by converting the interaction of an agent with the remaining multiple agents into an action with the average effect of them. Simulation results show that 1) the proposed algorithm can overcome the learning divergence problem and converge to the NE quickly, and 2) the proposed algorithm can achieve the bi-objective optimization of total throughput and fairness of the coexistence network, and can achieve better performance in terms of throughput and fairness compared with the baseline methods such as Cat-4 LBT, Cooperative LBT and Random schemes.

Deep Recurrent Reinforcement Learning-Based Distributed Dynamic Spectrum Access in Multichannel Wireless Networks With Imperfect Feedback

This paper investigates Dynamic Spectrum Access (DSA) paradigm with imperfect feedback for multiuser wireless network. Each user selects an orthogonal channel in particular time slot to transmit packet with a certain transmission probability. In next time slot, the user who has transmitted a packet receives an ACK signal based on local observation. Bearing in mind the dynamic nature of wireless networks, it is appealing to develop a blended strategy to perform effective DSA. This paper aims to design a distributed Deep Reinforcement Learning (DRL) based scheme with an objective of maximizing network utility. In conventional DRL framework, it is assumed that the feedback received (ACK packet) is always correct but in wireless networks, it may be lost or corrupted due to noise. Furthermore, it is challenging to promise that the multiple agents will cooperate to make coherent decisions in order to accomplish the same objective, particularly under imperfect feedback. To tackle these challenges, this work proposes (i) Deep Recurrent Reinforcement Learning network with integrated GRU layer to optimize the network utility function and (ii) a feedback recovery mechanism using complete and incomplete replay buffers. Extensive simulations corroborate the success of proposed scheme in complex multiuser scenario and exhibit robustness against the detrimental effects of the imperfect feedback.

Enhancing Hybrid Spectrum Access in CR-IoT Networks: Reducing Sensing Time in Low SNR Environments

The current utilization of the licensed spectrum band is not optimal. the abundance of Internet of Things (IoT) gadgets could lead to congestion in the unlicensed spectrum band. A potential solution is to integrate cognitive radios into IoT devices, specifically by developing CR-IoT (cognitive-radio-enabled IoT) devices that leverage the hybrid spectrum access (HSA) technique to access the licensed spectrum band and employ energy detectors for spectrum sensing. While HSA can enable high data throughput for CR-IoT networks, environments with low signal-to-noise ratio (SNR) may experience reduced performance. Particularly, in low SNR environments with SNR values ranging from -20dB to -24dB, the high level of noise uncertainty can cause the energy detector to spend more time sensing, which results in a significant reduction in transmission time and overall throughput. In this study, a novel approach is presented to address the challenge of noise uncertainties on the energy detector by implementing an adaptive sensing threshold. The simulation outcomes reveal that the proposed technique can significantly improve the throughput in scenarios characterized by low signal-to-noise ratio (SNR), with a maximum enhancement of up to 28%. This innovative approach can pave the way for more robust and efficient energy detectors in the presence of noise uncertainties .

Deep learning-based selective spectrum sensing and allocation in cognitive vehicular radio networks

The main challenge with Vehicular Ad-Hoc Networks (VANETs) for assisting Intelligent Transportation Services (ITSs) is ensuring effective data delivery under various network circumstances despite the scarcity of radio frequency spectrum channels. Meanwhile, Dynamic Spectrum Access (DSA) utilizing Cognitive Radio (CR) technology shows its potential to solve the channel shortage issue. Reliable Spectrum Sensing (SS) and Opportunistic Spectrum Allocation (OSA) are the two most important and challenging aspects of deciding the success of the deployment of CR-enabled VANETs (CR-VANETs). In this paper, we propose three relevant issues of CR-VANETs in a single framework, i.e., reliable Cooperative Spectrum Sensing (CSS), channel indexing for selective SS, and best channel allocation to the CR users. In CSS, we use the local SS decision with more critical attributes like the geographical position of sensing signal acquisition and timestamp to obtain the global CSS session using the Deep Reinforcement Learning (DRL) technique. Selective channel-based spectrum sensing is essential to minimize the sensing overload by CR users. The present work uses the time series analysis through the deep learning-based Long short-term memory (LSTM) model for indexing the primary user channels for selective SS. Finally, for the channel allocation to the CR-VANETs, we model the complex environment as a Partial Observable Markov Decision Process (POMDP) framework and solve using a value iteration-based algorithm. The simulation results show the proposed work's efficacy over the existing works.

Distributed dynamic spectrum access through multi-agent deep recurrent Q-learning in cognitive radio network

This paper addresses the problem of distributed dynamic spectrum access in a cognitive radio (CR) environment utilizing deep recurrent reinforcement learning. Specifically, the network consists of multiple primary users (PU) transmitting intermittently in their respective channels, while the secondary users (SU) attempt to access the channels when PUs are not transmitting. The problem is challenging considering the decentralized nature of CR network where each SU attempts to access a vacant channel, without coordination with other SUs, which result in collision and throughput loss. To address this issue, a multi-agent environment is considered where each of the SUs perform independent reinforcement learning to learn the appropriate policy to transmit opportunistically so as to minimize collisions with other users. In this article, we propose two long short-term memory (LSTM) based deep recurrent Q-network (DRQN) architectures for exploiting the temporal correlation in the transmissions by various nodes in the network. Furthermore, we investigate the effect of the architecture on success rate with varying number of users in the network and partial channel observations. Simulation results are compared with other existing reinforcement learning based techniques to establish the superiority of the proposed method.

A Review of Two Types of New Key Technologies of 6G

The increasing need of future 6G wireless communication of higher transmission rate, larger network capacity, higher energy efficiency has been calling for the development of new key technologies. Among them, the two most popular and promising technologies are new spectrum access technology and efficient wireless transmission technology. New spectrum access technology intends to achieve the utilization of more available spectrum resources with higher frequency and wider bandwidth so that it can highly improve the data transfer rate and expand the network capacity. It includes Terahertz communication technology and visible light communication technology. Efficient Wireless Transmission technology aims to make new breakthroughs in traditional signal transmission mode and achieve larger channel capacity so that network coverage and security can be improved, and signal transmission loss can be decreased. It contains IRS technology and Spatial multiplexing technology. Properties of each technology, such as principles, advantages, application scenarios, are discussed in this paper. In chapter 2, new spectrum access technology is introduced, and efficient wireless transmission technology is introduced in chapter 3.

Automatic modulation recognition based on a multiscale network with statistical features

Automatic modulation recognition (AMR) can be used in dynamic spectrum access (DSA) techniques to reduce the pressure on spectrum resources. In this paper, we propose a multiscale convolution-based network model called MSNet-SF, which combines traditional statistical features and deep learning (DL) to balance recognition accuracy and complexity. In the model, the feature information is extracted by two multiscale modules, which consist of unit convolution and three different sizes of convolution kernels arranged in parallel. Additionally, the sparse connectivity of unit convolution enables the network to be more lightweight. Five statistical features (four higher-order cumulants (HOCs) and one zero-centered normalized instantaneous magnitude tightness) are also input into the model and are fully fused with the main feature map by multiplication to achieve complementarity of long-term and short-term features. This approach yields a large performance gain at a small cost and greatly reduces the confusion between QAM16 and QAM64. Simulation results in the RML2018.10A dataset show that the average recognition accuracy of the model improved by 4% after adding the statistical features and achieved an accuracy of more than 97% from 12 dB.

Energy-Efficient Opportunistic Spectrum Access in Multichannel Cognitive Radio Networks

We study an opportunistic spectrum access problem in multichannel Cognitive Radio Networks in which the secondary user (SU) has an energy cost when switching across channels. Channel switching from one RF chain to another also involves a reconfiguration delay that results in a shorter duration of transmission opportunity for the SU. For an imperfect sensing, we formulate the spectrum access problem in the framework of Partially Observable Markov Decision Process from which we obtain an average energy optimal policy. Also, we propose Efficacy - Efficient channel access with switching policy, for channel sensing and access based on the structure of the optimal policy. We compare the optimum policy with Efficacy, and show that Efficacy has a near optimal average energy performance.

Time-Variant Spectrum Mapping via Reduced Basis Representation and Greedy Sampling Locations Optimization

Spectrum mapping has been increasingly important in various wireless communication applications, such as spectrum management, network planning, power control, dynamic spectrum access etc. In this letter, we consider the problem of time-varying spectrum situation map reconstruction using a small amount of sampled data. In contrast to traditional assumption that the statistical signal propagation model is known, we infer spatial propagation characteristics from the spectrum data. Then, a greedy optimization is applied to calculate the sampling locations with the reduced spatial spectrum situation basis representation, which also converts the original underdetermined problem of spectrum situation recovery into a least squares problem with spatial smoothness constraint, thus greatly reducing the matrix operations. Simulations confirm the superiority of our proposed algorithm in reconstruction performance and computational efficiency in online spectrum mapping tasks.

Device-to-device Communications at the Terahertz Band: Open Challenges for Realistic Implementation

One of the key parameters that plays a major role in enabling the data rate requirements is spectrum or bandwidth, which is scarce and expensive. Therefore, spectrum management policies are required to optimize its usage to meet all the requirements and the promised data rate growth. Another strategy to deal with spectrum scarcity is to move toward higher frequency bands (terahertz bands), which are expected in the next 6G communication standard. It is therefore important to develop not only new techniques that enable efficient dynamic spectrum access and sharing at such bands, but also suitable channel models for the terahertz bands. Meanwhile, offloading mechanisms are very promising for cellular networks where a plethora of options have been proposed in the research arena in terms of device-to-device, licensed assisted access, or WiFi offloading, among others; but their behavior, when operated at high frequencies (terahertz band) remains unclear. Therefore, this article will tackle two technologies that will shape future networks: terahertz channel modeling/communications and offloading mechanisms.

Learning Based Multi-Channel Access in Vehicular Networks With Illegal Operators and Users

This paper investigates the multi-channel access (MCA) problem for vehicular networks in presence of illegal operators and users. Due to the limited spectrum resource, vehicle-to-vehicle (V2V) links need to reuse the frequency spectrum allocated for vehicle-to-infrastructure (V2I) links in networks. For high-mobility environments of vehicular networks, the fast-varying channel conditions will inevitably lead to the significant uncertainty of the acquired channel state information (CSI). Hence, the traditional centralized MCA methods, which rely on the CSI, could not be performed in a timely manner. We formulate the MCA of the vehicle users as a distributed optimization problem, by maximizing the sum capacity of V2I links while guaranteeing the reliability of V2V links as well as reducing the cost due to frequency hopping. A Multi-Agent Deep Deterministic Policy Gradient (MADDPG) framework is introduced to tackle this problem, where each vehicle user, connecting with either a V2I or V2V link, acts as a learning agent to make spectrum access decisions rapidly and locally to meet the latency requirement. In order to adapt to the dynamic environment, we further propose a delayed interaction aided MADDPG algorithm, which enables online training for the distributed MCA problem. Moreover, to guarantee the proposed algorithm to converge stably, we optimize the selecting weights of experience traces and propose a Batch Prioritized Experience Replay (BPER) strategy to ensure that the high priority traces can be learned timely. Compared with baselines, simulation results showcase remarkable performance gain of our proposed algorithm.

Blockchain based smart contract for cooperative spectrum sensing in cognitive radio networks for sustainable beyond 5G wireless communication

Cognitive radio networks allow opportunistic spectrum access to the unlicensed users, which rely on effective spectrum sensing. Recently, cooperative sensing has been found to improve sensing reliability. However, sometimes the selfish users do not cooperate, and therefore, different incentive design mechanisms like contract theory have been proposed recently, where the cooperation is encouraged by offering appropriate reward to the sensing participants. However, still no secure structure exists for the reward distribution after a successful contract completion, and a user may also deny sensing even after accepting the contract. This work proposes the use of blockchain-based smart contracts, which are based upon a permissioned blockchain that helps in making the transactions private among the participants. Further, the efficiency of this approach is improved by introducing a reputation parameter for the users. This measures the trustworthiness of a particular user based upon the accuracy of their recent sensing results. Further, a money locking approach is proposed from which a penalty is deducted if the user denies or fails to submit a correct reading. Performance analysis of the proposed scheme is done to show the superiority of the scheme.

Deep Q-learning multiple networks based dynamic spectrum access with energy harvesting for green cognitive radio network

In cognitive radio (CR), dynamic spectrum access (DSA) can be used for secondary user (SU) to improve its spectrum efficiency under the limited spectrum resources. However, the SU has to consume more energy to sense the state of primary user (PU) before spectrum access. Therefore, high spectrum efficiency and lower energy consumption have become the requirements of CR. In this paper, we propose a sustainable green CR network combining DSA and energy harvesting, which can perform intelligent spectrum sensing, spectrum access and energy harvesting by a reward mechanism of deep Q-learning multiple networks (DQMN). The DQMN model consisting of multiple neural networks is designed to learn multiple learning objectives, including spectrum sensing, channel selection, communication power and access mode in DSA. A sleep mechanism is introduced to optimize the spectrum access mode and further reduce the energy consumption. The simulation results show that the DQMN-based DSA model can autonomously sense the channel state, harvest the energy, and then select the appropriate access strategy. Due to the energy harvesting and sleep mechanism, the proposed system model shows lower energy consumption and higher average effective throughput than the traditional models.

Innovative Spectrum Handoff Process Using a Machine Learning-Based Metaheuristic Algorithm.

A cognitive radio network (CRN) is an intelligent network that can detect unoccupied spectrum space without interfering with the primary user (PU). Spectrum scarcity arises due to the stable channel allocation, which the CRN handles. Spectrum handoff management is a critical problem that must be addressed in the CRN to ensure indefinite connection and profitable use of unallocated spectrum space for secondary users (SUs). Spectrum handoff (SHO) has some disadvantages, i.e., communication delay and power consumption. To overcome these drawbacks, a reduction in handoff should be a priority. This study proposes the use of dynamic spectrum access (DSA) to check for available channels for SU during handoff using a metaheuristic algorithm depending on machine learning. The simulation results show that the proposed "support vector machine-based red deer algorithm" (SVM-RDA) is resilient and has low complexity. The suggested algorithm's experimental setup offers several handoffs, unsuccessful handoffs, handoff delay, throughput, signal-to-noise ratio (SNR), SU bandwidth, and total spectrum bandwidth. This study provides an improved system performance during SHO. The inferred technique anticipates handoff delay and minimizes the handoff numbers. The results show that the recommended method is better at making predictions with fewer handoffs compared to the other three.

Integrated Cooperative Spectrum Sensing and Access Control for Cognitive Industrial Internet of Things

Industrial Internet of Things (IIoT) usually utilizes 2.4-GHz unlicensed frequency band, which is also heavily used by many other communication systems, such as ZigBee, WiFi, Bluetooth, etc. Therefore, the lack of spectrum resources has become a key technical bottleneck to restrict the development of IIoT. Integrating cognitive radio (CR) into IIoT, Cognitive IIoT (CIIoT) can cope with the spectrum resource shortage by accessing the frequency bands licensed to primary user (PU). However, spectrum sensing and access control must be performed to avoid bringing severe interference to the PU. In this article, an integrated cooperative spectrum sensing (CSS) and access control model is proposed to improve the transmission performance of the CIIoT while guaranteeing the CSS’s detection probability and controlling the interference to the PU. This model is optimized to maximize the total throughput of IIoT in each frame by jointly optimizing sensing time, the number of sensing nodes and the transmit power for each node under the constraints of the minimum detection probability, the total power control, the interference control, and the minimum rate for each node. The optimization problem is solved by the joint optimization of spectrum sensing and access control. A simultaneous CSS and access control model is also proposed to increase the communication time by using one time slot to perform CSS and access control simultaneously. The simulation results show that there exist optimal sensing and control parameters to maximize the total throughput of CIIoT.

When DSA Meets SWIPT: A Joint Power Allocation and Time Splitting Scheme Based on Multi-Agent Deep Reinforcement Learning

Dynamic spectrum access (DSA) and simultaneously wireless information and power transfer (SWIPT) are two promising approaches to address the spectrum and energy supply problems. However, most existing works studied these two techniques separately, and how to effectively realize them simultaneously is non-trivial, because the power allocation of DSA and time splitting of SWIPT will influence both data transmission and energy harvesting performance. Hence, a sophisticated joint design is necessary, which, however, is challenging considering the uncertain dynamic spectrum environment. In this paper, we propose a joint power allocation and time splitting scheme to facilitate devices to implement both DSA and SWIPT simultaneously, which aims at maximizing the long-term throughput while ensuring the interference limitation and energy supply requirements. Considering the environmental uncertainty, we develop a multi-agent deep reinforcement learning solution to make each device autonomously select the best decision that could optimize the whole network performance based on its local observations. Simulation results have shown the effectiveness of the proposed scheme.

Reconfigurable Intelligent Surface-Assisted Secondary Communication System Coexisting With Multiple Primary Networks

In a secondary communication system (SCS) coexisting with multiple primary networks, the coupling interference between the SCS and primary networks would deteriorate the spectral efficiency (SE) of the SCS and impede the spectrum access of massive secondary devices. In this paper, a well-designed reconfigurable intelligent surface (RIS) is employed to control wireless channels, which can achieve a high SE of the SCS and relieve the scarcity of the spectrum resource. Specifically, the SE enhancement problem is decomposed into two subproblems which are a second-order cone programming and a fractional programming of the convex quadratic form (CQFP), respectively, to optimize alternatively the beamforming vector at the secondary access point and the reflecting coefficients at the RIS. The CQFP subproblem is solved by the domain and envelope shrinking (DES) algorithm, providing the best SE performance. Besides, a low-complexity method of gradient-based linearization with domain (GLD) is proposed for obtaining a sub-optimal reflecting coefficients for fast deployment. Moreover, a nearest point searching with penalty method is also developed to realize the discretization of the phase shifts at the RIS in practice. The simulation results validate the effectiveness of the DES algorithm and show that the proposed GLD algorithm improves the SE performance significantly.

PRAM: A Practical Sybil-Proof Auction Mechanism for Dynamic Spectrum Access With Untruthful Attackers

Auction is becoming increasingly popular for dynamic spectrum access (DSA), while it is extremely vulnerable to sybil attacks. Existing studies on sybil-proof DSA auction impractically assume that attackers bid truthfully based on true appraisals. This paper, for the first time, considers untruthful attackers and investigates the sybil-proof auction design in such more hazardous scenarios. To justify the new assumption, we first show that attackers obtain higher utilities by bidding untruthfully, especially in networks with inadequate channels. Based on this novel finding, we then design a practical sybil attack model named EqualSumBid Sybil, where attackers follow an equal-sum rule (i.e., the sum bid value of the multiple identities of an attacker equals the bid value when it bids with only one identity) instead of their true appraisals. To ensure efficient DSA under the new attack, we finally propose the PRAM, a Practical sybil-pRoof Auction Mechanism, where suspicious identity merging and bid-independent bidder sorting methods are introduced to alleviate the effect of untruthfulness on spectrum auction. Furthermore, winner selection and payment methods are designed to resist the EqualSumBid Sybil attack. Theoretical analyses and numerical results show that PRAM not only resists the EqualSumBid Sybil attack but also achieves individual rationality and truthfulness.

Distributed Learning Meets 6G: A Communication and Computing Perspective

With the ever improving computing capabilities and storage capacities of mobile devices in line with evolving telecommunication network paradigms, there has been an explosion of research interest toward exploring distributed learning (DL) frameworks to realize stringent key performance indicators (KPIs) that are expected in next-generation/6G cellular networks. In conjunction with edge computing, federated learning (FL) has emerged as the DL architecture of choice in prominent wireless applications. This article provides an outline of how DL in general and FL-based strategies specifically can contribute toward realizing part of the 6G vision and strike a balance between communication and computing constraints. As a practical use case, we apply multi-agent reinforcement learning within the FL framework to the dynamic spectrum access (DSA) problem and present preliminary evaluation results. Top contemporary challenges in applying DL approaches to 6G networks are also highlighted.

Energy-efficient adaptive sensing for Cognitive Radio Sensor Network in the presence of Primary User Emulation Attack

A Cognitive Radio Sensor Network (CRSN) is a promising technology incorporating cognitive radio in traditional wireless sensor networks. In general, Primary User Emulation Attack (PUEA) is becoming the major bottleneck for implementing CRSN in reality. In CRSN-PUEA, malicious users are also equipped with a Spectrum Sensing capability, the so-called smart PEUA, which further complicates the spectrum access of Cognitive Users (CU). In this paper, we propose a novel Energy-Efficient Adaptive Sensing (EEAS) technique for CRSN to overcome the spectrum access problem in the presence of PUEA. The proposed EEAS enhances the sensing accuracy by differentiating the PUEA and Primary User signals that, reduce the sensing error probability and enhance the underlying system model’s throughput and energy efficiency. The numerical results show that the proposed EEAS technique has improved the throughput and energy efficiency by about 12% and 44%, respectively, compared to the state-of-the-art works in the field of PUEA-CRSN. In addition, it is observed that the sensing error probability is reduced by 22%. Furthermore, a Software Defined Radio (SDR) based hardware system setup has been developed to verify the functionality of the proposed system model.