Power Allocation Policy Research Articles

COSMO: Dynamic Uploading Scheduling in mmWave-Based Sensor Networks with Mobile Blockers

Wireless sensor networks (WSNs) leveraging millimeter wave (mmWave) communication for bandwidth-demanding applications is considered in this paper. Despite the large bandwidth, the delivery of delay-sensitive information collected by sensors may still face significant latency due to the vulnerability to intermittent link blockage. Hence, the guarantee of low age of information (AoI) in mmWave WSNs is not straightforward. In this paper, the wireless sensing and dynamic programming techniques are jointly exploited to relieve the above issue. The former tracks the human blockers and predicts the chance of link blockage; the latter optimizes the transmission of multiple sensors based on the prediction. Particularly, the long-term optimization of sampling, uplink time and power allocation policies in a sensor network can be formulated as an infinite-horizon Markov decision process (MDP) with discounted cost, where the state transition probabilities can be predicted via wireless sensing. A novel low-complexity solution framework, namely COSMO, with a guaranteed performance in the worst case, is proposed. Simulations show that compared with heuristic benchmarks, benefiting from the prediction of the link blockage, COSMO can significantly suppress the average system cost, which consists of both AoI and energy consumption.

Outage and throughput performance analysis of smart UAV‐assisted full‐duplex NOMA‐based energy harvesting cognitive radio networks

SummaryIn this paper, the throughput and outage performance for an unmanned aerial vehicle (UAV)‐assisted non‐orthogonal multiple access (NOMA)‐based hybrid cognitive radio (CR) network are examined under m‐Nakagami fading channel. To be more specific, a secondary source transmitter ( ) broadcasts a NOMA‐enabled mixed signal to secondary destinations via UAV relay, which operates in full‐duplex (FD) mode and employs decode‐and‐forward protocol. In our approach, an energy harvesting (EH) circuit is employed at both and UAV to harvest energy from the PU radio frequency (RF) signals. These secondary transmitters are also incorporated with energy detector (ED) circuit as well, which detects the activity of the primary user (PU). The status of PU to be busy or idle is confirmed on the basis of the energy detector's results. In turn, this drives and UAV to switch between underlay and overlay protocols in an adaptive manner to increase spectrum efficiency (SE). The closed form mathematical expressions of the secondary outage probability for NOMA‐based UAV equipped CR networks under imperfect successive interference cancellation (i‐SIC) conditions have been evaluated. The administration of the power allocation policy at the and UAV is assessed while maintaining the PU quality of service (QoS). Finally, MATLAB simulation testbed has been used to verify all the analytical closed form expressions.

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

The swarm relay and power allocation policy determines the bit error rate and the energy consumption of unmanned aerial vehicles (UAVs) and can be optimized based on the network and jamming model, which is rarely known by UAVs. In this paper, we propose a multi-agent reinforcement learning (RL)-based UAV swarm communication scheme to optimize the relay selection and power allocation against jamming. Based on the network topology, channel states, previous performance and observations shared by the neighboring UAVs, this scheme formulates the policy distribution to improve the policy exploration and applies a policy learning mechanism to stabilize the learning process. Based on transfer learning, the shared swarm experiences are exploited to accelerate the initial learning and improve policy optimization. A deep RL-based scheme is proposed to mitigate the state quantization error for the rapidly changing channel states under high swarm moving speed and thus further improve the anti-jamming performance. This scheme designs a policy network with four fully connected layers to approximate the policy distribution and uses another two neural networks to estimate the average policy distribution and the expected long-term utility, respectively, to update the policy network for stabilized deep learning. We investigate the computational complexity and derive the performance bound regarding the bit error rate, the energy consumption and the utility. Simulation and experimental results verify the performance gain of our proposed schemes over related works.

Simplified Deep Reinforcement Learning Approach for Channel Prediction in Power Domain NOMA System.

In this work, the impact of implementing Deep Reinforcement Learning (DRL) in predicting the channel parameters for user devices in a Power Domain Non-Orthogonal Multiple Access system (PD-NOMA) is investigated. In the channel prediction process, DRL based on deep Q networks (DQN) algorithm will be developed and incorporated into the NOMA system so that this developed DQN model can be employed to estimate the channel coefficients for each user device in NOMA system. The developed DQN scheme will be structured as a simplified approach to efficiently predict the channel parameters for each user in order to maximize the downlink sum rates for all users in the system. In order to approximate the channel parameters for each user device, this proposed DQN approach is first initialized using random channel statistics, and then the proposed DQN model will be dynamically updated based on the interaction with the environment. The predicted channel parameters will be utilized at the receiver side to recover the desired data. Furthermore, this work inspects how the channel estimation process based on the simplified DQN algorithm and the power allocation policy, can both be integrated for the purpose of multiuser detection in the examined NOMA system. Simulation results, based on several performance metrics, have demonstrated that the proposed simplified DQN algorithm can be a competitive algorithm for channel parameters estimation when compared to different benchmark schemes for channel estimation processes such as deep neural network (DNN) based long-short term memory (LSTM), RL based Q algorithm, and channel estimation scheme based on minimum mean square error (MMSE) procedure.

Multi-User Downlink NOMA Systems Aided by Ambient Backscattering: Achievable Rate Regions and Energy-Efficiency Maximization

In this paper, we investigate the energy efficiency of a multi-user downlink non-orthogonal multiple access (NOMA) system aided by an ambient backscatter device that modulates its own information by reflecting the incident signal coming from the NOMA transmitter. Because of the multiplicative operation at the ambient backscatter device when reflecting the transmitter’s signal, the achievable sum rate of the system is not known. Hence, we first derive the information-theoretic achievable rate region for a discrete memoryless channel and, subsequently, for Gaussian channels. We then propose a joint optimization framework for maximizing the system energy efficiency as the tradeoff and ratio between the overall sum rate and the power consumption, under user minimum rate constraints. For this, we propose a modification that simplifies the resulting non-convex optimization problems, which enables us to obtain the optimal reflection coefficient and power allocation policy analytically (up to a one line search). Numerical results demonstrate the negligible impact of the introduced modification on the optimality of our solution. Remarkably, our results show that the ambient backscatter-aided NOMA significantly outperforms OMA: the relative gain increases with the number of receivers, reaching up to 14× improvement over OMA. At last, we show the pertinence of our solution also under imperfect channel state information.

Performance of the NOMA user in FFR Millimeter Wave Networks

ABSTRACT Non-Orthogonal Multiple Access (NOMA) and Fractional Frequency Reuse (FFR) were touted as two key techniques of the millimetre wave cellular system to derive spectrum efficiency and user performance improvement. By combining the NOMA and FFR techniques, this paper proposes a system to improve the performance of the NOMA user that utilises the NOMA-power domain technique to reuse an occupied Resource Block (RB). Besides following the power allocation policy of the NOMA technique, the resource allocation algorithm for the NOMA user is based on the FFR technique. Particularly, if the downlink Signal-to-Interference-plus-Noise ratio (SINR) of the NOMA user is greater than the threshold, it is allocated the Cell-Edge RB with a high interference level. Otherwise, it is served on the Cell-Center RB with a low interference level. The analytical and simulation results state that the proposed system can increase the coverage probability of the NOMA user up to 250%.

Incentive learning-based energy management for hybrid energy storage system in electric vehicles

Deep reinforcement learning has emerged as a promising candidate for online optimal energy management of multi-energy storage vehicles. However, how to ensure the adaptability and optimality of the reinforcement learning agent under realistic driving conditions is still the main bottleneck. To enable the reinforcement learning agent to efficiently learn the optimal power allocation strategies under diverse driving conditions, this paper proposes an incentive learning-based energy management strategy for battery-supercapacitor electric vehicles to minimize the battery capacity loss cost and power loss cost. First, an incentive reward function based on supercapacitor state-of-charge and vehicle acceleration is proposed for proximal policy optimization-based energy management strategy, which can stimulate the agent to learn for optimal power allocation policy under high load power conditions quickly. Second, a random sampling-based velocity transfer probability surface is constructed for pre-training to guarantee strategy optimality under unfamiliar driving cycles. Third, the generalized advantage estimation and layer normalization of neural networks are incorporated to improve the learning convergence. Results show that the proposed method can reduce the above costs by 5.8%–13.8% and 11.7%–38.8% compared with existing deep reinforcement learning methods under the pre-training driving cycle and test driving cycles, respectively, which yields closer results to offline dynamic programming.

Low‐complexity joint equalization and CFO compensation in uplink SC‐FDMA NOMA system under different power allocation strategies

AbstractIn current wireless systems, the claim for rapid data services has become unavoidable in broadband communication. Single carrier frequency division multiple access (SC‐FDMA) has remained a multiple access (MA) technique with low peak‐to‐average power ratio (PAPR) in 4G systems. Non‐orthogonal multiple access (NOMA) is a complimentary MA technique in the fifth generation (5G) new radio with the capability of using similar frequency elements for multiuser communication within a single cellular system. In SC‐FDMA NOMA systems the carrier frequency offsets (CFO) destroy the orthogonal behavior in multiple carriers during transmission leading to inter‐carrier interference (ICI) and multiple access interference (MAI). Furthermore, the power allocated for each user in the NOMA cluster exhibits a noticeable part in enhancing the performance of the system. In this article, we propose a joint low‐complexity linear regularized zero forcing (JLC‐LRZF) for SC‐FDMA NOMA system and investigate its performance using discrete Fourier transform (DFT) and discrete cosine transform (DCT). The proposed JLC‐LRZF equalization algorithm is capable of performing equalization and CFO compensation with low‐complexity with banded‐matrix approximation (BMA). Additionally, we investigate the effectiveness of SC‐FDMA NOMA system in achieving improved performance with different values of CFO and power allocation policies in uniform random multipath channels.

Energy-Efficient Beamforming and Resource Optimization for AmBSC-Assisted Cooperative NOMA IoT Networks

In this manuscript, we present an energy-efficient alternating optimization framework based on the multi-antenna ambient backscatter communication (AmBSC) assisted cooperative non-orthogonal multiple access (NOMA) for next-generation (NG) internet-of-things (IoT) enabled communication networks. Specifically, the energy-efficiency maximization is achieved for the considered AmBSC-enabled multi-cluster cooperative IoT NOMA system by optimizing the active-beamforming vector and power-allocation coefficients (PAC) of IoT NOMA users at the transmitter, as well as passive-beamforming vector at the multi-antenna assisted backscatter node. Usually, increasing the number of IoT NOMA users in each cluster results in inter-cluster interference (ICI) (among different clusters) and intra-cluster interference (among IoT NOMA users). To combat the impact of ICI, we exploit a zero-forcing (ZF) based active-beamforming, as well as an efficient clustering technique at the source node. Further, the effect of intra-cluster interference is mitigated by exploiting an efficient power-allocation policy that determines the PAC of IoT NOMA users under the quality-of-service (QoS), cooperation, SIC decoding, and power-budget constraints. Moreover, the considered non-convex passive-beamforming problem is transformed into a standard semi-definite programming (SDP) problem by exploiting the successive-convex approximation (SCA) approximation, as well as the difference of convex (DC) programming, where Rank-1 solution of passive-beamforming is obtained based on the penalty-based method. Furthermore, the numerical analysis of simulation results demonstrates that the proposed energy-efficiency maximization algorithm exhibits an efficient performance by achieving convergence within only a few iterations.

QoS-aware secure transmission design for VLC with semi-grant-free NOMA scheme.

This work aims to enhance the physical layer security (PLS) of non-orthogonal multiple access (NOMA) aided indoor visible light communication (VLC) system with semi-grant-free (SGF) transmission scheme, in which a grant-free (GF) user shares the same resource block with a grant-based (GB) user whose quality of service (QoS) must be strictly guaranteed. Besides, the GF user is also provided with an acceptable QoS experience, which is closely aligned with the practical application. Both active and passive eavesdropping attacks are discussed in this work, where users' random distributions are taken into account. Specifically, to maximize the secrecy rate of the GB user in the presence of an active eavesdropper, the optimal power allocation policy is obtained in exact closed-form and the user fairness is then assessed by Jain's fairness index. Moreover, the secrecy outage performance of the GB user is analyzed in the presence of the passive eavesdropping attack. Both exact and asymptotic theoretical expressions for the secrecy outage probability (SOP) of the GB user are derived, respectively. Furthermore, the effective secrecy throughput (EST) is investigated on the basis of the derived SOP expression. Through simulations, it is found that the PLS of this VLC system can be significantly improved by the proposed optimal power allocation scheme. The radius of the protected zone, the outage target rate for the GF user, and the secrecy target rate for the GB user would have pronounced impacts on the PLS and user fairness performance of this SGF-NOMA assisted indoor VLC system. The maximum EST will increase with the increasing transmit power and it is hardly influenced by the target rate for the GF user. This work will benefit the design of indoor VLC system.

A Study on the Impact of Integrating Reinforcement Learning for Channel Prediction and Power Allocation Scheme in MISO-NOMA System.

In this study, the influence of adopting Reinforcement Learning (RL) to predict the channel parameters for user devices in a Power Domain Multi-Input Single-Output Non-Orthogonal Multiple Access (MISO-NOMA) system is inspected. In the channel prediction-based RL approach, the Q-learning algorithm is developed and incorporated into the NOMA system so that the developed Q-model can be employed to predict the channel coefficients for every user device. The purpose of adopting the developed Q-learning procedure is to maximize the received downlink sum-rate and decrease the estimation loss. To satisfy this aim, the developed Q-algorithm is initialized using different channel statistics and then the algorithm is updated based on the interaction with the environment in order to approximate the channel coefficients for each device. The predicted parameters are utilized at the receiver side to recover the desired data. Furthermore, based on maximizing the sum-rate of the examined user devices, the power factors for each user can be deduced analytically to allocate the optimal power factor for every user device in the system. In addition, this work inspects how the channel prediction based on the developed Q-learning model, and the power allocation policy, can both be incorporated for the purpose of multiuser recognition in the examined MISO-NOMA system. Simulation results, based on several performance metrics, have demonstrated that the developed Q-learning algorithm can be a competitive algorithm for channel estimation when compared to different benchmark schemes such as deep learning-based long short-term memory (LSTM), RL based actor-critic algorithm, RL based state-action-reward-state-action (SARSA) algorithm, and standard channel estimation scheme based on minimum mean square error procedure.

Secondary Network Throughput Optimization of NOMA Cognitive Radio Networks Under Power and Secure Constraints

Recently, the combination of cognitive radio networks with the nonorthogonal multiple access (NOMA) approach has emerged as a viable option for not only improving spectrum usage but also supporting large numbers of wireless communication connections. However, cognitive NOMA networks are unstable and vulnerable because multiple devices operate on the same frequency band. To overcome this drawback, many techniques have been proposed, such as optimal power allocation and interference cancellation. In this paper, we consider an approach by which the secondary transmitter (STx) is able to find the best licensed channel to send its confidential message to the secondary receivers (SRxs) by using the NOMA technique. To combat eavesdroppers and achieve reasonable performance, a power allocation policy that satisfies both the outage probability (OP) constraint of primary users and the security constraint of secondary users is optimized. The closed-form formulas for the OP at the primary base station and the leakage probability for the eavesdropper are obtained with imperfect channel state information. Furthermore, the throughput of the secondary network is analyzed to evaluate the system performance. Based on that, two algorithms (i.e., the continuous genetic algorithm (CGA) for CR NOMA (CGA-CRN) and particle swarm optimization (PSO) for CR NOMA (PSO-CRN)), are applied to optimize the throughput of the secondary network. These optimization algorithms guarantee not only the performance of the primary users but also the security constraints of the secondary users. Finally, simulations are presented to validate our research results and provide insights into how various factors affect system performance.

Secrecy Sum-Rate Enhancement for NOMA-VLC System With Pseudo User

Non-orthogonal multiple access (NOMA) enabled visible light communication (VLC) system is vulnerable to malicious eavesdropping, mainly because of the inherent broadcast nature of VLC technology. To this end, this letter proposes a novel concept called pseudo user (PU), which obstructs the successive interference cancellation (SIC) procedures of the eavesdropper. The minimum-feasible (MF) power allocation policy toward the PU is obtained under different power allocation strategies. Then, the secrecy sum-rate and user fairness along with the sum-rate are evaluated under both perfect SIC (pSIC) and imperfect SIC (ipSIC) scenarios. Results show that compared with the benchmark scheme without PU, the secrecy sum-rate and user fairness are both greatly enhanced with slight sum-rate loss. Furthermore, this letter proves that with much lower computations, the proposed scheme yields close-to-optimum secrecy sum-rate and user fairness performance provided by the exhaustive search algorithm in the pSIC scenario.

Fair Power Allocation Policies for Power-Domain Non-Orthogonal Multiple Access Transmission With Complete or Limited Successive Interference Cancellation

Fair Power Allocation Policies for Power-Domain Non-Orthogonal Multiple Access Transmission With Complete or Limited Successive Interference Cancellation

Power Allocation for Cooperative Jamming Against a Strategic Eavesdropper Over Parallel Channels

This paper considers a friendly interferer allocating jamming power to eavesdropping channels to increase the level of secrecy of a wireless network. The friendly interferer has access to limited power, while the eavesdropper may not have the ability to attack all channels simultaneously. When all channels used for secret communication are under the threat of eavesdropping attacks, the optimal power allocation policy results from solving a convex optimization problem. In this case, the optimal policy is unique and can be obtained via a water-filling scheme. When the eavesdropper can not attack all channels, the eavesdropper should behave strategically and may select targets probabilistically. We propose a non-zero-sum game that helps the friendly interferer predict and concentrate on the targets selected by the eavesdropper. Under certain conditions, we prove that there exists a unique Nash equilibrium (NE) strategy pair, which has a threshold type structure. We provide conditions under which the eavesdropper’s equilibrium strategy is deterministic. We devise a strategy iteration algorithm to compute an equilibrium power allocation strategy. We present examples showing that the game-theoretic power allocation strategy performs better than the conservative power allocation strategy that assumes every channel to be under attack.

Domain Knowledge-Assisted Deep Reinforcement Learning Power Allocation for MIMO Radar Detection

The power allocation of multiple-input multiple-output (MIMO) radars is a key point in target tracking and detection. The optimality of a multitarget and multiconstraint optimization problem strictly depends on a priori model, which is difficult to obtain in time-varying, complex, and noncooperative environments. Recently, deep reinforcement learning (DRL) has been applied for target tracking tasks, which provides a trial-and-error interactive learning mechanism to improve the policy. Unlike tracking tasks with complete target state transition models, it remains an open issue for DRL-based MIMO radar detection that requires efficiently adapting the control policy to the environment of randomly appearing targets and extensive power transmission actions, which leads to sparse final task rewards and hence slow policy learning for the agent. Through introducing both the analytic model (radar equation) and empirical rules (expert preferences) for domain knowledge, this article proposes a domain-knowledge-assisted DRL (DKADRL) framework in which a domain-knowledge-based timely reward generator is utilized to generate timely rewards that assist the agent’s policy learning. To adjust the role of the timely rewards and the final task rewards, a reward fusion module is designed, which gradually increases the role of the final task rewards as the training process progresses, thus allows agent’s policy to converge to the final optimization goal. The algorithm is validated under two target motion scenarios, showing the higher target detection probability and the faster training speed, compared to equal power allocation and proximal policy optimization (PPO)-based power allocation.

User Pairing and Power Allocation for UAV-NOMA Systems Based on Multi-Armed Bandit Framework

In this paper, we investigate the joint user pairing and power coefficient allocation for unmanned aerial vehicle (UAV) systems which employ non-orthogonal multiple access (NOMA) to communicate with multiple ground users. Aiming to maximize achievable sum rate and ensure the users' Quality-of-Service (QoS) requirements, we formulate an optimization problem which relies on reinforcement learning (RL) from Multi-Armed Bandit (MAB) framework to propose a solution based on Upper Confidence Bound (UCB) approach. The proposed solution can successfully identify the best action and selects it more often, which leads to maximum system throughput. The attained results show that the proposed scheme finds the best-performing action fast, while the others methods spend a lot of time exploring non-ideal user pairs. As a result, the proposed method accumulates less regret and achieves satisfactory results in terms of system throughput when compared to other user pairing strategies and power allocation (PA) policies.

Mobile Reconfigurable Intelligent Surfaces for NOMA Networks: Federated Learning Approaches

A novel framework of reconfigurable intelligent surfaces (RISs)-enhanced indoor wireless networks is proposed, where an RIS mounted on the robot is invoked to enable mobility of the RIS and enhance the service quality for mobile users. Meanwhile, non-orthogonal multiple access (NOMA) techniques are adopted to further increase the spectrum efficiency since RISs are capable of providing NOMA with artificial controlled channel conditions, which can be seen as a beneficial operation condition to obtain NOMA gains. To optimize the sum rate of all users, a deep deterministic policy gradient (DDPG) algorithm is invoked to optimize the deployment and phase shifts of the mobile RIS as well as the power allocation policy. In order to improve the efficiency and effectiveness of agent training for the DDPG agents, a federated learning (FL) concept is adopted to enable multiple agents to simultaneously explore similar environments and exchange experiences. We also proved that with the same random exploring policy, the FL armed deep reinforcement learning (DRL) agents can theoretically obtain a reward gain comparing to the independent agents. Our simulation results indicate that the mobile RIS scheme can significantly outperform the fixed RIS paradigm, which provides about three times data rate gain compared to the fixed RIS paradigm. Moreover, the NOMA scheme is capable of achieving a gain of 42% in contrast with the OMA scheme in terms of the sum rate. Finally, the multi-cell simulation proved that the FL enhanced DDPG algorithm has a superior convergence rate and optimization performance than the independent training framework.

Performance evaluation of cognitive relay networks for end user mobile over mixed realistic channels

Cognitive relay network is a spectrum dynamic paradigm that exploits the unused portions of the licensed spectrum. This is based on merging both cooperative relaying techniques and cognitive radio network to achieve spectrum efficiency and enhance the overall system performance. In this paper, the presence of mobile users at the destination node is considered. Here, the end users can navigate at relatively fast vehicular velocities causing dynamic multipath fading and high Doppler shift. which can be fairly modelled using Nakagami- m $m\;$ fading channel (i.e. m < 1 $\;m < 1$ ). In a spectrum scarcity environment, a secondary user must deploy an optimal power allocation policy to get higher transmission rates while the overall interference affecting the primary user (PU) is kept below a certain threshold value. In particular, the outage probability (OP) performance is studied over the mixed Rayleigh and Nakagami-m fading channels for different scenarios and a tight closed-form expressions are derived for the system OP of underlay dual-hop cognitive relay networks with a single amplifiy-and-forward (AF) relay with and without the use of the direct link transmission and selection diversity at the destination with interference power constraints for the primary network over independent and non-identical (i.n.i.d) Rayleigh and Nakagami-m fading channels when m < 1 $m < 1$ based on the statistical characteristics of signal-to-noise ratio. Numerical results are presented to evaluate the impact of the fading parameter, m, the maximum aggregated intrusion constraint, and the locations of the primary users (PUs) on different channel scenarios at high vehicular speeds. Monte Carlo simulations are presented to verify and validate analytical results.

A Proactive Eavesdropping Game in MIMO Systems Based on Multiagent Deep Reinforcement Learning

This paper considers an adversarial scenario between a legitimate eavesdropper and a suspicious communication pair. All three nodes are equipped with multiple antennas. The eavesdropper, which operates in a full-duplex model, aims to wiretap the dubious communication pair via proactive jamming. On the other hand, the suspicious transmitter, which can send artificial noise (AN) to disturb the wiretap channel, aims to guarantee secrecy. More specifically, the eavesdropper adjusts jamming power to enhance the wiretap rate, while the suspicious transmitter jointly adapts the transmit power and noise power against the eavesdropping. Considering the partial observation and complicated interactions between the eavesdropper and the suspicious pair in unknown system dynamics, we model the problem as an imperfect-information stochastic game. To approach the Nash equilibrium solution of the eavesdropping game, we develop a multi-agent reinforcement learning (MARL) algorithm, termed neural fictitious self-play with soft actor-critic (NFSP-SAC), by combining the fictitious self-play (FSP) with a deep reinforcement learning algorithm, SAC. The introduction of SAC enables FSP to handle the problems with continuous and high dimension observation and action space. The simulation results demonstrate that the power allocation policies learned by our method empirically converge to a Nash equilibrium, while the compared reinforcement learning algorithms suffer from severe fluctuations during the learning process.