Unmanned Aerial Vehicle Trajectory Research Articles

Low-Resolution Optimization for an Unmanned Aerial Vehicle Communication Network under a Passive Reconfigurable Intelligent Surface and Active Reconfigurable Intelligent Surface

This paper investigates the optimization of an unmanned aerial vehicle (UAV) network serving multiple downlink users equipped with single antennas. The network is enhanced by the deployment of either a passive reconfigurable intelligent surface (RIS) or an active RIS. The objective is to jointly design the UAV’s trajectory and the low-bit, quantized, RIS-programmable coefficients to maximize the minimum user rate in a multi-user scenario. To address this optimization challenge, an alternating optimization framework is employed, leveraging the successive convex approximation (SCA) method. Specifically, for the UAV trajectory design, the original non-convex optimization problem is reformulated into an equivalent convex problem through the introduction of slack variables and appropriate approximations. On the other hand, for the RIS-programmable coefficient design, an efficient algorithm is developed using a penalty-based approximation approach. To solve the problems with the proposed optimization, high-performance optimization tools such as CVX are utilized, despite their associated high time complexity. To mitigate this complexity, a low-complexity algorithm is specifically tailored for the optimization of passive RIS-programmable reflecting elements. This algorithm relies solely on closed-form expressions to generate improved feasible points, thereby reducing the computational burden while maintaining reasonable performance. Extensive simulations are created to validate the performance of the proposed algorithms. The results demonstrate that the active RIS-based approach outperforms the passive RIS-based approach. Additionally, for the passive RIS-based algorithms, the low-complexity variant achieves a reduced time complexity with a moderate loss in performance.

Multi-agent deep reinforcement learning for trajectory planning in UAVs-assisted mobile edge computing with heterogeneous requirements

In heterogeneous wireless networks, massive user equipments (UEs) generate computing tasks with time-varying heterogeneous requirements. To improve the service quality, this paper formulates a unmanned aerial vehicles (UAVs)-assisted mobile edge computing (MEC) framework for time-varying heterogeneous task requirements. In the framework, the task delay and the number of successfully executed tasks are optimized by jointly controlling the trajectories of multiple UAVs. To address the considered trajectory planning optimization problem, a collaborative multi-agent deep reinforcement learning (MADRL) algorithm is proposed, where each UAV is regarded as a learning agent. First, a counterfactual inference based personalized policy update mechanism is proposed to evaluate the independent policy of agents by comparing the policy with a designed counterfactual policy. Based on this idea, each agent updates a personalized policy from both group and individual interests to improve its cooperation ability in dynamic and complex environments. Then, a diversified experience sampling mechanism is proposed to enhance the efficiency of policy evaluation and update with rich experiences provided by the environment interaction and the modified whale optimization algorithm. Finally, evaluation results demonstrate the superiority and effectiveness of the proposed MADRL algorithm.

Energy-efficient Mobile Edge Computing Assisted by Layered UAVs Based on Convex Optimization

The utilization of unmanned aerial vehicles (UAVs) as aerial mobile edge computing (MEC) servers presents an effective approach for mitigating the computational limitations of intelligent mobile devices. However, the limited battery capacity of UAVs poses a significant challenge in ensuring sustained service provision. To counter this obstacle and enhance energy efficiency, we introduce a partial offloading MEC scheme assisted by layered UAVs (POAL). Our objective is to minimize the total energy consumption of UAVs by jointly optimizing several factors, including wireless channels, transmit power, task partition, computing power, and UAV trajectories. Due to the non-convexity of the problem, we decompose it into three sub-problems and address them iteratively. Firstly, we employ the CVX optimization tool to allocate wireless channels and utilize Lagrangian duality to determine the optimal transmit power. Secondly, leveraging the monotonicity of the optimization objective, we derive the optimal computing power. Lastly, we employ the successive convex approximation (SCA) technique to tackle the trajectory planning sub-problem. Numerical results demonstrate that our proposed algorithm can effectively position UAVs within 15 iterations. Furthermore, our layered scheme significantly reduces overall energy consumption.

UAVs Path Planning based on Combination of Rapidly Exploring Random Tree and Rauch-Tung-Striebel Filter

Aiming at the problem of Unmanned Aerial Vehicle(UAV) formation path planning under complex constraints, a UAV formation path planning method based on the combination of Rapidly exploring Random Tree (RRT) and Rauch-Tung-Striebel (RTS) filter is proposed. Firstly, a path planning algorithm based on the improved RRT algorithm with adaptive step size is de-signed to solve the problem that the RRT algorithm is easy to fall into local optimum. Then, an RTS filter is introduced to smooth the trajectory planned by the improved RRT algorithm to achieve curvature continuity. Finally, taking the smooth trajectory as the reference, a UAV formation path planning algorithm over the Artificial Potential Field (APF) method is designed. The simulation results show that the designed UAV formation path planning algorithm can solve the planning problems of single trajectory and formation trajectories in complex constrained space, and can plan the formation trajectory with continuous curvature, to facilitate the UAV trajectory tracking control.

Reinforcement learning-based unmanned aerial vehicle trajectory planning for ground users’ mobility management in heterogeneous networks

The surge of data traffic in wireless networks necessitates the provision of high-quality data services to meet users’ satisfaction levels. However, the limited spectral resources of the current network infrastructures and inherent challenges of achieving reliable line-of-sight (LoS) probability for ground users (GUs) in urban environments often lead to disruption to communication services delivery. This paper aims to address the challenges of frequent handover (HO) failures and disrupted communication services for mobile GUs by deploying an unmanned aerial vehicle as a flying base station (UAV-BS) in heterogeneous networks (HetNets). A channel model is investigated that considers both LoS and non-line-of-sight (NLoS) paths in three-dimensional (3D) air-to-ground (A2G) links using a detailed mathematical model with urban infrastructure parameters like building density and heights. In addition, a reinforcement learning (RL) algorithm is presented in this work to optimize UAV trajectories in response to the dynamic mobility of GUs for enhancing LoS connections. The proposed algorithm dynamically adjusts the UAV positions and enhances transmission channels by identifying both LoS and NLoS paths. Simulation results demonstrate that the proposed algorithm outperforms existing benchmarks through learning-based adaptive control of UAVs’ mobility, ensuring ubiquitous network connectivity for GUs and reducing HO failures in HetNets.

Unlocking the Potential of Mobile-Edge Cloud: A Comprehensive Review and Future Directions

With the rapid expansion of the Internet of Things (IoT), the demand for computational resources continues to soar, necessitating innovative solutions to address the needs of resource-constrained IoT users. Mobile edge computing (MEC) emerges as a promising remedy, mitigating the strain imposed by resource-intensive mobile applications. Concurrently, leveraging unmanned aerial vehicles (UAVs) as aerial platforms presents an enticing opportunity to enhance connectivity in wireless networks, owing to their on-demand deployment capabilities, high cruising altitudes, and maneuverability in three-dimensional space. This paper presents a comprehensive examination of UAV-enabled aerial MEC, elucidating its advantages, challenges, and recent advancements across various domains. Topics explored include joint optimization of UAV trajectory, computation offloading, and resource allocation, UAV deployment strategies, task scheduling, load balancing, interplay with other technologies, and machine learning-driven optimizations. Additionally, the paper outlines key avenues for future research endeavors.

Unlocking the Potential of Mobile-Edge Cloud: A Comprehensive Review and Future Directions

With the rapid expansion of the Internet of Things (IoT), the demand for computational resources continues to soar, necessitating innovative solutions to address the needs of resource-constrained IoT users. Mobile edge computing (MEC) emerges as a promising remedy, mitigating the strain imposed by resource-intensive mobile applications. Concurrently, leveraging unmanned aerial vehicles (UAVs) as aerial platforms presents an enticing opportunity to enhance connectivity in wireless networks, owing to their on-demand deployment capabilities, high cruising altitudes, and maneuverability in three-dimensional space. This paper presents a comprehensive examination of UAV-enabled aerial MEC, elucidating its advantages, challenges, and recent advancements across various domains. Topics explored include joint optimization of UAV trajectory, computation offloading, and resource allocation, UAV deployment strategies, task scheduling, load balancing, interplay with other technologies, and machine learning-driven optimizations. Additionally, the paper outlines key avenues for future research endeavors.

Quadrotor UAV Trajectory Tracking Control Based on Adaptive Non-singular Terminal Sliding Mode

To address the issue of the quadrotor unmanned aerial vehicle (UAV) being susceptible to external disturbances and model uncertainties during flight, an adaptive non-singular terminal sliding mode controller with disturbance compensation is proposed. Firstly, the quadrotor UAV dynamic model considering system disturbances is constructed using the Newton-Euler method. Next, the trajectory tracking problem is transformed into a command tracking control problem for each channel. To enhance the adaptability and robustness of trajectory tracking for the quadrotor UAV, an adaptive law is designed to dynamically adjust the sliding mode surface gain. In addition, a disturbance observer is introduced to estimate and compensate for external disturbances and model uncertainties. Finally, the stability of the designed controller is proven through the Lyapunov function. Simulation results demonstrate that the proposed controller effectively enhances the disturbance resistance and trajectory tracking response speed of the quadrotor UAV.

Trajectory Planning for UAV-Assisted Data Collection in IoT Network: A Double Deep Q Network Approach

Unmanned aerial vehicles (UAVs) are becoming increasingly valuable as a new type of mobile communication device and autonomous decision-making device in many application areas, including the Internet of Things (IoT). UAVs have advantages over other stationary devices in terms of high flexibility. However, a UAV, as a mobile device, still faces some challenges in optimizing its trajectory for data collection. Firstly, the high complexity of the movement action and state space of the UAV’s 3D trajectory is not negligible. Secondly, in unknown urban environments, a UAV must avoid obstacles accurately in order to ensure a safe flight. Furthermore, without a priori wireless channel characterization and ground device locations, a UAV must reliably and safely complete the data collection from the ground devices under the threat of unknown interference. All of these require the proposing of intelligent and automatic onboard trajectory optimization techniques. This paper transforms the trajectory optimization problem into a Markov decision process (MDP), and deep reinforcement learning (DRL) is applied to the data collection scenario. Specifically, the double deep Q-network (DDQN) algorithm is designed to address intelligent UAV trajectory planning that enables energy-efficient and safe data collection. Compared with the traditional algorithm, the DDQN algorithm is much better than the traditional Q-Learning algorithm, and the training time of the network is shorter than that of the deep Q-network (DQN) algorithm.

Real-Time 3D Routing Optimization for Unmanned Aerial Vehicle using Machine Learning

In the realm of Unmanned Aerial Vehicles (UAVs) for civilian applications, the surge in demand has underscored the need for sophisticated technologies. The integration of Unmanned Aerial Systems (UAS) with Artificial Intelligence (AI) has become paramount to address challenges in urban environments, particularly those involving obstacle collision risks. These UAVs are equipped with advanced sensor arrays, incorporating LiDAR and computer vision technologies. The AI algorithm undergoes comprehensive training on an embedded machine, fostering the development of a robust spatial perception model. This model enables the UAV to interpret and navigate through the intricate urban landscape with a human-like understanding of its surroundings. During mission execution, the AI-driven perception system detects and localizes objects, ensuring real-time awareness. This study proposes an innovative real-time three-dimensional (3D) path planner designed to optimize UAV trajectories through obstacle-laden environments. The path planner leverages a heuristic A* algorithm, a widely recognized search algorithm in artificial intelligence. A distinguishing feature of this proposed path planner is its ability to operate without the need to store frontier nodes in memory, diverging from conventional A* implementations. Instead, it relies on relative object positions obtained from the perception system, employing advanced techniques in simultaneous localization and mapping (SLAM). This approach ensures the generation of collision-free paths, enhancing the UAV's navigational efficiency. Moreover, the proposed path planner undergoes rigorous validation through Software-In-The-Loop (SITL) simulations in constrained environments, leveraging high-fidelity UAV dynamics models. Preliminary real flight tests are conducted to assess the real-world applicability of the system, considering factors such as wind disturbances and dynamic obstacles. The results showcase the path planner's effectiveness in providing swift and accurate guidance, thereby establishing its viability for real-time UAV missions in complex urban scenarios.

Trajectory control for a Hexacopter UAV using Nonsingular Terminal Sliding Mode Control

Research and development in the field of UAVs have witnessed a significant surge lately. This study aims to design a control strategy for tracking the trajectory of unmanned aerial vehicles (UAV) Hexacopter based on sliding mode control theory. First, the non-linear translational and rotational mathematical model of the Hexacopter is initially derived using the Newton-Euler formulation. Subsequently, a nonsingular fast terminal sliding mode controller (NSTSMC) is developed to enable the system to accurately follow the given flight trajectory, while accounting for variations in the three orientation angles. For that, six sliding manifolds are designed to have a fast dynamic response, and their stability analysis is verified using the Lyapunov direct method. To enhance the robustness of the flight controller and the control quality, external disturbances are taken into account in the modeling and control development, and chattering reduction is also taken into account. The proposed control's validity and performance are evaluated in MATLAB/Simulink compared to the classical PID controller. The comparative simulation results indicated that the aircraft under the NSTSMC could effectively flow along the predetermined trajectory, and counteract disturbances that had a negligible impact on the path without requiring undue exertion.

Unmanned aerial vehicle-assisted wideband cognitive radio network based on DDQN-SAC

Cognitive radio (CR) systems have emerged as effective tools for improving spectrum efficiency and meeting the growing demands of communication. This study focuses on a flexible CR system based on opportunistic spectrum access technology, which enables secondary networks to efficiently utilize unoccupied spectrum resources for information transmission by actively sensing the spectrum utilization of primary networks. Specifically, we introduce unmanned aerial vehicles (UAV) technology into the CR system to further enhance its flexibility and adaptability, which enables the transmission efficiency of low-altitude UAV networks. In this CR system, UAVs are employed for more flexible spectrum management. The objective of this research is to maximize the average achievable rate of SUs by jointly optimizing the trajectories of secondary UAV, the trajectories of primary UAV, the beamforming of secondary UAV, subchannel allocation and sensing time. To achieve this goal, we employ deep reinforcement learning (DRL) algorithms to optimize these variables. Compared to traditional optimization algorithms, DRL algorithms not only have lower computational complexity but also achieve faster convergence. To address the mixed-action space problem, we propose a Dueling DQN-Soft Actor Critic algorithm. Simulation results demonstrate that the proposed approach in this paper significantly enhances the performance of the CR system compared to traditional baseline schemes. This is manifested in higher spectrum efficiency and data transmission rates, while minimizing interference with the primary network. This innovative research combines drone technology and DRL algorithms, bringing new opportunities and challenges to the future development of cognitive communication systems.

Tradeoff Between Age of Information and Operation Time for UAV Sensing Over Multi-Cell Cellular Networks

Unmanned aerial vehicles (UAVs) have a significant potential for sensing applications in further cellular networks due to their extensive coverage and flexible deployment. In this paper, we consider a multi-cell cellular network with a cellular-connected UAV, which senses data with onboard sensors and uploads sensory data to the ground base stations (BSs). To evaluate the freshness of sensory data, we employ the concept of age of information (AoI), which is defined as the time elapsed since the latest successful transmission of sensory data. A lower AoI implies fresher sensory data, which may lead to the increase of UAV operation time. To balance such tradeoff, we aim to minimize the weighted sum of operation time and total AoI for the UAV by jointly optimizing transmission scheduling, BS association, as well as UAV trajectory. The problem is formulated as a mixed-integer nonlinear programming (MINLP) problem, which is difficult to solve due to the time-varying propagation channels. To this end, we first characterize the average communication performance with statistic channel information, and then develop a search algorithm to obtain the optimal solution via employing the optimal structure as well as convex optimization techniques, while a low-complexity Double Graph based Algorithm (DGA) is developed to obtain a suboptimal solution. Then, by taking into account the site-specific performance and making fast decisions online, we propose a Deep reinforcement Learning Algorithm (DLA). Compared to DGA, DLA can adapt to the specific local environment and obtain a solution more rapidly once the training process is completed. Simulation results show that the proposed algorithms outperform the benchmarks about 30%, and achieve flexible tradeoff between operation time and AoI of UAV sensing, which is not available by considering just one objective.

Delanalty Minimization With Reinforcement Learning in UAV-Aided Mobile Network

Unmanned aerial vehicle (UAV)-assisted mobile communication has been studied in recent years. UAVs can be used as aerial base stations (BSs) to improve the performance of terrestrial mobile network. In this article, mobile data offloading with UAV trajectory optimization is investigated. To tackle with the delay of requesting data and the immediacy of requested data at the same time, a new metric named delanalty is newly proposed. The delanalty metric jointly considers the delay of user requesting data, the immediacy of requested data file, and the quantity of residual requesting data. A find max delanalty user mechanism is proposed to eliminate the user who has the largest delay time. Furthermore, an actor–critic (AC)-based deep reinforcement learning (DRL) algorithm called AC -based delanalty trajectory optimization (ACDTO) algorithm is proposed to solve UAV’s trajectory optimization problem. Simulation results show that the proposed ACDTO algorithm can find an optimal flight trajectory with minimal delanalty for all users.

Deep reinforcement learning aided secure UAV communications in the presence of moving eavesdroppers

Integrating unmanned aerial vehicles (UAVs) with wireless systems is widely anticipated to bring enormous benefits to network users, and is considered as a key enabling technology for next generation wireless networks. However deployment of UAVs raises security concerns, as they are prone to eavesdropping attacks. In this paper we investigate secure data transmissions in a UAV-aided communication system from the physical layer security (PLS) perspective. The UAV is employed as a flying base station and transmit confidential information to a ground user. Specifically, an eavesdropper is moving in the vicinity of the ground user, attempting to intercept legitimate data transmission. We aim to optimize UAV trajectory to maximize secrecy throughput and due to the fast changing environment with unpredictable movements of the eave, this nonconvex problem is difficult to solve. Instead, we propose a deep reinforcement learning framework by reformulating this problem as a Markov Decision Process (MDP). By carefully designing state spaces, state-dependent action sets and time-dependant reward signals, we aim to help the agent learn an optimized trajectory and complete the flight task within time limit. We adapt the deep Q network (DQN) algorithm and extend it with prioritized experience replay (PER) and importance sampling method to ensure a stable and effective learning outcome. Simulation results validate the effectiveness of the learning algorithm and confirm that the agent is able to learn an effective policy with desirable and robust learning outcomes. Compared with benchmark schemes, our proposed scheme is shown to achieve considerable performance gain in secrecy throughput.

ISAC-enable mobility-aware multi-UAV placement for ultra-dense networks

This paper proposes a method for characterizing user mobility levels in multiple unmanned aerial vehicles (UAV)-assisted networks. In a dynamic environment with varying degrees of mobile ground users, optimizing the placement of UAVs is important in improving network throughput. Moreover, for integrated sensing and communication (ISAC) enabled UAVs, the resource allocation for sensing and communication relies on the dynamic nature of the network environment. To keep track of the changes in the environment UAVs are required to continuously update their trajectory, hence, characterizing the users’ mobility level is an important tool to improve the UAV trajectory optimization. In this paper, a mobility-aware resource allocation for joint sensing and communication is proposed. Our results demonstrate that the proposed algorithm can improve the resource allocation between sensing and communication in an ISAC-enabled UAV-assisted network.

UAV Control Method Combining Reptile Meta-Reinforcement Learning and Generative Adversarial Imitation Learning

Reinforcement learning (RL) is pivotal in empowering Unmanned Aerial Vehicles (UAVs) to navigate and make decisions efficiently and intelligently within complex and dynamic surroundings. Despite its significance, RL is hampered by inherent limitations such as low sample efficiency, restricted generalization capabilities, and a heavy reliance on the intricacies of reward function design. These challenges often render single-method RL approaches inadequate, particularly in the context of UAV operations where high costs and safety risks in real-world applications cannot be overlooked. To address these issues, this paper introduces a novel RL framework that synergistically integrates meta-learning and imitation learning. By leveraging the Reptile algorithm from meta-learning and Generative Adversarial Imitation Learning (GAIL), coupled with state normalization techniques for processing state data, this framework significantly enhances the model’s adaptability. It achieves this by identifying and leveraging commonalities across various tasks, allowing for swift adaptation to new challenges without the need for complex reward function designs. To ascertain the efficacy of this integrated approach, we conducted simulation experiments within both two-dimensional environments. The empirical results clearly indicate that our GAIL-enhanced Reptile method surpasses conventional single-method RL algorithms in terms of training efficiency. This evidence underscores the potential of combining meta-learning and imitation learning to surmount the traditional barriers faced by reinforcement learning in UAV trajectory planning and decision-making processes.

Optimization of Clustering and Trajectory for Minimizing Age of Information in Unmanned Aerial Vehicle-Assisted Mobile Edge Computing Network.

With the development of the Internet of Things (IoT) technology, massive amounts of sensor data in applications such as fire monitoring need to be transmitted to edge servers for timely processing. However, there is an energy-hole phenomenon in transmitting data only through terrestrial multi-hop networks. In this study, we focus on the data collection task in an unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) network, where a UAV is deployed as the mobile data collector for the ground sensor nodes (SNs) to ensure high information freshness. Meanwhile, the UAV is equipped with an edge server for data caching. We first establish a rigorous mathematical model in which the age of information (AoI) is used as a measure of information freshness, related to both the data collection time and the UAV's flight time. Then a mixed-integer non-convex optimization problem is formulated to minimize the peak AoI of the collected data. To solve the problem efficiently, we propose an iterative two-step algorithm named the AoI-minimized association and trajectory planning (AoI-MATP) algorithm. In each iteration, the optimal SN-collection point (CP) associations and CP locations for the parameter ε are first obtained by the affinity propagation clustering algorithm. The optimal UAV trajectory is found using an improved elite genetic algorithm. Simulation results show that based on the optimized ε, the AoI-MATP algorithm can achieve a balance between data collection time and flight time, reducing the peak AoI of the collected data.

Minimizing the searching time of multiple targets in uncertain environments with multiple UAVs

The focus of this paper is the use of Unmanned Aerial Vehicles (UAVs) for searching multiple targets under uncertain conditions in the minimal possible time. The problem, known as Minimum Time Search (MTS), belongs to the Probabilistic Search (PS) field and addresses critical missions, such as search & rescue, and military surveillance. These operations, characterized by complex and uncertain environments, demand efficient UAV trajectory optimization. The multi-target version of PS introduces additional challenges, due to their higher complexity and the need to wisely distribute the UAV’s efforts among multiple targets. In order to tackle the under-explored multi-target aspect of MTS, we optimize the time to find all targets with new Ant Colony Optimization (ACO)-based planner. This novel optimization criterion is formulated using Bayes’ theory, considering probability models of the targets (initial belief and motion model) and the sensor likelihood. Our work contributes significantly by (i) developing an objective function tailored for multi-target MTS, (ii) proposing an ACO-based planner designed to effectively handle the complexities of multiple moving targets, and (iii) introducing a novel constructive heuristic that is used by the ACO-based planner, specifically designed for the multi-target MTS problem. The efficacy of our approach is demonstrated through comprehensive analysis and validation across various scenarios, showing superior performance over existing methods in complex multi-target MTS problems.

A distributed containment tracking control of the UAV formation with prescribed performance subject to collision avoidance

SummaryIn this paper, the issue of finite‐time containment tracking control under collision avoidance constraints for unmanned aerial vehicle (UAV) formation is investigated. To tackle the challenge of collision avoidance in the context of formation containment tracking, combined with a prescribed performance function, a distributed containment tracking control scheme is proposed. Firstly, a UAV formation containment control model from a top view perspective is established. Next, to solve the internal collision avoidance within the formation, a restricted movement region is constructed for each UAV using a distributed prescribed performance function, which guarantees that each UAV's trajectory is maintained within a predefined area. Furthermore, invoking with the finite‐time control theory for the formation, a distributed containment tracking control scheme is developed to complete finite‐time tracking of the formation. Finally, simulation results are carried out to demonstrate the efficacy of the proposed approach.