Movement Of Unmanned Aerial Vehicles Research Articles

UAV maneuver decision-making via deep reinforcement learning for short-range air combat

The unmanned aerial vehicle (UAV) has been applied in unmanned air combat because of its flexibility and practicality. The short-range air combat situation is rapidly changing, and the UAV has to make the autonomous maneuver decision as quickly as possible. In this paper, a type of short-range air combat maneuver decision method based on deep reinforcement learning is proposed. Firstly, the combat environment, including UAV motion model and the position and velocity relationships, is described. On this basic, the combat process is established. Secondly, some improved points based on proximal policy optimization (PPO) are proposed to enhance the maneuver decision-making ability. The gate recurrent unit (GRU) can help PPO make decisions with continuous timestep data. The actor network's input is the observation of UAV, however, the input of the critic network, named state, includes the blood values which cannot be observed directly. In addition, the action space with 15 basic actions and well-designed reward function are proposed to combine the air combat environment and PPO. In particular, the reward function is divided into dense reward, event reward and end-game reward to ensure the training feasibility. The training process is composed of three phases to shorten the training time. Finally, the designed maneuver decision method is verified through the ablation study and confrontment tests. The results show that the UAV with the proposed maneuver decision method can obtain an effective action policy to make a more flexible decision in air combat.

Energy Efficient Path Planning for 3D Aerial Inspections

The use of Unmanned Aerial Vehicles (UAVs) in different inspection tasks is increasing. This technology reduces inspection costs and collects high quality data of distinct structures, including areas that are not easily accessible by human operators. However, the reduced energy available on the UAVs limits their flight endurance. To increase the autonomy of a single flight, it is important to optimize the path to be performed by the UAV, in terms of energy loss. Therefore, this work presents a novel formulation of the Travelling Salesman Problem (TSP) and a path planning algorithm that uses a UAV energy model to solve this optimization problem. The novel TSP formulation is defined as Asymmetric Travelling Salesman Problem with Precedence Loss (ATSP-PL), where the cost of moving the UAV depends on the previous position. The energy model relates each UAV movement with its energy consumption, while the path planning algorithm is focused on minimizing the energy loss of the UAV, ensuring that the structure is fully covered. The developed algorithm was tested in both simulated and real scenarios. The simulated experiments were performed with realistic models of wind turbines and a UAV, whereas the real experiments were performed with a real UAV and an illumination tower. The inspection paths generated presented improvements over 24% and 8%, when compared with other methods, for the simulated and real experiments, respectively, optimizing the energy consumption of the UAV.

Utilizing Deep Reinforcement Learning to Control UAV Movement for Environmental Monitoring

Unmanned aerial vehicles (UAVs) are increasingly used in various applications, including infrastructure inspection, traffic monitoring, remote sensing, mapping, and rescue. However, many applications have required UAVs to function autonomously, without human intervention to improve system performance. In this study, we propose a new approach to environmental monitoring using a group of UAVs equipped with sensors under the support of reinforcement learning. Regarding the communication system model, we assume that UAVs can cooperate with each other to learn and share information about the environment, and then relocate to an optimal position while managing connectivity and coverage. After that, we exploit reinforcement learning with a deep deterministic policy gradient (DDPG) algorithm to optimize environmental monitoring with the proposed algorithm. Specifically, the proposed algorithm aims to simulate an environmental monitoring system using UAVs with basic parameters. We further apply the proposed algorithm to evaluate network performance under different parameter settings. Numerical results validate the effectiveness of the proposed learning-based framework in monitoring and sensing data.

РАЗРАБОТКА АЛГОРИТМА УПРАВЛЕНИЯ ГРУППОЙ БЕСПИЛОТНЫХ ЛЕТАТЕЛЬНЫХ АППАРАТОВ САМОЛЕТНОГО ТИПА

Relevance Development of an algorithm for controlling a group of aircraft-type unmanned aerial vehicles (UAVs). Aim of research To improve the efficiency of using UAVs by controlling movement in dense group formations. The objectives of the research project are: 1. Analysis of mathematical models of UAV movement in a group; 2. Synthesis of an algorithm for coordinating autonomous UAVs in a group. Research methods The principles and methods of system analysis, theories of linear and nonlinear automatic control systems, linear algebra, functional analysis, and computer modeling. Results The algorithm for controlling a group of aircraft-type UAVs is distinguished by the fact that it ensures coordination of intra-system interactions between group members by transferring a vector of state variables to a given area of discrete space in one control cycle.

АППАРАТНО-АЛГОРИТМИЧЕСКОЕ ОБЕСПЕЧЕНИЕ ПЕРСПЕКТИВНОЙ СИСТЕМЫ ЭНЕРГОСБЕРЕЖЕНИЯ АВТОНОМНОЙ ГРУППЫ БПЛА

Due to the accelerated growth in the use of groups of autonomously functioning unmanned aerialvehicles (UAVs) in various environments, solving the problem of optimizing the functioning of groups ofsuch vehicles according to the criterion of the minimum energy consumed is an urgent scientific task. Inthis article, a new approach is being developed to ensure energy saving of a group of unmanned aerialvehicles (UAVs) by using UAV charging stations that provide the necessary versatility in servicing differenttypes of vehicles. The most effective variants of landing systems, precise positioning, power generationat the station are considered, and a hybrid charging system combining contact and contactless methods is also justified. A generalized scheme of multi-stage interaction of the UAV with the chargingstation is given, which provides for the possibility of repeating one of the stages if it is not fully completedwithin certain time intervals, and also takes into account the peculiarities of interaction betweenagents through the communication channels available to them. The problem of optimal distributionaccording to the energy criterion of the group's UAVs between destinations (PN) combined with chargingstations (SP) has been set and solved. Both the cruising mode and the maneuvering of the vehicleduring takeoff and landing, when the power consumption increases, are taken into account. The conceptof the effective distance to the destination is introduced, taking into account the estimated energy costsof each UAV to reach this destination, taking into account its arbitrary current position and the availablequeue of tasks at the moment. To study the developed approaches and algorithms for targeting andplanning the movement of the group's UAVs, software based on the Unity environment was created andtested. The flexibility of the latter allows modeling various algorithms of information interactionsof elements within a group of UAVs, a group of charging stations, as well as cross-interactionsbetween UAVs and charging stations. In particular, the software allows you to determine at eachdiscrete moment the degree of charge of each UAV, the queues of destinations for each UAV, itshistory of recharge at stations.

Source Direction Finding and Direct Localization Exploiting UAV Array With Unknown Gain-Phase Errors

An unmanned aerial vehicle (UAV) array is composed of multiple UAVs carrying array elements, which can sense signals synchronously through a global positioning system (GPS) trigger. However, gain-phase errors caused by synchronization errors and the inconsistent complex gains of receiving array channels result in array manifold perturbation, which makes the performance of traditional localization methods degrade or even fail. In this article, two efficient algorithms for source direction finding and direct localization using a UAV array are, respectively, presented. First, the array manifold changes with the movement of UAVs, which contributes to multiposition fusion, thus avoiding the infinite solutions of underdetermined equations. Then, the quadratic optimization problem can be constructed by using the orthogonal relation between noise subspaces and contaminated steering vectors obtained from multiple observations. Thereafter, we can construct the cost function and obtain the spectral function, from which the source directions and positions can be all estimated by grid search. Meanwhile, the gain-phase error values can be solved successively. Moreover, in order to avoid the influence of heteroscedasticity of different observation positions in direct localization, we carry out a blind weighting operation. Simulation results demonstrate the effectiveness and superiority of the proposed algorithms.

Voronoi-Based UAV Flight Method for Non-Uniform User Distribution in Delay-Tolerant Aerial Networks

This paper considers an emergency communication system controlling multiple unmanned aerial vehicles (UAVs) in the sky over a large-scale disaster-affected area. This system is based on delay-tolerant networking, and information from ground users is relayed by the UAVs through wireless transmission and the movement of UAVs in a store-and-forward manner. Each UAV moves autonomously according to a predetermined flight method, which uses the positions of other UAVs through communication. In this paper, we propose a new method for UAV flight considering the non-uniformity of user distributions. The method is based on the Voronoi cell using the predicted locations of other UAVs. We evaluate the performance of the proposed method through computer simulations with a non-uniform user distribution generated by a general cluster point process. The simulation results demonstrate the effectiveness of the proposed method.

Modeling of Object Monitoring Using 3D Cellular Automata

The purpose of this work is to develop a formal model for describing the properties of the environment for the functioning of unmanned aerial vehicles and to increase the speed of calculating the trajectory of their flight when monitoring critical infrastructure objects based on the mathematical apparatus of 3D cellular automata. This goal is achieved by solving the following problems: developing a method for describing the operating environment of unmanned aerial vehicles based on 3D cellular automata, developing a method for calculating the flight path of unmanned aerial vehicles. The construction of formal models is based on the apparatus of 3D cellular automata. The most significant results are a formalized description of the space, properties of zones and objects that restrict movement, as well as the development of a method for modeling the flight of an unmanned aerial vehicle in space when solving the monitoring problem, which will increase the speed of calculating the flight path. The significance of the results obtained lies in solving the complex problem of calculating the trajectory of movement of unmanned aerial vehicles for monitoring critical infrastructure objects using the apparatus of 3D cellular automata. The conducted studies have shown the effectiveness of using 3D - cellular automata to solve the problems of finding flight paths when monitoring critical infrastructure objects in various conditions. The proposed approach to the implementation of cellular automata will allow creating an effective monitoring system.

Energy-Aware Blockchain and Federated Learning-Supported Vehicular Networks

The aerial capabilities and flexibility in movement of Unmanned Aerial Vehicles (UAVs) has enabled them to adaptively provide both traditional and more contemporary services. In this article, we introduce a solution that integrates the capabilities of both UAVs and Unmanned Ground Vehicles (UGVs) to provide both intelligent connectivity and services to both aerial and ground connected devices. A cooperative solution is adopted that considers nodes’ power and movement constraints. The UAV and UGV cooperative process ensures continuous power availability to UAVs to support seamless and continuous service availability to end-devices. A Federated Learning (FL) approach is adopted at the edge to ensure accurate and up-to-date service provisioning in accordance with the surrounding environment and network constraints. Moreover, Blockchain technology is used to decentralize the provisioning and control aspects, and ensure authenticity and integrity. Extensive simulations are conducted to test the soundness and applicability of the proposed solution. Results show significant improvement in terms of connectivity, service availability, and UAV energy enhancements when compared to traditional mobile and vehicular communication techniques.

Dynamic Noncontiguous Location-Aware Spectrum Aggregation for UAV-to-UAV Communications

Limited spectrum access and complex resource allocation have impeded the cost-effective and robust deployment of large groups of noncellular unmanned aerial vehicles (UAVs) for new use cases in extreme conditions under dynamic spectrum management requirements. This letter introduces a novel design for UAV-to-UAV communications that relies on joint noncontiguous spectrum aggregation and allocation of sub-6 GHz and millimeter wave (mmWave) bands. To achieve bandwidth efficiency and reduced complexity, the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">location-aware waveform design</i> utilizes target sensing/positioning information in the aggregation as well as spectrum fragment allocation for UAV-based joint sensing and communications. The proposed method, which relies on the prediction of the position and direction of motion of the network UAVs, utilizes a delay-tolerant observer-based estimator–predictor in the presence of uncertainties. The performance of the proposed location-aware spectrum aggregation is validated using numerical simulations.

Autonomous Maneuver Decision of Air Combat Based on Simulated Operation Command and FRV-DDPG Algorithm

With the improvement of UAV performance and intelligence in recent years, it is particularly important for unmanned aerial vehicles (UAVs) to improve the ability of autonomous air combat. Aiming to solve the problem of how to improve the autonomous air combat maneuver decision ability of UAVs so that it can be close to manual manipulation, this paper proposes an autonomous air combat maneuvering decision method based on the combination of simulated operation command and the final reward value deep deterministic policy gradient (FRV-DDPG) algorithm. Firstly, the six-degree-of-freedom (6-DOF) model is established based on the air combat process, UAV motion, and missile motion. Secondly, a prediction method based on the Particle swarm optimization radial basis function (PSO-RBF) is designed to simulate the operation command of the enemy aircraft, which makes the training process more realistic, and then an improved DDPG strategy is proposed, which returns the final reward value to the previous reward value in a certain proportion of time for offline training, which can improve the convergence speed of the algorithm. Finally, the effectiveness of the algorithm is verified by building a simulation environment. The simulation results show that the algorithm can improve the autonomous air combat maneuver decision-making ability of UAVs.

Physical Modeling, Simulation and Validation of Small Fixed-Wing UAV

The flight dynamics of small Unmanned Aerial Vehicles (UAVs) exhibits substantial nonlinear features which should not be ignored in simulation or analysis. In this work, a complete six degrees of freedom (6DOF) and high-fidelity simulation model of a case study of a small fixed-wing UAV is developed. To accurately estimate the nonlinear UAV’s mathematical model, the mass-inertia properties are investigated through experiments. Moreover, the aerodynamic model characteristics are estimated using a semi-empirical technique to estimate the aerodynamic coefficients and derivatives. The propulsion system’s physical characteristics are estimated and analyzed through experimental measurements. The International Standard Atmosphere (ISA) tables are used to develop the atmospheric model. Furthermore, the actuation model is estimated and validated experimentally through system identification techniques. These estimated sub-model data are integrated to complete the nonlinear flight dynamics model in MATLAB® (Simulink). The UAV trim calculations are graphically derived and then the developed model is tested and verified. The open loop simulation results have proved the superiority of the utilized model techniques to describe the UAV motion. Moreover, it gives very optimistic results to test the flight dynamics of the UAV and design a consistent flight control and guidance system.

Дослідження взаємного впливу між каналами управління квадрокоптером за рахунок малої приводності БПЛА

The subject of study in this article is the processes of controlling an unmanned aerial vehicle (UAV) of the quadcopter type. A quadcopter is a special case of a multicopter built according to the helicopter scheme, which has four main rotors. Such aircraft are widely used for many purposes, both civilian and military: from video recording of any phenomena to performing aerial reconnaissance of inaccessible territories, adjusting artillery weapons. The quadcopter belongs to the class of mechanical systems with low driveability, as the number of its drives (four propeller motors) is less than the number of degrees of freedom (six degrees). This is significantly different from the control of an aircraft-type UAV, where each degree of freedom is controlled by its own actuator. The low driveability of the quadcopter imposes its own peculiarities during its control. For example, the horizontal movement of the quadcopter in a given direction is accomplished by tilting the quadcopter in that direction by influencing certain propeller motors. Thus, when demanding the execution of the simplest movement of the device (movement along the linear X coordinate), at least three coordinates are controlled by the executive motors of the quadcopter: the formation of a lifting force to control the hovering of the UAV at a certain height, the change of the angle of inclination of the quadcopter according to the pitch angle, and the direct control of the movement speed of the quadcopter. Simultaneously, this influence induces the emergence of movement conditions along other coordinates. The research method involves constructing a mathematical model of a quadcopter based on the known equations of UAV motion. Several methods and algorithms of quadrocopter control are known, in this work, PID coordinate controllers are used. The Matlab Simulink dynamic simulation environment was used to build the mathematical model. In the process of studying the mutual influence in the process of controlling coordinates, the influence of controlling one coordinate in the static state of others (not necessarily at rest, but also in the state of movement) was revealed. Further, the mutual influence between the degrees of freedom during simultaneous control of at least two coordinates is considered (for example, control of pitch angle and yaw angle, height change with simultaneous braking along one of the linear coordinates). Conclusions. Studies have confirmed the existence of mutual influences, although in most cases such influence is manifested by a slight deterioration in the quality of transient processes along the adjustable coordinates. Only the change of the Euler angles when changing the height significantly worsens the quality of transient processes, which must be considered when designing control systems or recording algorithms of movement by coordinates.

A Data-Driven Packet Routing Algorithm for an Unmanned Aerial Vehicle Swarm: A Multi-Agent Reinforcement Learning Approach

Routing decisions made by unmanned aerial vehicle (UAV) swarms are affected by complex and dynamically changing topologies. A centralized routing algorithm imposes the entire computational burden on one module, and the high data dimensionality renders computation burdensome. In this letter, we develop a multi-agent reinforcement learning-based routing algorithm for a UAV swarm. The UAVs are trained in a data-driven manner to make distributed routing decisions. Factors that include channel quality, UAV movement, UAV overhead, and the extent of neighbor variation are incorporated into link quality assessment. Long short-term memory is used to improve the Actor and Critic networks, and more information on temporal continuity is added to facilitate adaptation to the dynamically changing environment. Simulations show that the proposed routing algorithm reduces data transmission delay and enhances the transmission rate compared with traditional routing algorithms.

Modeling and Flight Control of Small UAV with Active Morphing Wings

In recent research works, morphing wings were studied as an interesting field for a small unmanned aerial vehicle (UAV). The previous studies either focused on selecting suitable material for the morphing wings or performing experimental tests on UAVs with morphing wings. Though, the dynamic modeling of active flexible morphing wings and their involved interactions with the aerodynamics of the UAV body are challenging subjects. Using such a model to control a small UAV to perform specific maneuvering is not investigated yet. The dynamic model of UAV with active morphing wings generates a multi-input multi-output (MIMO) system which rises the difficulty of the control system design. In this paper, the aeroelastic dynamic model of morphing wing activated by piezocomposite actuators is established using the finite element method and modal decomposition technique. Then, the dynamic model of the UAV is developed taking into consideration the coupling between the wing and piezocomposite actuators, as well as the dynamic properties of the morphing actuators with the aerodynamic wind disturbances. A model predictive control (MPC) is designed for the MIMO control system to perform specific flight maneuvering by tracking desired trajectories of UAV altitude and yaw angle. Additionally, the MPC achieves constrained behavior of pitch and roll angles to get satisfactory UAV motion. Also, the behaviors of the UAV control system using MPC are evaluated after adding Dryden wind turbulence to the UAV outputs. Finally, a UAV flight simulation is conducted which shows that the control system successfully rejects the applied disturbances and tracks the reference trajectories with acceptable behavior of pitch and roll angles.

Comprehensive Review on UAV Efficient Path Planning Techniques for Optimized Applications

This literature review article compiles works that describe the use of bio-inspired algorithms in Unmanned Aerial Vehicle (UAV) motion planning. This review demonstrates the usefulness of the various frameworks by presenting the contributions and limits of each article. The optimization method also decreases the amount of inaccuracy in the system’s convergence. Furthermore, this study discusses the assessment procedures and draws attention to the novelties and limitations of the explored methods. The paper wraps up with a detailed examination of the current difficulties and potential future research directions. This research will aid scholars in comprehending the state-of-the-art efforts made in UAV motion planning using a variety of optimization strategies.

Adaptive neural-network-based distributed fault estimation for heterogeneous multi-agent systems

This contribution addresses the issue of distributed fault estimation for heterogeneous multi-agent systems which are composed of unmanned ground vehicles and unmanned aerial vehicles in the presence of actuator faults, completely unknown nonlinearities and external disturbances. Given that these two types of agents have different state dimensions and the motion of unmanned aerial vehicles in the XOY plane and Z-axis is relatively independent, the heterogeneous multi-agent systems can be divided into the XOY plane of all agents’ position subsystem and the Z-axis of unmanned aerial vehicles’ position subsystem. Then, combining the influences of completely unknown nonlinearities and external disturbances, an adaptive neural-network-based distributed fault estimation scheme is proposed to effectively estimate unknown actuation effectiveness parameters and can be applied to XOY plane and Z-axis of heterogeneous multi-agent systems separately. During the design of the observer, the neural network methodology is adopted to approximate completely unknown nonlinearities and a proper adaptive update law to estimate the 2-norm upper bound of disturbances and compensate for the influences of disturbances is designed. With output from a local agent and its neighbors, the proposed observer can be built on this agent, realizing simultaneous estimation of possible faults occurring in both the selected agent and its neighbor agents, which presents a new distributed framework. At last, simulation results are shown to illustrate the feasibility and effectiveness of the presented fault estimation algorithm.

Clustering Algorithm Based on the Ground-Air Cooperative Architecture in Border Patrol Scenarios

The border security situation is complex and severe, and the border patrol system relying on the ground-air cooperative architecture has been paid attention to by all countries as an important means of protecting national security. In the flying ad-hoc network (FANET), under the ground-air cooperative architecture, an unmanned aerial vehicle (UAV) uses a patrol mobility model to improve patrol efficiency. Since the patrol mobility model leads to frequent changes in UAV movement direction to improve patrol efficiency, selecting some clustering utility factors and calculating utility factors in previous clustering algorithms do not apply to this scenario. To solve the above problems, in this paper, we propose a border patrol clustering algorithm (BPCA) based on the ground-air cooperative architecture, which is based on the existing weighted clustering algorithm and improved in terms of the selection of utility factors and calculations of utility factors in cluster head selection. This algorithm comprehensively considers the effects of relative speed, relative distance, and the movement model of the UAV on the network topology. Extensive simulation results show that this algorithm can extend the duration time of cluster heads and cluster members and improve the stability of clusters and the reliability of links.

Deep Learning for Channel Tracking in IRS-Assisted UAV Communication Systems

To boost the performance of wireless communication networks, unmanned aerial vehicles (UAVs) aided communications have drawn dramatically attention due to their flexibility in establishing the line of sight (LoS) communications. However, with the blockage in the complex urban environment, and due to the movement of UAVs and mobile users, the directional paths can be occasionally blocked by trees and high-rise buildings. Intelligent reflection surfaces (IRSs) that can reflect signals to generate virtual LoS paths are capable of providing stable communications and serving wider coverage. This is the first paper that exploits a three-dimensional geometry dynamic channel model in IRS- assisted UAV-enabled communication system. Moreover, we develop a novel deep learning based channel tracking algorithm consisting of two modules: channel pre-estimation and channel tracking. A deep neural network with off-line training is designed for denoising in the pre-estimation module. Moreover, for channel tracking, a stacked bi-directional long short term memory (Stacked Bi-LSTM) is developed based on a framework that can trace back historical time sequence together with bidirectional structure over multiple stacked layers. Simulations have shown that the proposed channel tracking algorithm requires fewer epochs to convergence compared to benchmark algorithms. It also demonstrates that the proposed algorithm is superior to different benchmarks with small pilot overheads and comparable computation complexity.

Cross-Layer and Energy-Aware AODV Routing Protocol for Flying Ad-Hoc Networks

In recent years, unmanned aerial vehicles (UAVs) have become the trend for different types of research and applications. UAVs can accomplish some technical and risky tasks while still being safe, mobile, and inexpensive to operate. However, UAVs need flying ad-hoc networks (FANET) to operate in inaccessible or infrastructure-less areas. Subsequently, in many military and civil applications, the UAVs are connected ad hoc. FANET-based UAV systems have been developed for search and rescue, wildlife surveys, real-time monitoring, and delivery services. Maintaining the reliability and connectivity among UAV nodes in FANET becomes challenging because of the UAV movement, environmental conditions, energy efficiency, etc. Energy-aware routing protocols have become essential for developing advanced and effective FANETs. This paper presents a proposed Cross-Layer and Energy-Aware Ad-hoc On-demand Distance Vector (CLEA-AODV) routing protocol for improving FANET performance. The CLEA-AODV protocol is mainly divided into three sections: routing with AODV protocol, Glow Swarm Optimization (GSO)-based Cluster Head Selection, and Cooperative Medium Access Control (MAC). The cross-layer approach is implemented on the network layer and the data layer. The major parameters considered to evaluate the performance of the FANET are Packet Success Rate (PSR), Throughput (TP), End-to-End (E2E) delay, and packet drop ratio (PDR). The Network Simulator version 2 (NS2) is used to implement the CLEA-AODV protocol and evaluate the network performance. The results are compared with the standard AODV, Self-Organization Clustering-GSO (SOC-GSO), and Energy Efficient Neuro-Fuzzy Cluster-based Topology Construction with Meta-Heuristic Route Planning (EENFC-MRP) protocols. The results show that the CLEA-AODV surpasses these protocols in terms of PSR, TP, E2E delay, and PDR.