Combat Maneuvering Research Articles

Stabilizing a Bio-Inspired Rotating Empennage Fighter Aircraft in Multiple Trim Scenarios

This paper considers the problem of stabilizing a bio-inspired fighter aircraft at its Air Combat Maneuver Condition in steady-level and coordinated-turning flight. The aircraft equations of motion are linearized, and an infinite-horizon linear quadratic regulator design is conducted. The open-loop system is unstable in the short-period and Dutch roll modes. This is mitigated in the closed-loop system, which is analyzed in the time and frequency domains. Included in the simulation dynamics are first-order actuator models, actuator deflection limits, and actuator rate limits. These are particularly important for this bio-inspired aircraft because control actuation requires rotation of the empennage, which has relatively large inertia. Simulation responses to initial condition dispersions, aerodynamic model error, and atmospheric turbulence are analyzed to characterize time-domain properties: settling time, region of attraction, control saturation, and robustness. Due to poor singular values for the throttle setting and rotating tail inputs, analysis is dedicated to control designs with these inputs fixed at trim values. Within the scope of these analyses, fixing the rotating tail at trim does not significantly degrade system performance.

Autonomous Maneuvering Decision-Making Algorithm for Unmanned Aerial Vehicles Based on Node Clustering and Deep Deterministic Policy Gradient

Decision-making for autonomous maneuvering in dynamic, uncertain, and nonlinear environments represents a challenging frontier problem. Deep deterministic policy gradient (DDPG) is an effective method to solve such problems, but it is found that complex strategies require extensive computation and time in the learning process. To address this issue, we propose a node clustering (NC) method, inspired by grid clustering, integrated into the DDPG algorithm for the learning of complex strategies. In the NC method, the node membership degree is defined according to the specific characteristics of the maneuvering decision-making problem, and error handling strategies are designed to reduce the number of transitions in the replay database effectively, ensuring that the most typical transitions are retained. Then, combining NC and DDPG, an autonomous learning and decision-making algorithm of maneuvering is designed. The algorithm flow and the pseudo-code of the algorithm are given. Finally, the NC_DDPG algorithm is applied to a typical short-range air combat maneuvering decision problem for verification. The results show that the NC_DDPG algorithm significantly accelerates the autonomous learning and decision-making process under both balanced and disadvantageous conditions, taking only about 77% of the time required by Vector DDPG. The scale of NC impacts learning speed; the simulation results across five scales indicate that smaller clustering scales significantly increase learning time, despite a high degree of randomness. Compared with Twin Delayed DDPG (TD3), NC_DDPG consumes only 0.58% of the time of traditional TD3. After applying the NC method to TD3, NC_DDPG requires approximately 20–30% of the time of NC_TD3.

KAPABILITAS INSTITUSI MILITER DALAM MENGATASI PELANGGARAN UDARA: STUDI PADA LANUD ROESMIN NURJADIN DALAM PENGGUNAAN AIR COMBAT MANEUVERING INSTRUMENTATION (ACMI) WING 6

The capability of Roesmin Nurjadin Air Force Base in utilizing Air Combat Maneuvering Instrumentation (ACMI) Wing 6 plays a crucial role in addressing violations of national airspace. This research aims to explore the capabilities of Lanud Roesmin Nurjadin in utilizing ACMI Wing 6 and to identify efforts that can be made to enhance them. The research problem includes the operational capabilities of ACMI Wing 6 and the challenges in addressing violations of national airspace. Factors such as technical limitations, personnel training, and supporting infrastructure are the main focus of the analysis. The research method used is qualitative, characterized by descriptive analysis. This research obtained data through interviews with several relevant informants, documentation studies, internet-based studies, and archival studies aimed at collecting data related to the capabilities of Lanud Roesmin Nurjadin in utilizing ACMI Wing 6. The research results indicate that Roesmin Nurjadin Air Base has sufficient operational capability in using ACMI Wing 6. However, there are still several technical challenges, personnel training issues, and supporting infrastructure that need to be addressed. This study finds that Roesmin Nurjadin Air Base has the potential to enhance its capabilities in using ACMI Wing 6 by improving supporting infrastructure, increasing personnel training, and overcoming technical obstacles. Although the ACMI operator personnel have good technical skills, there is a need to enhance conceptual analysis abilities, especially regarding responses to violations of national airspace. The teamwork among ACMI personnel is very good, demonstrating readiness to face emergency situations and complex airspace violations

UAV swarm air combat maneuver decision-making method based on multi-agent reinforcement learning and transferring

UAV swarm air combat maneuver decision-making method based on multi-agent reinforcement learning and transferring

Air combat maneuver decision based on deep reinforcement learning with auxiliary reward

Air combat maneuver decision based on deep reinforcement learning with auxiliary reward

Continuous Action Air Combat Maneuver Decision-Making Based on T-MGMM

Continuous Action Air Combat Maneuver Decision-Making Based on T-MGMM

Deep reinforcement learning-based air combat maneuver decision-making: literature review, implementation tutorial and future direction

Deep reinforcement learning-based air combat maneuver decision-making: literature review, implementation tutorial and future direction

An air combat maneuver pattern extraction based on time series segmentation and clustering analysis

Target maneuver recognition is a prerequisite for air combat situation awareness, trajectory prediction, threat assessment and maneuver decision. To get rid of the dependence of the current target maneuver recognition method on empirical criteria and sample data, and automatically and adaptively complete the task of extracting the target maneuver pattern, in this paper, an air combat maneuver pattern extraction based on time series segmentation and clustering analysis is proposed by combining autoencoder, G-G clustering algorithm and the selective ensemble clustering analysis algorithm. Firstly, the autoencoder is used to extract key features of maneuvering trajectory to remove the impacts of redundant variables and reduce the data dimension; Then, taking the time information into account, the segmentation of Maneuver characteristic time series is realized with the improved FSTS-AEGG algorithm, and a large number of maneuver primitives are extracted; Finally, the maneuver primitives are grouped into some categories by using the selective ensemble multiple time series clustering algorithm, which can prove that each class represents a maneuver action. The maneuver pattern extraction method is applied to small scale air combat trajectory and can recognize and correctly partition at least 71.3% of maneuver actions，indicating that the method is effective and satisfies the requirements for engineering accuracy. In addition, this method can provide data support for various target maneuvering recognition methods proposed in the literature, greatly reduce the workload and improve the recognition accuracy.

Beyond visual range maneuver intention recognition based on attention enhanced tuna swarm optimization parallel BiGRU

This paper researches the problem of Beyond Visual Range (BVR) air combat maneuver intention recognition. To achieve efficient and accurate intention recognition, an Attention enhanced Tuna Swarm Optimization-Parallel Bidirectional Gated Recurrent Unit network (A-TSO-PBiGRU) is proposed, which constructs a novel Parallel BiGRU (PBiGRU). Firstly, PBiGRU has a parallel network structure, whose proportion of forward and backward network can be adjusted by forward coefficient and backward coefficient. Secondly, to achieve object-oriented adjustment of forward and backward coefficients, the tuna swarm optimization algorithm is introduced and the negative log-likelihood estimation loss function is used as the objective function, it realizes the dynamic combination of sequence guidance and reverse correction. Finally, the attention mechanism is used to obtain more useful information to improve the recognition accuracy. Through offline recognition experiment, it is proved that A-TSO-PBiGRU can effectively improve the convergence speed and recognition accuracy compared with GRU-related networks. Compared with the other six comparison algorithms, maneuver intention recognition accuracy also has significant advantages. In the online recognition experiment, maneuver intention recognition accuracy of A-TSO-PBiGRU is 93.7%, it shows excellent maneuver intention recognition ability.

UCAV situation assessment method based on C-LSHADE-Means and SAE-LVQ

The unmanned combat aerial vehicle (UCAV) is a research hot issue in the world, and the situation assessment is an important part of it. To overcome shortcomings of the existing situation assessment methods, such as low accuracy and strong dependence on prior knowledge, a data-driven situation assessment method is proposed. The clustering and classification are combined, the former is used to mine situational knowledge, and the latter is used to realize rapid assessment. Angle evaluation factor and distance evaluation factor are proposed to transform multi-dimensional air combat information into two-dimensional features. A convolution success-history based adaptive differential evolution with linear population size reduction-means (C-LSHADE-Means) algorithm is proposed. The convolutional pooling layer is used to compress the size of data and preserve the distribution characteristics. The LSHADE algorithm is used to initialize the center of the mean clustering, which overcomes the defect of initialization sensitivity. Comparing experiment with the seven clustering algorithms is done on the UCI data set, through four clustering indexes, and it proves that the method proposed in this paper has better clustering performance. A situation assessment model based on stacked autoencoder and learning vector quantization (SAE-LVQ) network is constructed, and it uses SAE to reconstruct air combat data features, and uses the self-competition layer of the LVQ to achieve efficient classification. Compared with the five kinds of assessments models, the SAE-LVQ model has the highest accuracy. Finally, three kinds of confrontation processes from air combat maneuvering instrumentation (ACMI) are selected, and the model in this paper is used for situation assessment. The assessment results are in line with the actual situation.

Short-range air combat maneuver decision of UAV swarm based on multi-agent Transformer introducing virtual objects

With the development of Unmanned Aerial Vehicle (UAV) swarm technology, there has been a growing interest in using Artificial Intelligence (AI) to drive UAV swarms for short-range air combat. However, due to the complexity of situation information in UAV swarm air combat, making accurate air combat decisions based on air combat situation information has become a challenge. In this paper, we propose the multi-agent Transformer introducing virtual objects (MTVO) to address this issue. First, the proposed approach designs a multi-agent Transformer network structure by exploiting the homogeneity feature of the UAV state information in the swarm. This structure enables the structured processing of complex situation information. Specifically, the local situation information of each UAV is calculated by self-attention, which reduces the size of the swarm situation information while retaining the information of key UAVs. This approach reduces the difficulty of processing UAV swarm situation information. Moreover, we add a virtual object to the UAV swarm information to assist in calculating the weight distribution of the local situation. The weighted fusion of local situations allows us to obtain a more effective representation of the global situation, which serves as the basis for more accurate air combat maneuver decisions. We demonstrate the performance of the proposed method through air combat simulation results using Reinforcement Learning (RL) methods, and validate the applicability and effectiveness of the MTVO network for the short-range air combat problem of UAV swarms.

A Game-Theoretic Model for One-on-One Air Combat

The primary goal of air combat maneuvering is to neutralize the enemy under any circumstances. A critical step in achieving this goal is to gain and maintain aerial advantage by choosing and executing appropriate flighter maneuvers. In this paper, one-on-one air combat is modeled as a two-player zero-sum game to obtain optimal strategies for the players to select the flighter maneuvers. A new approach for constructing the game payoff function that accounts for the loss/gain of aerial advantage during the execution of a maneuver is proposed. A case study to evaluate the proposed approach, by comparing it with an existing approach, is performed through computer simulations.

Hierarchical reinforcement learning from competitive self-play for dual-aircraft formation air combat

AbstractThe recent development of technology helps in the revolutionary war and it controls the war which is influenced by brilliant planning. The maneuver aircraft of intelligent algorithm aids the pilot to decide the particular position on the battlefield. Nowadays the hardware components of radar and missiles are widely used and the beyond visual range is the most popular method applied in air combat. The introduction of close-range air combat maneuver decisions generates the attention of researchers in artificial intelligence. Most of the existing methods are based on autonomous aircraft focused in air combat scenario but manual air combats are widely applied in dual aircraft. Based on the factors mentioned above, a novel hierarchical maneuver decision architecture is applied to a dual-aircraft close-range air combat scenario. Subsequently, the soft actor-critic algorithm is merged with competitive self-play which integrates the knowledge of sub-strategies. Further, the reinforcement learning technique is employed to achieve an approximate Nash equilibrium master strategy. The experimental results show that the hierarchical architecture exhibits good performance, symmetry, and robustness. The research generates a solution for intelligent formation of air combat in the future and guidance for manned or unmanned aircraft cooperative combat.

UAV's air combat decision-making based on deep deterministic policy gradient and prediction

To solve the enemy uncertain manipulation problem during a UAV's autonomous air combat maneuver decision-making, this paper proposes an autonomous air combat maneuver decision-making method that combines target maneuver command prediction with the deep deterministic policy algorithm. The situation data of both sides of air combat are effectively fused and processed, the UAV's six-degree-of-freedom model and maneuver library are built. In air combat, the target generates its corresponding maneuver library instructions through the deep Q network algorithm; at the same time, the UAV on our side gives the target maneuver prediction results through the probabilistic neural network. A deep deterministic policy gradient reinforcement learning method that considers both the situation information of two aircraft and the prediction results of enemy aircraft is proposed, so that the UAV can choose the appropriate maneuver decision according to the current air combat situation. The simulation results show that the method can effectively use the air combat situation information and target maneuver prediction information so that it can improve the effectiveness of the reinforcement learning method for UAV's autonomous air combat decision-making on the premise of ensuring convergence.

UAV maneuver decision-making via deep reinforcement learning for short-range air combat

The unmanned aerial vehicle (UAV) has been applied in unmanned air combat because of its flexibility and practicality. The short-range air combat situation is rapidly changing, and the UAV has to make the autonomous maneuver decision as quickly as possible. In this paper, a type of short-range air combat maneuver decision method based on deep reinforcement learning is proposed. Firstly, the combat environment, including UAV motion model and the position and velocity relationships, is described. On this basic, the combat process is established. Secondly, some improved points based on proximal policy optimization (PPO) are proposed to enhance the maneuver decision-making ability. The gate recurrent unit (GRU) can help PPO make decisions with continuous timestep data. The actor network's input is the observation of UAV, however, the input of the critic network, named state, includes the blood values which cannot be observed directly. In addition, the action space with 15 basic actions and well-designed reward function are proposed to combine the air combat environment and PPO. In particular, the reward function is divided into dense reward, event reward and end-game reward to ensure the training feasibility. The training process is composed of three phases to shorten the training time. Finally, the designed maneuver decision method is verified through the ablation study and confrontment tests. The results show that the UAV with the proposed maneuver decision method can obtain an effective action policy to make a more flexible decision in air combat.

Air Combat Maneuver Decision Method Based on A3C Deep Reinforcement Learning

To solve the maneuvering decision problem in air combat of unmanned combat aircraft vehicles (UCAVs), in this paper, an autonomous maneuver decision method is proposed for a UCAV based on deep reinforcement learning. Firstly, the UCAV flight maneuver model and maneuver library of both opposing sides are established. Then, considering the different state transition effects of various actions when the pitch angles of the UCAVs are different, the 10 state variables including the pitch angle, are taken as the state space. Combined with the air combat situation threat assessment index model, a two-layer reward mechanism combining internal reward and sparse reward is designed as the evaluation basis of reinforcement learning. Then, the neural network model of the full connection layer is built according to an Asynchronous Advantage Actor–Critic (A3C) algorithm. In the way of multi-threading, our UCAV keeps interactively learning with the environment to train the model and gradually learns the optimal air combat maneuver countermeasure strategy, and guides our UCAV to conduct action selection. The algorithm reduces the correlation between samples through multi-threading asynchronous learning. Finally, the effectiveness and feasibility of the method are verified in three different air combat scenarios.

Autonomous Maneuver Decision of Air Combat Based on Simulated Operation Command and FRV-DDPG Algorithm

With the improvement of UAV performance and intelligence in recent years, it is particularly important for unmanned aerial vehicles (UAVs) to improve the ability of autonomous air combat. Aiming to solve the problem of how to improve the autonomous air combat maneuver decision ability of UAVs so that it can be close to manual manipulation, this paper proposes an autonomous air combat maneuvering decision method based on the combination of simulated operation command and the final reward value deep deterministic policy gradient (FRV-DDPG) algorithm. Firstly, the six-degree-of-freedom (6-DOF) model is established based on the air combat process, UAV motion, and missile motion. Secondly, a prediction method based on the Particle swarm optimization radial basis function (PSO-RBF) is designed to simulate the operation command of the enemy aircraft, which makes the training process more realistic, and then an improved DDPG strategy is proposed, which returns the final reward value to the previous reward value in a certain proportion of time for offline training, which can improve the convergence speed of the algorithm. Finally, the effectiveness of the algorithm is verified by building a simulation environment. The simulation results show that the algorithm can improve the autonomous air combat maneuver decision-making ability of UAVs.

Autonomous Air Combat Maneuvering Decision Method of UCAV Based on LSHADE-TSO-MPC under Enemy Trajectory Prediction

In this paper, an autonomous UCAV air combat maneuvering decision method based on LSHADE-TSO optimization in a model predictive control framework is proposed, along with enemy trajectory prediction. First, a sliding window recursive prediction method for multi-step enemy trajectory prediction using a Bi-LSTM network is proposed. Second, Model Predictive Control (MPC) theory is introduced, and when combined with enemy trajectory prediction, a UCAV maneuver decision model based on the MPC framework is proposed. The LSHADE-TSO algorithm is proposed by combining the LSHADE and TSO algorithms, which overcomes the problem of traditional sequential quadratic programming falling into local optimum when solving complex nonlinear models. The LSHADE-TSO-MPC air combat maneuver decision method is then proposed, which combines the LSHADE-TSO algorithm with the MPC framework and employs the LSHADE-TSO algorithm as the optimal control sequence solver. To validate the effectiveness of the maneuvering decision method proposed in this paper, it is tested against the test maneuver and the LSHADE-TSO decision algorithm, respectively, and the experimental results show that the maneuvering decision method proposed in this paper can beat the opponent and win the air combat using the same weapons and flight platform. Finally, to demonstrate that LSHADE-TSO can better exploit the decision-making ability of the MPC model, LSHADE-TSO is compared to various optimization algorithms based on the MPC model, and the results show that LSHADE-TSO-MPC can not only help obtain air combat victory faster but also demonstrates better decision-making ability.

Maneuver Decision-Making for Autonomous Air Combat Based on FRE-PPO

Maneuver decision-making is the core of autonomous air combat, and reinforcement learning is a potential and ideal approach for addressing decision-making problems. However, when reinforcement learning is used for maneuver decision-making for autonomous air combat, it often suffers from awful training efficiency and poor performance of maneuver decision-making. In this paper, an air combat maneuver decision-making method based on final reward estimation and proximal policy optimization is proposed to solve the above problems. First, an air combat environment based on aircraft and missile models is constructed, and an intermediate reward and final reward are designed. Second, the final reward estimation is proposed to replace the original advantage estimation function of the surrogate objective of proximal policy optimization to improve the training performance of reinforcement learning. Third, sampling according to the final reward estimation is proposed to improve the training efficiency. Finally, the proposed method is used in a self-play framework to train agents for maneuver decision-making. Simulations show that final reward estimation and sampling according to final reward estimation are effective and efficient.

Interdependent Autonomous Human-Machine Systems: The Complementarity of Fitness, Vulnerability and Evolution.

For the science of autonomous human–machine systems, traditional causal-time interpretations of reality in known contexts are sufficient for rational decisions and actions to be taken, but not for uncertain or dynamic contexts, nor for building the best teams. First, unlike game theory where the contexts are constructed for players, or machine learning where contexts must be stable, when facing uncertainty or conflict, a rational process is insufficient for decisions or actions to be taken; second, as supported by the literature, rational explanations cannot disaggregate human–machine teams. In the first case, interdependent humans facing uncertainty spontaneously engage in debate over complementary tradeoffs in a search for the best path forward, characterized by maximum entropy production (MEP); however, in the second case, signified by a reduction in structural entropy production (SEP), interdependent team structures make it rationally impossible to discern what creates better teams. In our review of evidence for SEP–MEP complementarity for teams, we found that structural redundancy for top global oil producers, replicated for top global militaries, impedes interdependence and promotes corruption. Next, using UN data for Middle Eastern North African nations plus Israel, we found that a nation’s structure of education is significantly associated with MEP by the number of patents it produces; this conflicts with our earlier finding that a U.S. Air Force education in air combat maneuvering was not associated with the best performance in air combat, but air combat flight training was. These last two results exemplify that SEP–MEP interactions by the team’s best members are made by orthogonal contributions. We extend our theory to find that competition between teams hinges on vulnerability, a complementary excess of SEP and reduced MEP, which generalizes to autonomous human–machine systems.