Air Combat Environment Research Articles

Predictive air combat decision model with segmented reward allocation

In air combat missions, unmanned combat aerial vehicles (UCAVs) must take strategic actions to establish combat advantages, enabling effective tracking and attacking of enemy UCAVs. Currently, a lot of reinforcement learning algorithms are applied to the air combat mission of unmanned fighter aircraft. However, most algorithms can only select policies based on the current state of both sides. This leads to the inability to effectively track and attack when the enemy performs large angle maneuvering. Additionally, these algorithms cannot adapt to different situations, resulting in the unmanned fighter aircraft being at a disadvantage in some cases. To solve these problems, this paper proposes predictive air combat decision model with segmented reward allocation for air combat tracking and attacking. On the basis of the air combat environment, we propose the prediction soft actor-critic (Pre-SAC) algorithm, which combines the prediction of enemy states with the states of UCAV for model training. This enables the UCAV to predict the next move of the enemy UCAV in advance and establish a greater air combat advantage for us. Furthermore, by adopting a segmented reward allocation model and combining it with the Pre-SAC algorithm, we propose the segmented reward allocation soft actor-critic (Sra-SAC) algorithm, which solves the problem of UCAVs being unable to adapt to different situations. The results show that the prediction-based segmented reward allocation the Sra-SAC algorithm outperforms the traditional soft actor-critic (SAC) algorithm in terms of overall reward, travel distance, and relative advantage.

Towards Global Explainability of Artificial Intelligence Agent Tactics in Close Air Combat

In this paper, we explore the development of an explainability system for air combat agents trained with reinforcement learning, thus addressing a crucial need in the dynamic and complex realm of air combat. The safety-critical nature of air combat demands not only improved performance but also a deep understanding of artificial intelligence (AI) decision-making processes. Although AI has been applied significantly to air combat, a gap remains in comprehensively explaining an AI agent’s decisions, which is essential for their effective integration and for fostering trust in their actions. Our research involves the creation of an explainability system tailored for agents trained in an air combat environment. Using reinforcement learning, combined with a reward decomposition approach, the system clarifies the agent’s decision making in various tactical situations. This transparency allows for a nuanced understanding of the agent’s behavior, thereby uncovering their strategic preferences and operational patterns. The findings reveal that our system effectively identifies the strengths and weaknesses of an agent’s tactics in different air combat scenarios. This knowledge is essential for debugging and refining the agent’s performance and to ensure that AI agents operate optimally within their intended contexts. The insights gained from our study highlight the crucial role of explainability in improving the integration of AI technologies within air combat systems, thus facilitating more informed tactical decisions and potential advancements in air combat strategies.

Distributed cooperative fencing scheme for UAV swarm based on self-organized behaviors

In hunting mission for Unmanned Aerial Vehicle (UAV) swarm, the distributed target-fencing problem is investigated. Generally, the most of researches are done by formation control. However, due to the dynamic and complex air-combat environment, an accurate formation strategy can not be provided for all situations. And inspired by natural creatures, such as ants and bees, this paper proposed a novel distributed cooperative fencing scheme without predefined formation. Firstly, the model of UAV swarm and distributed target-fencing problem are described. Secondly, considering the moving target's position could not be accessed in real time for some UAVs, a distributed position observer (DPO) is designed. Thirdly, a distributed cooperative fencing controller (DCFC) with DPO is proposed. The self-behavior rules and consensus theory are introduced to ensure that UAV swarm can enclose target in its convex hull without preset formation and collision. Meanwhile, the target's speed can be estimated adaptively by UAVs and their states tend to be consistent eventually. Finally, simulation results exhibit the effectiveness and robustness of proposed scheme.

UAV maneuver decision-making via deep reinforcement learning for short-range air combat

The unmanned aerial vehicle (UAV) has been applied in unmanned air combat because of its flexibility and practicality. The short-range air combat situation is rapidly changing, and the UAV has to make the autonomous maneuver decision as quickly as possible. In this paper, a type of short-range air combat maneuver decision method based on deep reinforcement learning is proposed. Firstly, the combat environment, including UAV motion model and the position and velocity relationships, is described. On this basic, the combat process is established. Secondly, some improved points based on proximal policy optimization (PPO) are proposed to enhance the maneuver decision-making ability. The gate recurrent unit (GRU) can help PPO make decisions with continuous timestep data. The actor network's input is the observation of UAV, however, the input of the critic network, named state, includes the blood values which cannot be observed directly. In addition, the action space with 15 basic actions and well-designed reward function are proposed to combine the air combat environment and PPO. In particular, the reward function is divided into dense reward, event reward and end-game reward to ensure the training feasibility. The training process is composed of three phases to shorten the training time. Finally, the designed maneuver decision method is verified through the ablation study and confrontment tests. The results show that the UAV with the proposed maneuver decision method can obtain an effective action policy to make a more flexible decision in air combat.

Human-computer gaming decision-making method in air combat under an incomplete strategy set

In thereal air combat environment, uncertainty is widespread. This paper considers the air combat decision-making problem in the human-computer gaming scenario, and a game decision-making method is proposed under anincomplete strategy set.First, the dynamic game process of unmanned and manned air vehicle swarms is decomposed into different decision-making stages, and the current decision-making stage is modeled as a zero-sum game model.In the established model, the situation where the strategy of the manned air vehicle swarm is not completely known is considered, and an uncertain payoff matrix is used to describe the incomplete strategies. Then, a solutionmethod of the established game under anincomplete strategy setis proposed based on the concept of the maximum-minimum strategy in matrix games. Thus, the decision sequence of unmanned air vehicle swarms is given. Finally, the effectiveness of the proposed method is verified through simulation experiments.

Maneuver Decision-Making for Autonomous Air Combat Based on FRE-PPO

Maneuver decision-making is the core of autonomous air combat, and reinforcement learning is a potential and ideal approach for addressing decision-making problems. However, when reinforcement learning is used for maneuver decision-making for autonomous air combat, it often suffers from awful training efficiency and poor performance of maneuver decision-making. In this paper, an air combat maneuver decision-making method based on final reward estimation and proximal policy optimization is proposed to solve the above problems. First, an air combat environment based on aircraft and missile models is constructed, and an intermediate reward and final reward are designed. Second, the final reward estimation is proposed to replace the original advantage estimation function of the surrogate objective of proximal policy optimization to improve the training performance of reinforcement learning. Third, sampling according to the final reward estimation is proposed to improve the training efficiency. Finally, the proposed method is used in a self-play framework to train agents for maneuver decision-making. Simulations show that final reward estimation and sampling according to final reward estimation are effective and efficient.

Threat sequencing of multiple UCAVs with incomplete information based on game theory

The threat sequencing of multiple unmanned combat air vehicles (UCAVs) is a multi-attribute decision-making (MADM) problem. In the threat sequencing process of multiple UCAVs, due to the strong confrontation and high dynamics of the air combat environment, the weight coefficients of the threat indicators are usually time-varying. Moreover, the air combat data is difficult to be obtained accurately. In this study, a threat sequencing method of multiple UCAVs is proposed based on game theory by considering the incomplete information. Firstly, a zero-sum game model of decision maker ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{D}$</tex> ) and nature ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{N}$</tex> ) with fuzzy payoffs is established to obtain the uncertain parameters which are the weight coefficient parameters of the threat indicators and the interval parameters of the threat matrix. Then, the established zero-sum game with fuzzy payoffs is transformed into a zero-sum game with crisp payoffs (matrix game) to solve. Moreover, a decision rule is addressed for the threat sequencing problem of multiple UCAVs based on the obtained uncertain parameters. Finally, numerical simulation results are presented to show the effectiveness of the proposed approach.

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Focusing on the problem of multi-UAV cooperative air combat decision-making, a multi-UAV cooperative maneuvering decision-making approach is proposed based on multi-agent deep reinforcement learning (MARL) theory. First, the multi-UAV cooperative short-range air combat environment is established. Then, by combining the value-decomposition networks (VDNs) deep reinforcement learning theory with the embedded expert collaborative air combat experience reward function, an air combat cooperative strategy framework is proposed based on the networked decentralized partially observable Markov decision process (NDec-POMDP). The air combat maneuvering strategy is then optimized to improve the cooperative degree between UAVs in cooperative combat scenarios. Finally, multi-UAV cooperative air combat simulations are carried out and the results show the feasibility and effectiveness of the proposed cooperative air combat decision-making framework and method.

Attention based trajectory prediction method under the air combat environment

Attention based trajectory prediction method under the air combat environment

A Decision-Making Method for Air Combat Maneuver Based on Hybrid Deep Learning Network

In this paper, a hybrid deep learning network-based model is proposed and implemented for maneuver decision-making in an air combat environment. The model consists of stacked sparse auto-encoder network for dimensionality reduction of high-dimensional, dynamic time series combat-related data and long short-term memory network for capturing the quantitative relationship between maneuver control variables and the time series combat-related data after dimensionality reduction. This model features: 1) time series data is used as the basis of decision-making, which is more in line with the actual decision-making process. 2) using stacked sparse auto-encoder network to reduce the dimension of time series data to predict the result more accurately. 3) the model takes the maneuver control variables as the output to control the maneuver, making the maneuver process more flexible. The relevant experiments have demonstrated that the proposed model can effectively improve the prediction accuracy and convergence rate in the prediction of maneuver control variables.

Intention prediction of UAVs based on improved DDQN

Intention prediction plays an indispensable role in future informationized air combat, which will help the command and control center to make a more correct decision. In this paper, an intelligent intention prediction approach based on an improved double deep Q network (DDQN) is developed to generate real-time intention flight paths for unmanned aerial vehicles (UAVs) in complex air combat environment. Initially, by introducing an actual topographic map, the different threats of UAVs in different terrains on the map are analyzed, based on which, a terrain environment reward function is constructed. Secondly, by splitting a complete maneuver action into six basic maneuver units and determining the probability value of each unit, a maneuver reward function is obtained. Further, to improve the real-time performance and accuracy of the standard DDQN algorithm, an improved DDQN algorithm is proposed by using temporal-difference (TD) method and binary tree data structure. Finally, the simulation results verify show that the proposed algorithm has achieved better results under complex terrain conditions.

Air Target Recognition Algorithm based on Mixed Depth Features in the Interference Environment

Infrared aircraft target recognition technologies under complex interference conditions have been a research hotspot. The development of infrared countermeasure technologies has made the air combat environment increasingly polymorphic, thus hindering the accurate recognition of infrared targets. In particular, a complex artificial interference can severely obscure target, resulting in the loss of continuity and saliency of target features, and the inability to fully describe the characteristics of a target object, thus making aerial target recognition inaccurate. To overcome this problem, we propose an anti-interference recognition algorithm for air targets based on mixed depth features. First, to improve the ability of the algorithm accurately in selecting candidate regions in a full trajectory, an improved fuzzy C-means (FCM) adaptive segmentation algorithm is developed to extract the target candidate region. Second, a deep convolutional feature (convolutional neural network) and the histogram of gradient feature are used to construct hybrid depth features for a more comprehensive representation of the target. Finally, to consider the diverse confrontation scenarios between the target and the interference in a combat scene, a three-class support vector machine model is proposed to deal with the target, interference, and interference adhesion states. The experimental results show that in a complex interference environment, the air target anti-interference recognition algorithm based on the mixed depth features exhibits an accuracy rate of 92.29%, which represents an accurate target anti-interference recognition.

Dynamic Air Target Threat Assessment Based on Interval-Valued Intuitionistic Fuzzy Sets, Game Theory, and Evidential Reasoning Methodology

In order to reduce the uncertainty of target threat assessment results and improve exact target assessment in the complicated and changeable air combat environment, a novel method based on the combination of interval-valued intuitionistic fuzzy sets (IVIFSs), game theory, and evidential reasoning methodology is proposed in this paper. First, the imprecise and fuzzy information of battlefield air target is expressed by IVIFS. Second, the optimal index weight is determined by the interval intuitionistic fuzzy entropy and game theory. And the time series weight is calculated by the inverse Poisson distribution method. Then, the target evaluation information at different times is dynamically fused through an evidential reasoning algorithm. Finally, the accuracy function is applied to obtain the threat ranking of all the targets. A case of the threat assessment of air targets is provided to demonstrate the implementation process of the method proposed in this paper. Simulation experiments show that in a rapidly evolving combat environment, this algorithm can effectively reduce the uncertainty of target threat assessment results. It provides us with a useful way for target threat assessment based on interval-valued intuitionistic fuzzy sets, game theory, and evidential reasoning methodology.

Anti-interference recognition method of aerial infrared targets based on the Bayesian network

The development of infrared decoy countermeasure technologies has resulted in the complexity of the air combat environment. Therefore, higher requirements are put forward considering the infrared imaging-guided air-to-air missile anti-interference target recognition technology. The release of infrared decoys destroys the completeness, saliency and stability of target features. It is difficult for statistical pattern recognition methods based on feature fusion and matching to cover all of the confrontation conditions. In the present study, a Bayesian probabilistic recognition model is proposed that deals with uncertain information about targets and decoys. The proposed model deals with the interference of the air battlefield with the confrontation environment. Moreover, it simulates some functions of the human visual cognition and improves the target recognition ability when interference exists. Furthermore, numerous simulation data samples are utilized to solve the network structure as well as parameters of the Bayesian recognition model and the probabilistic definitions of the target and interference are achieved. Finally, a new aerial infrared target recognition algorithm is constructed based on prior information and the probability recognition model. Results of simulation experiments indicate that the recognition rate of the anti-interference recognition algorithm based on the Bayesian network reaches 90.73%, which is 6.905% higher than that of the anti-interference recognition algorithm based on Naive Bayes classifier in the tested air combat anti-interference simulation image dataset. Moreover, it can solve the problem of anti-interference target recognition to a certain extent such as false targets and target occlusion.

Adaptive Multi-Agent Control Strategy in Heterogeneous Countermeasure Environments

In the rapidly changing air combat environment, it is quite difficult for pilots to make speedy and reasonable decisions in a very short period due to lack of experience and the uncertainty of perception situation. Hence, the authors propose an intelligent cognitive tactical strategy framework of air combat on multi-source information in uncertain air combat situations for decision support. A fuzzy inferring tree method is proposed to simulate human intellection. Then, to further improve the accuracy of the reasoning results, a genetic algorithm is introduced to optimize the structure and parameters of fuzzy rules. The simulation results show that the proposed model is reasonable, fast, accurate, repeatable, and fatigue-free, which lays a good foundation for future high-end unmanned combat explorations.

Air combat decision-making of multiple UCAVs based on constraint strategy games

Game theory can be applied to the air combat decision-making problem of multiple unmanned combat air vehicles (UCAVs). However, it is difficult to have satisfactory decision-making results completely relying on air combat situation information, because there is a lot of time-sensitive information in a complex air combat environment. In this paper, a constraint strategy game approach is developed to generate intelligent decision-making for multiple UCAVs in complex air combat environment with air combat situation information and time-sensitive information. Initially, a constraint strategy game is employed to model attack-defense decision-making problem in complex air combat environment. Then, an algorithm is proposed for solving the constraint strategy game based on linear programming and linear inequality (CSG-LL). Finally, an example is given to illustrate the effectiveness of the proposed approach.

A Novel Elitism Co-Evolutionary Algorithm for Antagonistic Weapon-Target Assignment

The Antagonistic Weapon-Target Assignment (AGWTA) problem is a crucial decision issue in Command & Control (C2). Since this is a minimax problem, co-evolutionary algorithms can be used to solve it effectively. However, the co-evolutionary algorithm is originally designed for continuous minimax problems which loses its efficiency to discrete contexts. In this paper, a novel elitism co-evolutionary algorithm is proposed to solve the AGWTA. Firstly, an improved AGWTA model for air combat based on the attack and evasion strategies is proposed. Secondly, an elite cooperative genetic algorithm based on the framework of the co-evolutionary algorithm is put forward. In this proposed algorithm, a problem-specific coding method and evolution operator are designed. Meanwhile, an elite individual update mechanism is presented. Finally, based on the analysis of the relationship between the feasible solutions under the air combat environment, an evaluation index is proposed. Experiments show that the proposed algorithm has higher accuracy than traditional co-evolutionary algorithms for solving AGWTA problems.

Dynamic assessment method of air target threat based on improved GIFSS

As the air combat environment becomes more complicated and changeable, accurate threat assessment of air target has a significant impact on air defense operations. This paper proposes an improved generalized intuitionistic fuzzy soft set (GIFSS) method for dynamic assessment of air target threat. Firstly, the threat assessment index is reasonably determined by analyzing the typical characteristics of air targets. Secondly, after the GIFSS at different time is obtained, the index weight is determined by the intuitionistic fuzzy set entropy and the relative entropy theory. Then, the inverse Poisson distribution method is used to determine the weight of time series, and then the time-weighted GIFSS is obtained. Finally, threat assessment of five air targets is carried out by using the improved GIFSS (I-GIFSS) and comparison methods. The validity and superiority of the proposed method are verified by calculation and comparison.

Exploring the Concept of Team Situation Awareness in a Simulated Air Combat Environment

This paper evaluates team SA in air combat simulations. Team composition (in terms of diversity in team members' experience) as a factor that could influence team SA is analyzed in a study of 16 military fighter pilots performing simulated air combat missions in teams of two. The study used different measures to assess team SA, including SAGAT, self-appraisal, appraisal of teammate's state, and teamwork behavior and communication. The analysis of intercorrelations between different measures of team SA shows that they are not fully overlapping. The results show high association between good SA and team interaction as measured by information sharing and feedback, support, and backup behavior. The results also suggest the potential negative impact of diversity in team members' experience on team SA and interaction processes. Finally, we reveal that teams that perform well are characterized by individuals with high SA related to own task, well-calibrated confidence level, and good team interaction. On the other hand, teams that perform well do not necessarily do better in estimating their self-SA and teammate's status.

Multi-method Approach to Team Situation Awareness

In this paper, we present our definition and multi-method approach to assess team SA, followed by an application of the measurement approach in a simulated air combat environment. In the simulation, team SA is measured using Situation Awareness Global Assessment Technique (SAGAT) on the elements related to own and teammate's responsibilities, confidence level on the accuracy of SAGAT responses, and awareness of teammate's workload and SA levels. The interrelationships between various measures of team SA, and between each measure of team SA and teamwork behavior and performance outcome are analyzed. The result suggests that team SA is a multi-dimensional construct, and that each aspect can be developed from different information sources. The correlation analyses with teamwork behavior measures reveal the strong associations between team SA and information sharing and backup behavior. Finally, SAGAT scores, confidence bias, and backup behavior are found to have strong associations with team performance outcome.