Pursuit Evasion Research Articles

A Collision‐Free Pursuit‐Evasion Framework for Indirect Herding and Formation Control of Noncooperative UAVs

ABSTRACTIn this article, we aim to design a general framework for the pursuit‐evasion (PE) game of multiple UAVs. Based on the distance between the pursuers and the evader, the whole pursuit‐evasion process is decoupled into a seeking stage and a herding stage. A formation control method is proposed to make sure that the pursuers can find and drive the noncooperative evaders along a desired trajectory towards the designated target while maintaining a preset formation. In the meantime, an adaptive potential function is designed to achieve collision avoidance to both obstacles and other agents in the formation. The main contribution is that it combines formation control and collision avoidance in the problem of indirect herding, which is rarely addressed before. Also, the convergence and effectiveness of the designed controller for trajectory tracking and formation maintenance are proven and verified in the article. Simulation results in two and three dimensional space show that the proposed framework achieves the goal of coordinated herding and collision avoidance in a pursuit‐evasion scenario under uncertainties and external disturbances.

A Novel Method for a Pursuit–Evasion Game Based on Fuzzy Q-Learning and Model-Predictive Control

This paper explores a pursuit–evasion game (PEG) based on quadrotors by combining fuzzy Q-learning (FQL) and model-predictive control (MPC) algorithms. Initially, the FQL algorithm is employed to perceive, make decisions, and predict the trajectory of the evader. Based on the position and velocity information of both players in the game, the pursuer quadrotor determines its action strategy using the FQL algorithm. Subsequently, a state feedback controller is designed using the MPC algorithm, with reference inputs derived from the FQL algorithm. Within each MPC cycle, the FQL algorithm dynamically provides reference inputs to the MPC, thereby enhancing its robust control and dynamic optimization for the quadrotor. Finally, simulation results verify the effectiveness of the proposed algorithm.

Time-Dependent Reachable Domain and Its Application to Impulsive Orbital Pursuit–Evasion Analysis

Orbital pursuit–evasion (OPE) has garnered significant attention due to its potential applications in space rendezvous and proximity operations. Conventional methods using differential games do not fully address the impulsive OPE problem, which involves a discontinuous differential system. This paper presents the concept of a time-dependent reachable domain (TDRD) for analyzing the impulsive OPE problem. The range and time constraints for determining the TDRD are first given. These constraints can be solved to identify two different TDRDs: the RD at a given time and the RD over a duration. The numerical results obtained from the proposed method exhibit good agreement with those from Monte Carlo simulations, validating the practicality of using TDRDs for OPE analysis. Four different qualitatively OPE outcomes are defined based on the geometrical relationship between the spacecraft’s TDRDs. A series of application cases are comprehensively examined to prove that TDRDs directly predict the sufficient conditions leading to a specific OPE outcome. These findings demonstrate that the TDRD concept is a potent and efficacious instrument for studying the impulsive OPE problem.

Luring cooperative capture guidance strategy for the pursuit—evasion game under incomplete target information

Luring cooperative capture guidance strategy for the pursuit—evasion game under incomplete target information

Resilient pursuit evasion guidance with feedback game strategy

This paper proposed a resilient guidance strategy for planar pursuit evasion, in which the homing guidance problem is formulated as a Stackelberg game. As is known, the most salient challenge for pursuit evasion guidance is how to ensure an interception despite both players (i.e., missile and target) actively optimize their own interest with preferably incomplete information. Toward this, we introduce a game-theoretic guidance strategy that effectively integrates incentive feedback strategy into pursuit evasion game. In particular, the proposed resilient guidance law takes the form of Stackelberg game conjecture with missile/interceptor as the designated leader, and rigorously proves that leader's interest can be best served with a properly designed feedback gain, and an interception can be made possible in spite of incomplete knowledge on target's intentions (i.e., performance index). Simulation results verify the performance of the proposed strategy.

Maneuver Planning for Multiple Pursuit Intelligent Surface Vehicles in a Sequence of Zero-Sum Pursuit–Evasion Games

Unmanned surface pursuit is a complex and challenging engineering problem, especially when conducted by multiple intelligent surface vehicles (ISVs). To enhance the pursuit performance and facilitate strategic interaction during the target pursuit, this paper proposes a novel game theory-based maneuver planning method for pursuit ISVs. Firstly, a specific two-player zero-sum pursuit–evasion game (ZSPEG)-based target-pursuit model is formed. To ensure the vehicles reach a quick consensus, a target-guided relay-pursuit mechanism and the corresponding pursuit payoffs are designed. Meanwhile, under the fictitious play framework, the behavioral pattern and the strategies of the target could be fictitiously learned. Furthermore, mixed-strategy Nash equilibrium (MNE) is employed to determine the motions for the vehicles, the value of which is the best response in the proposed ZSPEG model. Finally, simulations verify the effectiveness of the above methods in multi-ISV surface pursuit.

A Deep Reinforcement Learning-Based Intelligent Maneuvering Strategy for the High-Speed UAV Pursuit-Evasion Game

Given the rapid advancements in kinetic pursuit technology, this paper introduces an innovative maneuvering strategy, denoted as LSRC-TD3, which integrates line-of-sight (LOS) angle rate correction with deep reinforcement learning (DRL) for high-speed unmanned aerial vehicle (UAV) pursuit–evasion (PE) game scenarios, with the aim of effectively evading high-speed and high-dynamic pursuers. In the challenging situations of the game, where both speed and maximum available overload are at a disadvantage, the playing field of UAVs is severely compressed, and the difficulty of evasion is significantly increased, placing higher demands on the strategy and timing of maneuvering to change orbit. While considering evasion, trajectory constraint, and energy consumption, we formulated the reward function by combining “terminal” and “process” rewards, as well as “strong” and “weak” incentive guidance to reduce pre-exploration difficulty and accelerate convergence of the game network. Additionally, this paper presents a correction factor for LOS angle rate into the double-delay deterministic gradient strategy (TD3), thereby enhancing the sensitivity of high-speed UAVs to changes in LOS rate, as well as the accuracy of evasion timing, which improves the effectiveness and adaptive capability of the intelligent maneuvering strategy. The Monte Carlo simulation results demonstrate that the proposed method achieves a high level of evasion performance—integrating energy optimization with the requisite miss distance for high-speed UAVs—and accomplishes efficient evasion under highly challenging PE game scenarios.

Deep Reinforcement Learning-Based Differential Game Guidance Law against Maneuvering Evaders

To achieve the intelligent interception of different types of maneuvering evaders, based on deep reinforcement learning, a novel intelligent differential game guidance law is proposed in the continuous action domain. Different from traditional guidance laws, the proposed guidance law can avoid tedious manual settings and save cost efforts. First, the interception problem is transformed into the pursuit–evasion game problem, which is solved by zero-sum differential game theory. Next, the Nash equilibrium strategy is obtained through the Markov game process. To implement the proposed intelligent differential game guidance law, an actor–critic neural network based on deep deterministic policy gradient is constructed to calculate the saddle point of the differential game guidance problem. Then, a reward function is designed, which includes the tradeoffs among guidance accuracy, energy consumption, and interception time. Finally, compared with traditional methods, the interception accuracy of the proposed intelligent differential game guidance law is 99.2%, energy consumption is reduced by 47%, and simulation time is shortened by 1.58 s. All results reveal that the proposed intelligent differential game guidance law has better intelligent decision-making ability.

Orbital Impulsive Pursuit–Evasion Game Formulation and Stackelberg Equilibrium Solutions

The spacecraft pursuit–evasion game in orbit with impulsive maneuvers is investigated based on dynamic game theory. First, the game models are developed progressively from the single-impulse game to the multi-impulse game. In the models, the two-sided optimal strategies of both players form a Stackelberg equilibrium in single-impulse games and constitute a multistage Stackelberg equilibrium in multi-impulse games. Then, the optimality necessary conditions for the two equilibriums are derived and simplified as two multipoint boundary value problems (MPBVPs). To handle the complicated MPBVPs, a rolling backward induction based on the Lambert algorithm accompanied by a homotopy shooting method is proposed to solve the equilibrium solutions. Simulations showed the effectiveness of the proposed methods and verified that the obtained solutions are the (multistage) Stackelberg equilibrium. Meanwhile, the differences between the Stackelberg equilibrium and the Nash equilibrium are illustrated in detail. Finally, an analysis of the influence of the observation delay on the game results is provided.

Orbital Pursuit–Evasion–Defense Linear-Quadratic Differential Game

Orbital Pursuit–Evasion–Defense Linear-Quadratic Differential Game

Boundedness of Solutions to a Fully Parabolic Indirect Pursuit–Evasion Predator–Prey System with Density-Dependent Diffusion in R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\mathbb{R}}}^2$$\\end{document}

This paper deals with a fully parabolic indirect pursuit–evasion predator–prey system with density-dependent diffusion ut=Δ(ψ1(w)u)+u(λ-u+αv),vt=Δ(ψ2(z)v)+v(μ-v-βu),wt=Δw-w+v,zt=Δz-z+u\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$u_{t}=\\Delta (\\psi _1(w)u)+u(\\lambda -u+\\alpha v), v_{t}=\\Delta (\\psi _2(z) v)+v(\\mu -v-\\beta u), w_{t}=\\Delta w -w+v, z_{t}=\\Delta z-z+u$$\\end{document} under a smooth bounded domain Ω⊂R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\Omega \\subset {\\mathbb{R}}^2$$\\end{document} with homogeneous Neumann boundary conditions, where the parameters λ,μ,α\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\lambda , \\mu , \\alpha$$\\end{document} and β\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\beta$$\\end{document} are assumed to be positive. Through the establishment of appropriate conditions for the density-dependent diffusion functions ψ1(w)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\psi _1(w)$$\\end{document} and ψ2(z),\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\psi _2(z),$$\\end{document} it is revealed that a unique classical solution exists for the corresponding initial-boundary problem, which remains uniformly bounded over time.

Team formation through an assessor: choosing MARL agents in pursuit–evasion games

Team formation in multi-agent systems usually assumes the capabilities of each team member are known, and the best formation can be derived from that information. As AI agents become more sophisticated, this characterisation is becoming more elusive and less predictive about the performance of a team in cooperative or competitive situations. In this paper, we introduce a general and flexible way of anticipating the outcome of a game for any lineups (the agents, sociality regimes and any other hyperparameters for the team). To this purpose, we simply train an assessor using an appropriate team representation and standard machine learning techniques. We illustrate how we can interrogate the assessor to find the best formations in a pursuit–evasion game for several scenarios: offline team formation, where teams have to be decided before the game and not changed afterwards, and online team formation, where teams can see the lineups of the other teams and can be changed at any time.

Game Tree Search-based Impulsive Orbital Pursuit–Evasion Game with Limited Actions

This paper focuses on the impulsive orbital pursuit–evasion game (OPEG) with limited action sets for the pursuer and evader. Initially, a mathematical model is developed by combining game theory and orbital dynamics, forming a finite-round impulsive OPEG problem. The problem is then formulated as a bilateral optimization problem, employing a minimum-maximum optimization index based on terminal distance. To tackle this problem, an algorithm based on game tree search is designed, enabling the determination of the optimal pursuit–evasion strategy with limited action sets. Additionally, we explore the influence of the initial pursuing orientation on OPEG. The optimal initial pursuit orientation is analytically derived using relative motion dynamics under uncontrolled conditions. Furthermore, considering factors such as the initial status of the pursuit spacecraft, initial relative distance, transfer time, and maneuverability, the impulsive OPEG problem with limited action sets is numerically solved using game tree search. The findings of this study showcase the efficacy of game tree search in addressing impulsive OPEG problems with limited action sets. The study also demonstrates that the initial pursuing orientation selection at the start of the game plays a crucial role in increasing the success rate of pursuit. The research findings of this study have important implications for future practical engineering applications.

Approximate Optimal Strategy for Multiagent System Pursuit–Evasion Game

Approximate Optimal Strategy for Multiagent System Pursuit–Evasion Game

Approximate Methods for Visibility-Based Pursuit–Evasion

Approximate Methods for Visibility-Based Pursuit–Evasion

Global existence and boundedness in a two-species chemotaxis-fluid system with indirect pursuit–evasion interaction

Abstract This paper investigates a two-species chemotaxis-fluid system with indirect pursuit–evasion interaction in a bounded domain with smooth boundary. Under suitably regular initial data and no-flux/no-flux/no-flux/no-flux/Dirichlet boundary conditions, we prove that the system possesses a global bounded classical solution in the two-dimensional and three-dimensional cases. Our results extend the result obtained in previously known ones and partly result is new.

On a Linear Differential Game in the Hilbert Space ℓ2

Two player pursuit evasion differential game and time optimal zero control problem in <i>ℓ</i><sup>2</sup> are considered. Optimal control for the corresponding zero control problem is found. A strategy for the pursuer that guarantees the solution for the pursuit problem is constructed.

Modeling the Solution of the Pursuit–Evasion Problem Based on the Intelligent–Geometric Control Theory

An important action-planning problem is considered for participants of the pursuit–evasion game with multiple pursuers and a high-speed evader. The objects of study are mobile robotic systems and specifically small unmanned aerial vehicles (UAVs). The problem is complicated by the presence of significant wind loads that affect the trajectory and motion strategies of the players. It is assumed that UAVs have limited computing resources, which involves the use of computationally fast and real-time heuristic approaches. A novel and rapidly developing intelligent–geometric theory is applied to address the discussed problem. To accurately calculate the points of the participant’s rapprochement, we use a geometric approach based on the construction of circles or spheres of Apollonius. Intelligent control methods are applied to synthesize complex motion strategies of participants. A method for quickly predicting the evader’s trajectory is proposed based on a two-layer neural network containing a new activation function of the “s-parabola” type. We consider a special backpropagation training scheme for the model under study. A simulation scheme has been developed and tested, which includes mathematical models of dynamic objects and wind loads. The conducted simulations on pursuit–evasion games in close to real conditions showed the prospects and expediency of the presented approach.

Guidance strategy of motion camouflage for spacecraft pursuit-evasion game

This work is inspired by a stealth pursuit behavior called motion camouflage whereby a pursuer approaches an evader while the pursuer camouflages itself against a predetermined background. We formulate the spacecraft pursuit-evasion problem as a stealth pursuit strategy of motion camouflage, in which the pursuer tries to minimize a motion camouflage index defined in this paper. The Euler-Hill reference frame whose origin is set on the circular reference orbit is used to describe the dynamics. Based on the rule of motion camouflage, a guidance strategy in open-loop form to achieve motion camouflage index is derived in which the pursuer lies on the camouflage constraint line connecting the central spacecraft and evader. In order to dispose of the dependence on the evader acceleration in the open-loop guidance strategy, we further consider the motion camouflage pursuit problem within an infinite-horizon nonlinear quadratic differential game. The saddle point solution to the game is derived by using the state-dependent Riccati equation method, and the resulting closed-loop guidance strategy is effective in achieving motion camouflage. Simulations are performed to demonstrate the capabilities of the proposed guidance strategies for the pursuit–evasion game scenario.

On the Lifeline Game of the Inertial Players with Integral and Geometric Constraints

In this paper, we consider a pursuit–evasion game of inertial players, where the pursuer’s control is subject to integral constraint and the evader’s control is subject to geometric constraint. In the pursuit problem, the main tool is the strategy of parallel pursuit. Sufficient conditions are obtained for the solvability of pursuit–evasion problems. Additionally, the main lemma describing the monotonicity of an attainability domain of the evader is proved, and an explicit analytical formula for this domain is given. One of the main results of the paper is the solution of the Isaacs lifeline game for a special case.