Pursuit-evasion Game Research Articles

A Region-Based Relay Pursuit Scheme for a Pursuit–Evasion Game With Curvature-Bound Players

A Region-Based Relay Pursuit Scheme for a Pursuit–Evasion Game With Curvature-Bound Players

A pursuit-evasion game robot controller design based on a neural network with an improved optimization algorithm

A pursuit-evasion game (PEG) is a type of game that utilizes one or several cooperative pursuers to capture one or several evaders. The PEG game concept has been used in different multi-robot applications such as transportation or navigation applications, search and rescue, surveillance applications such as collision avoidance and air traffic control systems, multi-defense applications such as missile guidance systems, and medical applications such as analyzing biological behaviors. Regardless of the benefits of PEG, one of the main drawbacks of such systems is the computational burden and the immense time required to learn such systems. For this reason, this work proposes a neural network game based on the pursuit-evasion game, where the leader (evader) robot tries to eat several particles/apples distributed inside a closed game environment with boundary and inner obstacles. In contrast, a follower (pursuer) robot tries to capture the leader robot and stop the particle-eating process. The leader and follower robots were designed based on a differential two-wheel robot (DTWR). The neural network is presented to control and learn the leader and follower robot directions with respect to the boundary and inside obstacles in the game environment. The neural network weights are learned using an improved sine cosine algorithm based on chaotic theory (ISCACT). The ISCACT is proposed to solve and avoid the proposed game of being trapped in the local minimum problem. The ISCACT is tested based on five multimodal benchmark functions. The ISCACT has been used in two cases, the first case arises when ISCACT is used in the follower robot’s learning process. In the second case, the ISCACT has been used in the leader robot’s learning process. The results for the first and second cases prove the superiority of the ISCACT compared with other existing works in enhancing the PEG performance time and reducing the computational burden for multi-robot applications.

Evade Unknown Pursuer via Pursuit Strategy Identification and Model Reference Policy Adaptation (MRPA) Algorithm

The game of pursuit–evasion has always been a popular research subject in the field of Unmanned Aerial Vehicles (UAVs). Current evasion decision making based on reinforcement learning is generally trained only for specific pursuers, and it has limited performance for evading unknown pursuers and exhibits poor generalizability. To enhance the ability of an evasion policy learned by reinforcement learning (RL) to evade unknown pursuers, this paper proposes a pursuit UAV attitude estimation and pursuit strategy identification method and a Model Reference Policy Adaptation (MRPA) algorithm. Firstly, this paper constructs a Markov decision model for the pursuit–evasion game of UAVs that includes the pursuer’s attitude and trains an evasion policy for a specific pursuit strategy using the Soft Actor–Critic (SAC) algorithm. Secondly, this paper establishes a novel relative motion model of UAVs in pursuit–evasion games under the assumption that proportional guidance is used as the pursuit strategy, based on which the pursuit UAV attitude estimation and pursuit strategy identification algorithm is proposed to provide adequate information for decision making and policy adaptation. Furthermore, a Model Reference Policy Adaptation (MRPA) algorithm is presented to improve the generalizability of the evasion policy trained by RL in certain environments. Finally, various numerical simulations imply the precision of pursuit UAV attitude estimation and the accuracy of pursuit strategy identification. Also, the ablation experiment verifies that the MRPA algorithm can effectively enhance the performance of the evasion policy to deal with unknown pursuers.

Compact Metric Spaces with Infinite Cop Number

AbstractMohar recently adapted the classical game of Cops and Robber from graphs to metric spaces, thereby unifying previously studied pursuit-evasion games. He conjectured that finitely many cops can win on any compact geodesic metric space, and that their number can be upper-bounded in terms of the ranks of the homology groups when the space is a simplicial pseudo-manifold. We disprove these conjectures by constructing a metric on $$\mathbb {S}^3$$ S 3 with infinite cop number. More problems are raised than settled.

Cooperative defense of a territorial-constrained target in a target-attacker-defender game

Abstract Multi-player pursuit-evasion games are crucial for addressing the maneuver decision problem arising in the cooperative control of multi-agent systems. This paper presents a cooperative defense strategy involving cooperation and confrontation among the target, attacker, and multiple defenders based on location information only. The primary objective of the attacker is to capture the target while avoiding being captured by multiple defenders. Meanwhile, the target is confined to a restricted area and can only move within its boundaries. The proposed cooperative defense strategy aims to prevent the attacker from capturing the target while minimizing the time required to neutralize the threat. Therefore, the multiple defenders are classified into two categories: the primary defender and the auxiliary defenders. The primary defender is to prevent the attacker from approaching the target by predicting the intention of the attacker. On the other hand, the auxiliary defenders adopt a surround-shrink-capture strategy to reduce the time consumption to capture the attacker. Numerical simulations have been conducted to validate the effectiveness of the proposed strategy.

A maneuvering strategy based on motion camouflage in three-player differential game

In this paper, we consider a three-player differential game in which the defender provides coverage for the evader by maneuvering toward the line of sight between the pursuer and the evader. The TAD game is divided into two pursuit-evasion games to obtain the saddle point strategy for each player. Different from the general solution, the defender uses motion camouflage conditions as its payoff function and gives the real-time maneuvering strategy. This approach ensures that the constraint in the gaming process is met by providing the saddle point solution for each time interval. Additionally, interception strategies are created for each player to get the required heading angle. These strategies and game dynamics can be predicted by examining the current state of the game. Consequently, we establish the advantage region for the pursuer and the defender based on these predictions. Based on the given initial conditions, the advantage region for the defender is presented in its initial velocity vector. Finally, the simulations validate the effectiveness of the proposed strategy and the capture region analysis.

Iterative algorithms for solving one-sided partially observable stochastic shortest path games

Real-world scenarios often involve dynamic interactions among competing agents, where decisions are made considering actions taken by others. These situations can be modeled as partially observable stochastic games (POSGs), with zero-sum variants capturing strictly competitive interactions (e.g., security scenarios). While such models address a broad range of problems, they commonly focus on infinite-horizon scenarios with discounted-sum objectives. Using the discounted-sum objective, however, can lead to suboptimal solutions in cases where the length of the interaction does not directly affect the gained rewards of the players.We thus focus on games with undiscounted objective and an indefinite horizon where every realization of the game is guaranteed to terminate after some unspecified number of turns. To manage the computational complexity of solving POSGs in general, we restrict to games with one-sided partial observability where only one player has imperfect information while their opponent is provided with full information about the current situation. We introduce two novel algorithms based on the heuristic search value iteration (HSVI) algorithm that iteratively solve sequences of easier-to-solve approximations of the game using fundamentally different approaches for constructing the sequences: (1) in GoalHorizon, the game approximations are based on a limited number of turns in which players can change their actions, (2) in GoalDiscount, the game approximations are constructed using an increasing discount factor. We provide theoretical qualitative guarantees for algorithms, and we also experimentally demonstrate that these algorithms are able to find near-optimal solutions on pursuit-evasion games and a game modeling privilege escalation problem from computer security.

Distributed Pursuit-Evasion Game of Limited Perception USV Swarm Based on Multiagent Proximal Policy Optimization

Distributed Pursuit-Evasion Game of Limited Perception USV Swarm Based on Multiagent Proximal Policy Optimization

Multi-AUV Pursuit-Evasion Game in the Internet of Underwater Things: An Efficient Training Framework via Offline Reinforcement Learning

Multi-AUV Pursuit-Evasion Game in the Internet of Underwater Things: An Efficient Training Framework via Offline Reinforcement Learning

Optimal strategies design of active target defense differential game

In this paper, the optimal strategies are investigated for the differential game of active target defense. Specifically, we describe a pursuitevasion game with three agents (namely, an attacker, a target and a defender). For the three agents engagement scenario, two different differential game guidance laws–pure pursuit (PP) and proportional navigation (PN)–are presented for defender. Firstly, the relative motion kinematic models of the three agents are established. We subsequently derive the coupled Hamilton-Jacobi (HJ) equations to design the optimal control strategies for the three agents. Through the construction of actor-critic neural networks, optimal solutions are obtained. Then, the stability of the three-agent system is ensured, and the convergence to the Nash equilibrium equations is demonstrated. Finally, we simulate the optimal strategies employed by the attacker and the defender-target team, thereby highlighting the contrasting outcomes and superiority of different defensive approaches.

Impulsive maneuver strategy for multi-agent orbital pursuit-evasion game under sparse rewards

To address the subjectivity of dense reward designs for the orbital pursuit-evasion game with multiple optimization objectives, this paper proposes the reinforcement learning method with a hierarchical network structure to guide game strategies under sparse rewards. Initially, to overcome the convergence challenges in the reinforcement learning training process under sparse rewards, a hierarchical network structure is proposed based on the hindsight experience replay. Subsequently, considering the strict constraints imposed by orbital dynamics on spacecraft state space, the reachable domain method is introduced to refine the subgoal space in the hierarchical network, further facilitating the achievement of subgoals. Finally, by adopting the centralized training-layered execution approach, a complete multi-agent reinforcement learning method with the hierarchical network structure is established, enabling networks at each level to learn effectively in parallel within sparse reward environments. Numerical simulations indicate that, under the single-agent reinforcement learning framework, the proposed method exhibits superior stability in the late training stage and enhances exploration efficiency in the early stage by 38.89% to 55.56% to the baseline method. Under the multi-agent reinforcement learning framework, as the relative distance decreases, the subgoals generated by the hierarchical network transition from long-term to short-term, aligning with human behavioral logic.

A Novel Method for a Pursuit–Evasion Game Based on Fuzzy Q-Learning and Model-Predictive Control

This paper explores a pursuit–evasion game (PEG) based on quadrotors by combining fuzzy Q-learning (FQL) and model-predictive control (MPC) algorithms. Initially, the FQL algorithm is employed to perceive, make decisions, and predict the trajectory of the evader. Based on the position and velocity information of both players in the game, the pursuer quadrotor determines its action strategy using the FQL algorithm. Subsequently, a state feedback controller is designed using the MPC algorithm, with reference inputs derived from the FQL algorithm. Within each MPC cycle, the FQL algorithm dynamically provides reference inputs to the MPC, thereby enhancing its robust control and dynamic optimization for the quadrotor. Finally, simulation results verify the effectiveness of the proposed algorithm.

Optimal avoidance strategy based on nonlinear approximate analytic solution of non-cooperative differential game

Abstract This study examines the pursuit-evasion game involving unmanned aerial vehicles (UAVs), with a specific focus on the scenario of N-pursuers-one-escapee. The primary objective is to develop an optimal strategy for the escapee when the pursuers possess superior capabilities. To obtain this objective, we conduct the following study. Firstly, to enhance realism, a non-cooperative differential game model is formulated, incorporating multiple motion characteristics, including aerodynamics, overloading, and imposed constraints. Secondly, the end-value performance index is subsequently converted to an integral one, simplifying the solution process of the Hamilton-Jacobi-Bellman (HJB) equation. An iterative method is utilised to determine the covariates using the Cauchy initial value problem, and its convergence and uniqueness are established. The optimal avoidance strategy is subsequently derived from the covariates. Finally, the superiority of the proposed strategy is validated through simulation experiments and compared to three advanced optimal avoidance strategies. A total of 1,000 anti-jamming simulation experiments are conducted to verify the robustness of the proposed strategy.

A mixed Nash equilibrium solution for visibility-based pursuit-evasion game with multiple obstacles

A mixed Nash equilibrium solution for visibility-based pursuit-evasion game with multiple obstacles

Luring cooperative capture guidance strategy for the pursuit—evasion game under incomplete target information

Luring cooperative capture guidance strategy for the pursuit—evasion game under incomplete target information

A model predictive Stackelberg solution to orbital pursuit-evasion game

In this paper, we investigate analytical numerical iterative strategies for the pursuit-evasion game involving spacecraft with leader–follower information. In the proposed problem, the interplay between two spacecraft gives rise to a dynamic and real-time game, complicated further by the presence of J2 perturbation. The primary challenge lies in crafting control strategies that are both efficient and applicable to real-time game problems within a nonlinear system. To overcome this challenge, we introduce the model prediction and iterative correction technique proposed in model predictive static programming, enabling the generation of strategies in analytical iterative form for nonlinear systems. Subsequently, we proceed by integrating this model predictive framework into a simplified Stackelberg equilibrium formulation, tailored to address the practical complexities of leader–follower pursuit-evasion scenarios. Simulation results validate the effectiveness and exceptional efficiency of the proposed solution within a receding horizon framework.

Impulsive thrust strategy for orbital pursuit-evasion games based on impulse-like constraint

This paper proposes a novel impulsive thrust strategy guided by optimal continuous thrust strategy to address two-player orbital pursuit-evasion game under impulsive thrust control. The strategy seeks to enhance the interpretability of impulsive thrust strategy by integrating it within the framework of differential game in traditional continuous systems. First, this paper introduces an impulse-like constraint, with periodical changes in thrust amplitude, to characterize the impulsive thrust control. Then, the game with the impulse-like constraint is converted into the two-point boundary value problem, which is solved by the combined shooting and deep learning method proposed in this paper. Deep learning and numerical optimization are employed to obtain the guesses for unknown terminal adjoint variables and the game terminal time. Subsequently, the accurate values are solved by the shooting method to yield the optimal continuous thrust strategy with the impulse-like constraint. Finally, the shooting method is iteratively employed at each impulse decision moment to derive the impulsive thrust strategy guided by the optimal continuous thrust strategy. Numerical examples demonstrate the convergence of the combined shooting and deep learning method, even if the strongly nonlinear impulse-like constraint is introduced. The effect of the impulsive thrust strategy guided by the optimal continuous thrust strategy is also discussed.

Resilient pursuit evasion guidance with feedback game strategy

This paper proposed a resilient guidance strategy for planar pursuit evasion, in which the homing guidance problem is formulated as a Stackelberg game. As is known, the most salient challenge for pursuit evasion guidance is how to ensure an interception despite both players (i.e., missile and target) actively optimize their own interest with preferably incomplete information. Toward this, we introduce a game-theoretic guidance strategy that effectively integrates incentive feedback strategy into pursuit evasion game. In particular, the proposed resilient guidance law takes the form of Stackelberg game conjecture with missile/interceptor as the designated leader, and rigorously proves that leader's interest can be best served with a properly designed feedback gain, and an interception can be made possible in spite of incomplete knowledge on target's intentions (i.e., performance index). Simulation results verify the performance of the proposed strategy.

Cops and robbers game and applications of its variant

Pursuit-evasion game is a game with a pursuer player and an evader player, it has numerous applications including artificial intelligence, robot motion planning and so on. The cops and robbers game is a type of pursuit-evasion game played on graphs, and its variant is a very fruitful research area in graph theory. The purpose of this paper is to find the specific relations between pursuit-evasion game applications and different types of variants of the cops and robbers game. The research method is to find variants with different game rules and winning strategies and classify them. Then compare these rules and strategies with the applications of pursuit-evasion games to find their relationships. The comparison result shows that there are many differences between the variants and the applications, which lead to their inability to be directly related. The application of the pursuit-evasion game is mainly based on route planning and object distance, and generally contains multiple variant types, which is different from the current research direction of variant.

Maneuver Planning for Multiple Pursuit Intelligent Surface Vehicles in a Sequence of Zero-Sum Pursuit–Evasion Games

Unmanned surface pursuit is a complex and challenging engineering problem, especially when conducted by multiple intelligent surface vehicles (ISVs). To enhance the pursuit performance and facilitate strategic interaction during the target pursuit, this paper proposes a novel game theory-based maneuver planning method for pursuit ISVs. Firstly, a specific two-player zero-sum pursuit–evasion game (ZSPEG)-based target-pursuit model is formed. To ensure the vehicles reach a quick consensus, a target-guided relay-pursuit mechanism and the corresponding pursuit payoffs are designed. Meanwhile, under the fictitious play framework, the behavioral pattern and the strategies of the target could be fictitiously learned. Furthermore, mixed-strategy Nash equilibrium (MNE) is employed to determine the motions for the vehicles, the value of which is the best response in the proposed ZSPEG model. Finally, simulations verify the effectiveness of the above methods in multi-ISV surface pursuit.