Pursuit-evasion Differential Games Research Articles

Multiplayer Pursuit-Evasion Differential Games With Malicious Pursuers

This article investigates the multiplayer pursuit-evasion (MPE) differential games in the presence of malicious pursuers. First, three types of malicious pursuers, namely <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">greedy</i> , <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lazy</i> , and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">traitorous</i> pursuers are formulated by using two <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">subjectively malicious factors</i> in the performance index. Then, an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> -player nonzero-sum game framework is established to study the relationships among healthy pursuers, malicious pursuers, and evaders. The capturability and Nash equilibrium of such a new MPE differential game are rigorously analyzed. Finally, the effects of malicious pursuers on the capture time are explicitly quantified by the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">minimal pursuit coefficient</i> , whose explicit form is obtained by the solutions from coupled Riccati differential equations.

Pursuit-evasion differential games with Gr-constraints on controls

In the paper, a pursuit-evasion differential game is considered when controls of the players are subject to differential constraints in the form of Grönwall's integral inequality. The strategy of parallel pursuit (briefly, $\Pi$-strategy) was introduced and used by L.A. Petrosyan to solve simple pursuit problems under phase constraints on the states of the players in the case when control functions of both players are chosen from the class $L_\infty$. In the present work, the $\Pi$-strategy is constructed for a simple pursuit problem in the cases when control functions of both players are chosen from different classes of the Grönwall type constraints, and sufficient conditions of capture and optimal capture time are found in these cases. To solve the evasion problem, we suggest a control function for the Evader and find sufficient conditions of evasion. In addition, an attainability domain of the players is constructed and its conditions of embedding in respect to time are given. Results of this work continue and extend the works of R. Isaacs, L.A. Petrosyan, B.N. Pshenichnyi, A.A. Chirii, A.A. Azamov and other researchers, including the authors.

Survey on spacecraft orbital pursuit-evasion differential games

Orbital pursuit-evasion (PE) of spacecraft has drawn increasingly significant attention from the research community in recent years. This paper surveys the state-of-the-art advances in solutions to the PE problem, particularly focusing on differential game theory methods. First, general mathematical descriptions of the PE game are presented and recent theoretical developments are discussed. Further, major results on spacecraft orbital PE differential games under a wide range of formulations are reviewed, including qualitative, linear quadratic, and free-time games, as well as a game with incomplete or imperfect information and one with impulsive thrust. Finally, we conclude with a discussion of open challenges and directions for future research on spacecraft orbital PE.

Cooperative Pursuit by Multiple Pursuers of a Single Evader

This paper considers pursuit-evasion differential games in the Euclidean plane where an evader is engaged by multiple pursuers and point capture is required. The players have simple motion (i.e., holonomic) in the manner of Isaacs, and the pursuers are faster than the evader. The attention of this paper is confined to the case where the pursuers have the same speed, and so the game’s parameter is that the evader/pursuers speed ratio is . State feedback capture strategies and an evader strategy that yields a lower bound on his/her time-to-capture are devised using a geometric method. It is shown that, in group/swarm pursuit, when the players are in general position, capture is effected by one, two, or three critical pursuers, and this is irrespective of the size of the pursuit pack. Group pursuit devolves into pure pursuit by one of the pursuers or into a pincer movement pursuit by two or three pursuers who isochronously capture the evader. The critical pursuers are identified. However, these geometric method-based pursuit and evasion strategies are optimal only in a part of the state space where a strategic saddle point is obtained and the value of the differential game is established. As such, these strategies are suboptimal. To fully explore the differential game’s high-dimensional state space and get a better understanding of group pursuit, numerical experimentation is undertaken. The state space region where the geometric solution of the group pursuit differential game is the optimal solution becomes larger the smaller the speed ratio parameter is.

A numerical method based on the complementarity and optimal control formulations for solving a family of zero-sum pursuit-evasion differential games

This paper presents an efficient numerical method for a class of Zero-Sum Pursuit-Evasion Differential Games (ZSPEDGs). The aim of the presented method is to resolve the drawbacks of the indirect methods in solving ZSPEDGs. In the indirect methods, the solution of ZSPEDG is found by solving a Two-Point Boundary Value Problem (TPBVP) derived from the necessary conditions. The indirect methods are accurate and fast. However, they are very sensitive to initial guess, and when the control is bounded, some non-smooth equations appear in the resulting TPBVP. These drawbacks restrict the use of indirect methods for solving ZSPEDGs. To overcome these drawbacks, at first, we reformulate the discontinuous equations of TPBVP by complementarity conditions, and as a result a Differential Complementarity System (DCS) is obtained. Then, the resulting DCS is reformulated to an optimal control problem, which can be solved by a well-developed direct or indirect method. The efficiency and robustness of the method are reported by means of two benchmarks and two real-life ZSPEDG problems.

A fuzzy deterministic policy gradient algorithm for pursuit-evasion differential games

Fuzzy inference systems with reinforcement learning are currently being used in differential games to train agents with no prior experience. However, the reinforcement learning algorithms based on actor-critic structure have a drawback that the policy is depended on a probability distribution. In this paper, a novel fuzzy deterministic policy gradient algorithm is introduced and applied to classical 1-vs-1 constant-velocity pursuit-evasion differential games. The key goal is to self-learn the optimal strategy in the continuous action domain and obtain a specific physical meaning of the fuzzy rules. The novel proposed algorithm is based on the deterministic policy gradient theorem and the agent learns the near-optimal strategy under the actor-critic structure. The fuzzy inference system is applied as approximators so that the specific physical meaning can be obtained by the linguistic fuzzy rules. Furthermore, the proposed algorithm is applied to solve the decision-making problem of pursuit-evasion differential games. The result is compared with other existing algorithms and it elucidates that the proposed algorithm outperforms the precision and convergence efficiency.

The Classical Homicidal Chauffeur Game

The Homicidal Chauffeur differential game is often mentioned in the literature, but the game’s complete solution is not readily available. In this work, the complete solution process of the Homicidal Chauffeur differential game is illustrated for the parameter range in the heart of the speed ratio-capture radius parameter space initially investigated by Isaacs, and referred to by Breakwell and Merz as the classical Homicidal Chauffeur differential game. Some salient features of the solution of this foundational differential game are highlighted, and some popular misconceptions are dispelled. This tutorial paper fills a gap in the literature on pursuit-evasion differential games.

Toward a Solution of the Active Target Defense Differential Game

A novel pursuit-evasion differential game involving three agents is considered. An Attacker missile is pursuing a Target aircraft. The Target aircraft is aided by a Defender missile launched by, say, the wingman, to intercept the Attacker before it reaches the Target aircraft. Thus, a team is formed by the Target and the Defender which cooperate to maximize the separation between the Target aircraft and the point where the Attacker missile is intercepted by the Defender missile, while at the same time the Attacker tries to minimize said distance. A long-range Beyond Visual Range engagement which is in line with current CONcepts of OPeration is envisaged, and it is therefore assumed that the players have simple motion kinematics a la Isaacs. Also, the speed of the Attacker is equal to the speed of the Defender and the latter is interested in point capture. It is also assumed that at all time the Attacker is aware of the Defender’s position, i.e., it is a perfect information game. The analytic/closed-form solution of the target defense pursuit-evasion differential game delineates the state space region where the Attacker can reach the Target without being intercepted by the Defender, thus disposing of the Game of Kind. The target defense Game of Degree is played in the remaining state space. The analytic solution of the Game of Degree yields the agents’ optimal state feedback strategies, that is, the instantaneous heading angles for the Target and the Defender team to maximize the terminal separation between Target and Attacker at the instant of interception of the Attacker by the Defender, and also the instantaneous optimal heading for the Attacker to minimize said separation. Their calculation hinges on the real-time solution of a quartic equation. In this paper we contribute to the solution of a differential game with three states—an additional example to the, admittedly small, repertoire of pursuit-evasion differential games in 3-D which can be solved in closed form.

A Study of the Generalized Pontryagin Test Example from the Theory of Differential Games

A generalization of L.S.Pontryagin’s test example from the theory of differential games is considered. The study is based on Pontryagin’s first direct method, which was developed for the constructive solution of linear pursuit–evasion differential games of kind.

A Residual Gradient Fuzzy Reinforcement Learning Algorithm for Differential Games

In this work, we propose a new fuzzy reinforcement learning algorithm for differential games that have continuous state and action spaces. The proposed algorithm uses function approximation systems whose parameters are updated differently from the updating mechanisms used in the algorithms proposed in the literature. Unlike the algorithms presented in the literature which use the direct algorithms to update the parameters of their function approximation systems, the proposed algorithm uses the residual gradient value iteration algorithm to tune the input and output parameters of its function approximation systems. It has been shown in the literature that the direct algorithms may not converge to an answer in some cases, while the residual gradient algorithms are always guaranteed to converge to a local minimum. The proposed algorithm is called the residual gradient fuzzy actor–critic learning (RGFACL) algorithm. The proposed algorithm is used to learn three different pursuit–evasion differential games. Simulation results show that the performance of the proposed RGFACL algorithm outperforms the performance of the fuzzy actor–critic learning and the Q-learning fuzzy inference system algorithms in terms of convergence and speed of learning.

Erratum to: Zero-Sum Pursuit-Evasion Differential Games with Many Objects: Survey of Publications

Erratum to: Zero-Sum Pursuit-Evasion Differential Games with Many Objects: Survey of Publications

Zero-Sum Pursuit-Evasion Differential Games with Many Objects: Survey of Publications

If a pursuit game with many persons can be formalized in the framework of zero-sum differential games, then general methods can be applied to solve it. But difficulties arise connected with very high dimension of the phase vector when there are too many objects. Just due to this problem, special formulations and methods have been elaborated for conflict interaction of groups of objects. This paper is a survey of publications and results on group pursuit games.

A Decentralized Fuzzy Learning Algorithm for Pursuit-Evasion Differential Games with Superior Evaders

In this paper, we consider multi-pursuer single-superior-evader pursuit-evasion differential games where the evader has a speed that is similar to or higher than the speed of each pursuer. A new fuzzy reinforcement learning algorithm is proposed in this work. The proposed algorithm uses the well-known Apollonius circle mechanism to define the capture region of the learning pursuer based on its location and the location of the superior evader. The proposed algorithm uses the Apollonius circle with a developed formation control approach in the tuning mechanism of the fuzzy logic controller (FLC) of the learning pursuer so that one or some of the learning pursuers can capture the superior evader. The formation control mechanism used by the proposed algorithm guarantees that the pursuers are distributed around the superior evader in order to avoid collision between pursuers. The formation control mechanism used by the proposed algorithm also makes the Apollonius circles of each two adjacent pursuers intersect or be at least tangent to each other so that the capture of the superior evader can occur. The proposed algorithm is a decentralized algorithm as no communication among the pursuers is required. The only information the proposed algorithm requires is the position and the speed of the superior evader. The proposed algorithm is used to learn different multi-pursuer single-superior-evader pursuit-evasion differential games. The simulation results show the effectiveness of the proposed algorithm.

Nash strategies for pursuit-evasion differential games involving limited observations

A linear-quadratic N-pursuer single-evader differential game is considered. The evader can observe all the pursuers but pursuers have limited observations of themselves and the evader. The evader implements the conventional feedback Nash strategy and the pursuers implement Nash strategies based on a novel concept of best achievable performance indices. This problem has potential applications in situations where a well-equipped unmanned vehicle is evading several weakly equipped pursuing vehicles. An illustrative example is solved, and several scenarios are presented.

Q(λ)-learning adaptive fuzzy logic controllers for pursuit-evasion differential games

This paper addresses the problem of tuning the input and the output parameters of a fuzzy logic controller. A novel technique that combines Q(λ)-learning with function approximation (fuzzy inference system) is proposed. The system learns autonomously without supervision or a priori training data. The proposed technique is applied to three different pursuit–evasion differential games. The proposed technique is compared with the classical control strategy, Q(λ)-learning only, and the technique proposed by Dai et al. (2005) in which a neural network is used as a function approximation for Q-learning. Computer simulations show the usefulness of the proposed technique. Copyright © 2011 John Wiley & Sons, Ltd.

Self-learning fuzzy logic controllers for pursuit–evasion differential games

This paper addresses the problem of tuning the input and the output parameters of a fuzzy logic controller. The system learns autonomously without supervision or a priori training data. Two novel techniques are proposed. The first technique combines Q( λ )-learning with function approximation (fuzzy inference system) to tune the parameters of a fuzzy logic controller operating in continuous state and action spaces. The second technique combines Q( λ )-learning with genetic algorithms to tune the parameters of a fuzzy logic controller in the discrete state and action spaces. The proposed techniques are applied to different pursuit–evasion differential games. The proposed techniques are compared with the classical control strategy, Q( λ )-learning only, reward-based genetic algorithms learning, and with the technique proposed by Dai et al. (2005) [19] in which a neural network is used as a function approximation for Q-learning. Computer simulations show the usefulness of the proposed techniques.

On some problems of guaranteed search on graphs

Pursuit-evasion differential games on graphs with no information on the evader are considered. Special attention is given to the following ɛ-search problem posed by Golovach. A topological graph embedded in three-dimensional Euclidean space is considered. For simplicity, its edges are assumed to be polygonal, only simple motions of the pursuers and the evader are allowed, and the graph is a phase constraint for all players. The goal of the pursuers is to construct a program depending only on the structure of the graph which ensures capturing the invisible evader, i.e., approaching the evader at a distance of at most ɛ (in the intrinsic metric of the graph), where ɛ is a given nonnegative number. The problem consists in finding the minimum number of pursuers (called the ɛ-search number) needed to capture the evader. Properties of the Golovach function, which assigns the ɛ-search number to every nonnegative ɛ, are investigated. Golovach and Petrov proved that the Golovach function for the complete graph on more than five vertices may have non-unit jumps. The authors of this paper are aware of examples of similar degeneration for trees. These examples disprove the conjecture that the Golovach function of any planar graph has only unit jumps. A subclass of trees for which the Golovach function has only unit jumps is distinguished.

Game of Defending a Target: A Linear Quadratic Differential Game Approach

Pursuit-evasion (PE) differential games have recently received much attention in military applications involving adversaries. We extend the PE game problem to a problem of defending target, where the roles of the players are changed. The evader is to attack some fixed target, whereas the pursuer is to defend the target by intercepting the evader. We propose a practical strategy design approach based on the linear quadratic game theory with a receding horizon implementation. We prove the existence of solutions for the Riccati equations associated with games with simple dynamics. Simulation results justify the method.

Stochastic multi‐player pursuit–evasion differential games

AbstractAutonomous aerial vehicles play an important role in military applications such as in search, surveillance and reconnaissance. Multi‐player stochastic pursuit–evasion (PE) differential game is a natural model for such operations involving intelligent moving targets with uncertainties. In this paper, some fundamental issues of stochastic PE games are addressed. We first model a general stochastic multi‐player PE differential game with perfect state information. To avoid the difficulty of multiplicity of the players, we extend the iterative method for deterministic multi‐player PE games to the stochastic case. Starting from certain suboptimal solutions with an improving property, the optimization based on limited look‐ahead can be used for improvement. The process converges when this improvement is applied iteratively. Furthermore, we introduce a hierarchical approach that can determine a valid starting point of the iterative process. As a basis for multi‐player games, stochastic two‐player PE games are also addressed. We also briefly discuss the games with imperfect state information and propose a suboptimal approach from a practical point of view. Finally, we demonstrate the usefulness and the feasibility of the method through simulations. Copyright © 2007 John Wiley &amp; Sons, Ltd.

Hybrid control synthesis for eventuality specifications using level set methods

This paper is concerned with the extraction of controllers for hybrid systems with respect to eventuality specifications. Given a hybrid system modelled by a hybrid automaton and a target set of states, the objective is to compute the maximal set of initial states together with the hybrid control policy such that all the trajectories of the controlled system reach the target in finite time. Due to the existence of set-valued disturbance inputs, the problem is studied in a game-theoretic framework. Having shown that a least restrictive solution does not exist, we propose a dynamic programming algorithm that computes the maximal initial set and a controller with the desired property. To implement the algorithm, reachable sets of pursuit-evasion differential games need to be computed. For that reason level set methods are employed, where the boundary of the reachable set is characterized as the zero level set of a Hamilton–Jacobi equation. The procedure for the numerical extraction of the controller is presented in detail and examples illustrate the methodology. Finally, to demonstrate the practical character of our results, a control design problem in the benchmark system of the batch evaporator is considered as an eventuality synthesis problem and solved using the proposed methodology.