Pursuit-evasion Problem Research Articles

A two-phase evasive strategy for a pursuit-evasion problem involving two non-holonomic agents with incomplete information

In this paper, we consider the problem of pursuit-evasion between two non-holonomic agents. To make the problem more realistic, we assume that the pursuer and evader have incomplete information regarding each other’s motion. We propose a novel, two-phase, evasive strategy against a higher speed pursuer, based on worst-case scenario planning and proximity-based maneuver. In the first phase, the evader assumes the pursuer to have zero turn radius and executes a best-response strategy, which is analytically shown to be movement along a tangent to the turn circle of the evader. In the second phase, when the pursuer gets close to the evader, the latter switches to movement along high-curvature paths to side-step the former. The effectiveness of the two phases is shown through simulations.

Near-optimal interception strategy for orbital pursuit-evasion using deep reinforcement learning

Two participants with competing objectives are considered in one-to-one orbital pursuit-evasion problem, one of which attempts to satisfy some certain terminal conditions with a minimum cost through its own control while the other player attempts to maximize the specified payoff and avoids entering this terminal range. This paper treats the problem of minimax time to a given final distance, within which the interception between pursuer and evader is defined to occur. To achieve the successful interception with an evader that adopts any unknown maneuvers, a near-optimal interception strategy for the orbital pursuit-evasion is presented in this paper. The pursuit-evasion problem is formulated as a differential game first, and a closed-form barrier solution is provided next. A near-optimal guidance law using deep learning is proposed to intercept the evader inside the capture zone. For the games that start outside the barrier, a learning algorithm for the capture zone embedding strategy is presented based on deep reinforcement learning to help the game state cross the barrier surfaces. The simulation cases of intercepting the evaders that adopt different maneuvering strategies are provided, and the results demonstrate the effectiveness and onboard feasibility of the proposed strategy in solving the orbital interception problem.

Cheap Control in a Non-Scalarizable Linear-Quadratic Pursuit-Evasion Game: Asymptotic Analysis

In this work, a finite-horizon zero-sum linear-quadratic differential game, modeling a pursuit-evasion problem, was considered. In the game’s cost function, the cost of the control of the minimizing player (the minimizer/the pursuer) was much smaller than the cost of the control of the maximizing player (the maximizer/the evader) and the cost of the state variable. This smallness was expressed by a positive small multiplier (a small parameter) of the square of the L2-norm of the minimizer’s control in the cost function. Parameter-free sufficient conditions for the existence of the game’s solution (the players’ optimal state-feedback controls and the game value), valid for all sufficiently small values of the parameter, were presented. The boundedness (with respect to the small parameter) of the time realizations of the optimal state-feedback controls along the corresponding game’s trajectory was established. The best achievable game value from the minimizer’s viewpoint was derived. A relation between solutions of the original cheap control game and the game that was obtained from the original one by replacing the small minimizer’s control cost with zero, was established. An illustrative real-life example is presented.

Optimal Vehicle-Target Assignment: A Swarm of Pursuers to Intercept Maneuvering Evaders Based on Ideal Proportional Navigation

The problem of vehicle-target assignment (VTA) to capture a team of evading targets using a swarm of pursuing vehicles is investigated in this article. The VTA problem is formulated as an integer linear programming (ILP), such that the time to intercept all the targets is minimized subject to a number of constraints. To obtain closed-form formulas for the time-to-go matrix in the framework of ILP optimization, a one-on-one pursuit-evasion problem based on the ideal proportional navigation (IPN) guidance law is investigated. By considering two different scenarios of non-maneuvering and maneuvering evaders, analytical closed-form solutions for the pursuit-evasion time-to-go as explicit functions of the position and velocity vectors of the pursuers and evaders are developed, and efficient evasion strategies based on IPN guidance scheme are presented. The efficacy of the theoretical results in estimating the elements of time-to-go matrix is demonstrated by solving the VTA problem in simulations.

The Pursuit–Evasion problems in a differential game with GGr-constraints on controls

The Pursuit–Evasion problems in a differential game with GGr-constraints on controls

DIFFERENTIAL GAME WITH A LIFELINE FOR THE INERTIAL MOVEMENTS OF PLAYERS

In this paper, we study the well-known problem of Isaacs called the "Life line" game when movements of players occur by acceleration vectors, that is, by inertia in Euclidean space. To solve this problem, we investigate the dynamics of the attainability domain of an evader through finding solvability conditions of the pursuit-evasion problems in favor of a pursuer or an evader. Here a pursuit problem is solved by a parallel pursuit strategy. To solve an evasion problem, we propose a strategy for the evader and show that the evasion is possible from given initial positions of players. Note that this work develops and continues studies of Isaacs, Petrosjan, Pshenichnii, Azamov, and others performed for the case of players' movements without inertia.

Pursuer Assignment and Control Strategies in Multi-Agent Pursuit-Evasion Under Uncertainties.

We consider a pursuit-evasion problem with a heterogeneous team of multiple pursuers and multiple evaders. Although both the pursuers and the evaders are aware of each others’ control and assignment strategies, they do not have exact information about the other type of agents’ location or action. Using only noisy on-board sensors the pursuers (or evaders) make probabilistic estimation of positions of the evaders (or pursuers). Each type of agent use Markov localization to update the probability distribution of the other type. A search-based control strategy is developed for the pursuers that intrinsically takes the probability distribution of the evaders into account. Pursuers are assigned using an assignment algorithm that takes redundancy (i.e., an excess in the number of pursuers than the number of evaders) into account, such that the total or maximum estimated time to capture the evaders is minimized. In this respect we assume the pursuers to have clear advantage over the evaders. However, the objective of this work is to use assignment strategies that minimize the capture time. This assignment strategy is based on a modified Hungarian algorithm as well as a novel algorithm for determining assignment of redundant pursuers. The evaders, in order to effectively avoid the pursuers, predict the assignment based on their probabilistic knowledge of the pursuers and use a control strategy to actively move away from those pursues. Our experimental evaluation shows that the redundant assignment algorithm performs better than an alternative nearest-neighbor based assignment algorithm1.

Minmax fuzzy deterministic policy gradient for zero-sum differential game: Take pursuit-evasion problem as example

A novel actor-critic algorithm is introduced and applied to zero-sum differential game. The proposed novel structure consists of two actors and a critic. Different actors represent the control policies of different players, and the critic is used to approximate the state-action utility function. Instead of neural network, the fuzzy inference system is applied as approximators for the actors and critic so that the specific practical meaning can be represented by the linguistic fuzzy rules. Since the goals of the players in the game are completely opposite, the actors for different players are simultaneously updated in opposite directions during the training. One actor is updated updated toward the direction that can minimize the Q value while the other updated toward the direction that can maximize the Q value. A pursuit-evasion problem with two pursuers and one evader is taken as an example to illustrate the validity of our method. In this problem, the two pursuers the same actor and the symmetry in the problem is used to improve the replay buffer. At the end of this paper, some confrontations between the policies with different training episodes are conducted.

A game problem of pursuit-evasion with nth order differential equations describing players' dynamics

In this paper, we study a pursuit-evasion differential game problem in the Hilbert space Dynamics of countable number of pursuers and evader expressed as nth-order differential equations with geometric constraints on the control functions of the players. The game terminates at a given fixed time which is denoted by ϑ. The game's payoff is the infimum of the distances between the evader and pursuers at the time ϑ. According to the rule of the game, pursuers try to minimise the distance to the evader and the evader tries to maximises it. We found value of the game and constructed players' optimal strategies.

Compensation Control Strategy for Orbital Pursuit-Evasion Problem with Imperfect Information

This paper studies the orbital pursuit-evasion problem with imperfect information, including measurement noise and input delay. The presence of imperfect information will degrade the players’ control performance and lead to mission failure. To solve this problem, a compensation control strategy for the players is proposed. The compensation control strategy consists of two parts: the guaranteed cost strategy and the time delay compensation method. First, a near-optimal feedback strategy called guaranteed cost strategy with perfect information is proposed based on a Lyapunov-like function and matrix analysis theory. Second, a time delay compensation method based on an uncertainty set is proposed to compensate for delayed information. The compensation control strategy is derived by combining the time delay compensation method with the guaranteed cost strategy. While applying this strategy to the game, the input of the strategy is generated by processing the measured data with the state estimation algorithm based on the unscented Kalman filter (UKF). The simulation results show that the proposed strategy can handle the orbital pursuit-evasion problem with imperfect information effectively.

Game tree search for minimizing detectability and maximizing visibility

We introduce and study the problem of planning a trajectory for an agent to carry out a scouting mission while avoiding being detected by an adversarial opponent. This introduces a multi-objective version of classical visibility-based target search and pursuit-evasion problem. In our formulation, the agent receives a positive reward for increasing its visibility (by exploring new regions) and a negative penalty every time it is detected by the opponent. The objective is to find a finite-horizon path for the agent that balances the trade off between maximizing visibility and minimizing detectability. We model this problem as a discrete, sequential, two-player, zero-sum game. We use two types of game tree search algorithms to solve this problem: minimax search tree and Monte-Carlo search tree. Both search trees can yield the optimal policy but may require possibly exponential computational time and space. We first propose three pruning techniques to reduce the computational time while preserving optimality guarantees. When the agent and the opponent are located far from each other initially, we present a variable resolution technique with longer planning horizon to further reduce computational time. Simulation results show the effectiveness of the proposed strategies in terms of computational time.

Blocking an Evader in a Polygon by a 2-Modem Searcher

In this paper, we introduce a version of Pursuit-Evasion problem in which the pursuer is a 2-modem which pursues an unpredictable evader in a polygonal environment. A 2-modem searcher is a wireless device whose radio signal can penetrate two walls. Furthermore, a query path inside the polygon is given so that the 2-modem moves on it. We want to find out that when the 2- modem moves on this path if evaders can move from an invisible region to another one without detecting by the pursuer or not. In the other word, we want to find out if the invisible regions merge together or not.

Linear quadratic Gaussian Stackelberg game under asymmetric information patterns

This paper investigates the linear quadratic Gaussian Stackelberg game under asymmetric information. Two decision makers implement control strategies using different information patterns: The follower uses its observation information to design its strategy, whereas the leader implements its strategy using global information. For this problem, the separation principle becomes invalid. Instead, we apply the classical variation method to derive conditions to be satisfied by both decision makers. We show that the solution attributes to solving a two-point boundary value problem of stochastic version whose drift terms contain some conditional mean terms. We then propose a layered calculation method to solve this problem. In the inner layer calculation, the adjoint state variable’s estimate is expressed as a linear functional of the state estimate. In the outer layer calculation, the adjoint state variable is expressed as a linear functional of the state and its estimate. We also verify that the Kalman–Bucy filter works for computing the state estimate. The result is also applied to dynamic duopoly with sticky prices and pursuit–evasion problem.

Clearing an orthogonal polygon to find the evaders

In a multi-robot system, a number of autonomous robots would sense, communicate, and decide to move within a given domain to achieve a common goal. In the pursuit-evasion problem, a polygonal region is given and a robot called a pursuer tries to find some mobile targets called evaders. The goal of this problem is to design a motion strategy for the pursuer such that it can detect all the evaders. In this paper, we consider a new variant of the pursuit-evasion problem in which the robots (pursuers) each moves back and forth along an orthogonal line segment inside a simple orthogonal polygon P. We assume that P includes unpredictable, moving evaders that have bounded speed. We propose the first motion-planning algorithm for a group of robots, assuming that they move along the pre-located line segments with a constant speed to detect all the evaders with bounded speed. Also, we prove an upper bound for the length of the paths that all pursuers move in the proposed algorithm.

Distributed multi‐agent deep reinforcement learning for cooperative multi‐robot pursuit

As a popular research topic in the area of distributed artificial intelligence, the multi-robot pursuit problem is widely used as a testbed for evaluating coordinated and cooperative strategies in multi-robot systems. This study the problem of multi-robot pursuit game using reinforcement learning (RL) techniques is studied. Unlike most existing studies that apply fully centralised deep RL methods based on the centralised-learning and decentralised-execution scheme, the authors propose a fully decentralised multi-agent deep RL approach by modelling each agent as an individual deep RL agent that has its own individual learning system (i.e. individual action-value function, individual leaning update process, and individual action output). To realise coordination among agents, the limited information of other environmental agents is used as input of the learning process. Experimental results show that both distributed and centralised approaches can ultimately solve the pursuit-evasion problem in different dimensions, but the learning efficiency and coordination performance of the proposed distributed approach are much better than the traditional centralised approach.

A pursuit–evasion game between two identical differential drive robots

This paper addresses a pursuit-evasion problem between two identical Differential Drive Robots (DDRs). The pursuer wants to capture the evader in minimal time, while the evader wants to delay capture as much as possible. The game takes place in a Euclidean plane without obstacles. In this work, the motion primitives and time-optimal motion strategies for both players are presented. Based on the initial positions of the players, it is possible to solve the decision problem of determining the winner of the game. Simulations of the pursuit-evasion game showing the time-optimal motion primitives of the players are provided for both cases, when the pursuer wins and when the evader escapes.

Enhanced Social Spider Optimization Algorithm for Increasing Performance of Multiple Pursuer Drones in Neutralizing Attacks From Multiple Evader Drones

We propose an optimization algorithm for reducing execution time needed by multiple pursuers in solving a variant of the Multiple-Pursuer Multiple-Evader (MPME) problem where each evader tries to attack an area defended by pursuers. This problem is a variant of the Multi-Agent Pursuit Evasion problem. In our discussed problem, a group of pursuers tries to defend an area from a group of evaders’ attacks. The main task given in this problem is how pursuers can capture or immobilize as soon as possible any evader trying to get closer to the defended area (evaders’ target). We use Social Spider Optimization (SSO) algorithm as the basis of our proposed method. In SSO, there are female spiders, dominant-male spiders, and non-dominant-male spiders collaborating to catch their prey. In SSO, there are three main procedures usually exist: calculation of fitness value, the vibrational summons of surrounding spiders, and mating procedure. In this paper, we develop an enhanced SSO algorithm where excludes the mating procedure and propose a practical calculation process for solving our discussed problem. SSO is one of the recent optimization algorithms developed in the computer science field. Developing this algorithm for solving dynamic problem like the MPME variant surely brings a novelty in the computer science research area. We test our proposed method in a 3D simulation environment where we manifest all pursuers and evaders as drones. Based on our experiment result, our algorithm performs better than commonly used methods for solving the MPME problem.

Smarter Lions: Efficient Cooperative Pursuit in General Bounded Arenas

We study a full-knowledge pursuit-evasion problem where cooperating pursuers attempt to capture a single evader of equal (and without loss of generality, unit) speed in a closed, bounded, two-dimen...

PN Modified Differential Game Strategy for Two-on-one Cooperative Interception

In order to deal with the Chaff, Infrared Decoy, Electronic Jamming and the escaping maneuvering of the target, two-on-one cooperative interception strategy is discussed in this paper. The planar engagement of two air-to-air missiles cooperative intercepting one target by the alliance of differential game (DG) strategy and proportional navigation (PN) guidance law is investigated. The equations of motion for two-on-one cooperative intercepting are derived first, and zero-effort-miss (ZEM) representation technique is employed. Then a linear quadratic differential game cooperative zero-sum cost function is phrased, and the cooperative control variables of two-on-one cooperative guidance law under pure DG strategy are derivate. Finally, the proportional navigation guidance law is introduced to alleviate the acceleration command oscillation. The simulation results in the ideal dynamic scenarios show that, the combination of DG and PN guidance law is effective to the two-on-one cooperative pursuit-evasion problem, especially for the high maneuvering target at the cost of much more control effort.

On-line Search in Two-Dimensional Environment

We consider the following on-line pursuit-evasion problem. A team of mobile agents called searchers starts at an arbitrary node of an unknown network. Their goal is to execute a search strategy that guarantees capturing a fast and invisible intruder regardless of its movements using as few searchers as possible. We require that the strategy is connected and monotone, that is, at each point of the execution the part of the graph that is guaranteed to be free of the fugitive is connected and whenever some node gains a property that it cannot be occupied by the fugitive, the strategy must operate in such a way to keep this property till its end. As a way of modeling two-dimensional shapes, we restrict our attention to networks that are embedded into partial grids: nodes are placed on the plane at integer coordinates and only nodes at distance one can be adjacent. Agents do not have any knowledge about the graph a priori, but they recognize the direction of the incident edge (up, down, left or right). We give an on-line algorithm for the searchers that allows them to compute a connected and monotone strategy that guarantees searching any unknown partial grid with the use of O(sqrt {n}) searchers, where n is the number of nodes in the grid. As for a lower bound, there exist partial grids that require {varOmega }(sqrt {n}) searchers. Moreover, we prove that for each on-line searching algorithm there is a partial grid that forces the algorithm to use {varOmega }(sqrt {n}) searchers but O(log n) searchers are sufficient in the off-line scenario. This gives a lower bound on {varOmega }(sqrt {n}/log n) in terms of achievable competitive ratio of any on-line algorithm.