Nonlinear Differential Games Research Articles

Formation Control with Feedback Nash Strategy for Multiple Unmanned Aerial Vehicles in Unknown Environments

In the paper, the formation control problem in unknown environments for networked multi-unmanned aerial vehicle systems (UAVSs) is resolved under a nonlinear differential game (NL-DDG) framework. The main challenge of this framework is how to obtain feedback Nash strategies, which typically overly rely on global information and cannot ensure the existence of Nash strategies in unknown environments. Toward this goal, we initially design collision avoidance rules to ensure the safety of each UAV. Subsequently, we utilize an inverse optimal control method to construct the NL-DDG that incorporates both formation control and collision avoidance costs, enabling the derivation of analytical forms of Nash strategies relying solely on local information and minimizing performance metrics. In addition, the existence of feedback Nash strategy can be guaranteed with an undirected and connected information topology, which represents the optimality for the UAVSs. Moreover, we analyze the stability of the closed-loop system. Finally, the simulation results validate the effectiveness of the proposed scheme.

Dynamic optimization of nonlinear differential game problems using orthogonal collocation with analytical sensitivities

AbstractThis article presents an effective computational method based on the orthogonal collocation on finite element for nonlinear pursuit‐evasion differential game problems. The original problems are transformed into two dynamic optimization problems at first, so that the difficulty of obtaining the solution is reduced. To improve the convergence rate and the efficiency, the sensitivities describing the influence of control and interval parameters on state are derived through the discretized dynamic equations for the resulting nonlinear programming problem. The convergence speed is introduced to measure the performance in the upper level iteration. The main structure and the algorithm of the method are also given. Two demonstrative differential game problems with different scenarios from practice are studied. Compared with the approach without sensitivity information, the proposed method needs less function evaluations and saves at least 68.4% of the computational time. The research results show the effectiveness of proposed approach.

Two-player nonlinear Stackelberg differential game via off-policy integral reinforcement learning

For the nonzero-sum games, players have taken different strategies to achieve the Nash equilibrium. However, regarding hierarchical optimization and asymmetric information, Nash equilibrium is ineffective. The Stackelberg game provides us with a leader–follower strategy to cope with this problem. This paper solved two-player continuous-time nonlinear Stackelberg differential games with unknown system dynamics using an off-policy integral reinforcement learning (IRL) technique. A two-level hierarchy optimal control problem is the Stackelberg differential game, and locating the open-loop Stackelberg equilibrium is equivalent to solving the coupled Hamiltonian-Jacobi (HJ) equations. First, the optimal strategies for the leader and the follower are derived. Then, the closed-loop system’s asymptotic stability of optimal control solution is proved. Based on the policy iteration (PI) algorithm, an off-policy IRL algorithm is developed to solve the Stackelberg game for completely unknown system dynamics. The control strategies and cost functions of the leader and follower are approximated using neural networks (NNs). It is demonstrated that NNs weight errors of off-policy algorithm are uniformly ultimately bounded (UUB). Lastly, two simulation examples demonstrate that the proposed learning algorithm is capable of obtaining two-player Stackelberg equilibrium strategies.

Guidance strategy of motion camouflage for spacecraft pursuit-evasion game

This work is inspired by a stealth pursuit behavior called motion camouflage whereby a pursuer approaches an evader while the pursuer camouflages itself against a predetermined background. We formulate the spacecraft pursuit-evasion problem as a stealth pursuit strategy of motion camouflage, in which the pursuer tries to minimize a motion camouflage index defined in this paper. The Euler-Hill reference frame whose origin is set on the circular reference orbit is used to describe the dynamics. Based on the rule of motion camouflage, a guidance strategy in open-loop form to achieve motion camouflage index is derived in which the pursuer lies on the camouflage constraint line connecting the central spacecraft and evader. In order to dispose of the dependence on the evader acceleration in the open-loop guidance strategy, we further consider the motion camouflage pursuit problem within an infinite-horizon nonlinear quadratic differential game. The saddle point solution to the game is derived by using the state-dependent Riccati equation method, and the resulting closed-loop guidance strategy is effective in achieving motion camouflage. Simulations are performed to demonstrate the capabilities of the proposed guidance strategies for the pursuit–evasion game scenario.

Nonlinear Multi-Object Differential Game Simulation Model in LabVIEW

This article presents the synthesis of a nonlinear multi-object differential game model in relation to the process of safe ship control in collision situations at sea. Nonlinear dynamic equations of a target ship and linear kinematic equations of passing ships were used to formulate the game state equations. The model of such a differential game was developed using LabVIEW 2022 version software. This was then subjected to simulation tests using the example of a navigational situation in which the target ship passed three encountered ships at a safe distance under the conditions of non-cooperation of ships, their cooperation, and optimal non-game control. The results of the computer simulation are presented in the form of ship trajectories and time courses of individual game control variables. The distinguishing feature of the model built in LabVIEW software is the ability to conduct research in online mode, where the user has the opportunity to track the impact of changes in the model parameters on the course of the differential game simulation on an ongoing basis. Further refinements of the simulation model should concern the larger number of ships and test the sensitivity of the game control quality to inaccuracies in the measured state variables and to changes in the parameters of the ship’s dynamics.

Zero-sum differential game guidance law for missile interception engagement via neuro-dynamic programming

In this paper, for solving the nonlinear control problem of the missile intercepting the maneuvering target, a novel nonlinear zero-sum differential game guidance law is proposed via the neuro-dynamic programming approach. First, the continuous-time nonlinear differential game problem is transformed into solving the nonlinear Hamilton–Jacobi–Isaacs (HJI) equation. Then, a critic neural network is designed to solve the corresponding nonlinear HJI equation. An adaptive weight tuning law for the critic weights is proposed, where an additional term is added to ensure the stability of the closed-loop nonlinear system. Furthermore, the uniform ultimate boundedness of the closed-loop system and the critic NN weights estimation error are proved with the Lyapunov approach. Finally, some simulation results are presented to demonstrate the effectiveness of the proposed differential game guidance law for nonlinear interception.

One-Sided Capture in Nonlinear Differential Games

A differential game described by a nonlinear system of differential equations is considered in a finite-dimensional Euclidean space. The value set of the pursuer control is a finite set. The value set of the evader control is a compact set. The purpose of the pursuer is a translation of the system in a finite time to any given neighborhood of zero. The pursuer uses a piecewise open-loop strategy constructed only by using information on the state coordinates and the velocity in the partition points of a time interval. In the past work, sufficient conditions were obtained for existence of a neighborhood of zero from which the capture occurs. The statement of the capture theorem contains such a condition that some vectors set up a positive basis. In this research, we consider the case when these vectors set up a one-sided set. For this case, sufficient conditions are obtained for existence of a set of initial position, from which the capture occurs.

Relative optimality in nonlinear differential games with discrete control

Рассматриваются две задачи управления с помехой, в качестве которой выступает второй игрок в дифференциальной игре. Динамика первой задачи описывается нелинейной системой дифференциальных уравнений первого порядка, динамика второй - нелинейной системой дифференциальных уравнений второго порядка. Управление осуществляется посредством кусочно постоянного управления, множество значений которого является конечным. Целью управления является движение сколь угодно близко к конечной траектории, описываемой вспомогательной системой управления простого вида, при любых действиях помехи. В обеих задачах получены фазовые ограничения на вспомогательную систему, в рамках которых управление вспомогательной системы может быть любым. Для любой окрестности и произвольного управления вспомогательной системы, которое удовлетворяет полученным ограничениям, в исходных задачах существуют допустимые управления, обеспечивающие в каждый момент времени нахождение фазовой точки исходной системы в указанной окрестности соответствующей фазовой точки вспомогательной системы. Таким образом, с учетом полученных ограничений, выбирая управление вспомогательной системы оптимальным в каком-либо смысле, можно осуществить сколь угодно близкое движение исходной системы к такому решению вспомогательной системы при любых действиях помехи. Библиография: 29 названий.

Relative optimality in nonlinear differential games with discrete control

Two control problems with an obstacle that is the second player in a differential game are considered. The dynamics in the first problem is described by a nonlinear system of differential equations of the first order, whereas the dynamics in the second is described by a nonlinear system of differential equations of the second order. A piecewise constant control with finite set of values is used. The control is aimed at moving arbitrarily closely to a finite trajectory described by an auxiliary control system of simple form, for any actions of the obstacle. For both problems phase constraints on the auxiliary system under which the control of the auxiliary system can be arbitrary are obtained. For any neighbourhood and any control of the auxiliary system satisfying these constraints, there are admissible controls in the original problems ensuring that at each moment of time the phase point of the original system is in the indicated neighbourhood of the corresponding phase point of the auxiliary system. Thus, in view of the above constraints, when the control of the auxiliary system is chosen to be optimal in a certain sense, the original system can move arbitrarily closely to such a solution of the auxiliary system for any actions of the obstacle. Bibliography: 29 titles.

A Nonlinear Finite-Time Robust Differential Game Guidance Law.

In this paper, a robust differential game guidance law is proposed for the nonlinear zero-sum system with unknown dynamics and external disturbances. First, the continuous-time nonlinear zero-sum differential game problem is transformed into solving the nonlinear Hamilton–Jacobi–Isaacs equation, a time-varying cost function is developed to reflect the fixed terminal time, and the robust guidance law is developed to compensate for the external disturbance. Then, a novel neural network identifier is designed to approximate the unknown nonlinear dynamics with online weight tuning. Subsequently, an online critic neural network approximator is presented to estimate the cost function, and time-varying activation functions are considered to deal with the fixed final time problem. An adaptive weight tuning law is given, where two additional terms are added to ensure the stability of the closed-loop nonlinear system and so as to meet the terminal cost at a fixed final time. Furthermore, the uniform ultimate boundedness of the closed-loop system and the critic neural network weights estimation error are proven based upon the Lyapunov approach. Finally, some simulation results are presented to demonstrate the effectiveness of the proposed robust differential game guidance law for nonlinear interception.

On the analysis of open-loop Nash equilibria admitting a feedback synthesis in nonlinear differential games

Open-loop Nash equilibrium strategies for differential games described by nonlinear, input-affine, systems and cost functionals that are quadratic with respect to the control input are studied. First it is shown that the computation of such strategies hinges upon the solution of a system of nonlinear, time-varying, partial differential equations (PDEs) obtained by building on arguments borrowed from Pontryagin’s Minimum Principle and combined with Dynamic Programming considerations. Then, by relying on a state/costate interpretation of the above characterization, a feedback synthesis of the underlying open-loop strategy is obtained by solving linear first-order PDEs that ensure invariance of certain submanifolds in the state–space of the extended state/costate dynamics. These PDEs are the nonlinear counterpart of the well-known asymmetric Algebraic Riccati Equations arising in the study of linear quadratic Nash games.

A numerical iterative method for solving two-point BVPs in infinite-horizon nonzero-sum differential games: Economic applications

In this work, the Nash equilibrium solution of nonlinear infinite-horizon nonzero-sum differential games with open-loop information is investigated numerically. For this class of games, some difficulties are involved in finding an open-loop Nash equilibrium. For instance, we must solve a nonlinear differential equations system with split boundary conditions, namely the two-point boundary value problem (TPBVP), in which some of the boundary conditions are specified at the initial time and some at the infinite final time. In the current study, we provide a combined numerical algorithm based on a new set of basis functions on the half-line, called the exponential Chelyshkov functions (ECFs), and quasilinearization (QL) method to solve TPBVPs. In the first step, we reduce the nonlinear TPBVP to a sequence of linear differential equations by using the QL method. Although we have now a linearized system, another difficulty is finding the approximate solution of this linear system such that the transversality conditions at the infinite final time are satisfied. So, in the second step, we apply a collocation method based on the ECFs to solve the obtained linear system in the semi-infinite domain. The convergence of the proposed method is discussed in detail. To confirm the validity and efficiency of the proposed scheme, we compute the approximate solution of TPBVP as well as the open-loop Nash equilibrium for four applications of differential games in economics and management science.

Predictive Differential Game Guidance Approach for Hypersonic Target Interception Based on CQPSO

In the descent phase of the hypersonic target flight trajectory, the hypersonic speed of the target makes the reaction time shorter. Due to overload restrictions, it is challenging for the interceptor to achieve successful interception. To address this problem, a predictive differential game guidance approach based on dynamic optimization algorithm is proposed to relax the interceptor acceleration requirements. Two-dimensional kinematics and dynamics engagement mode for hypersonic target interception is formulated as a nonlinear differential control model in the presence of matched uncertainties. The nonlinear differential model is transformed into a differential game model by combining a performance index. The performance index is positively correlated with the Line-Of-Sight (LOS) rate and control effort of interceptor and negatively correlated with the target maneuver. The Chaotic Quantum Particle Swarm Optimization (CQPSO) algorithm is utilized to address the nonlinear differential game problem to generate the instantaneous and predicted guidance commands for the interceptor and target. Numerical examples are given to verify the performance of the proposed guidance law in various engagement scenarios, including different target maneuvers, initial heading angles and target overload. Moreover, the performance of the algorithm is validated comparing with traditional and state-of-the-art guidance laws.

An Evader Control Strategy in the Non-Linear Differential Game Problem with Terminal Limitations

An Evader Control Strategy in the Non-Linear Differential Game Problem with Terminal Limitations

Estimate of the Capture Time and Construction of the Pursuer’s Strategy in a Nonlinear Two-Person Differential Game

Estimate of the Capture Time and Construction of the Pursuer’s Strategy in a Nonlinear Two-Person Differential Game

Informational redefinition of the differential game and information dualism: a look at the principles

The article discusses the informational aspects of the nonlinear differential game “pursuit-evasion”. The mini- max quality criterion (the price of the game), mathematical models of the movement of the same type of players are set. The principle of information redefinition by Yu.B. Hermeyer and the principle of information dualism are considered. The common features of the principles and the differences between them are revealed. Understanding this makes it easier to solve game problems and simplifies practical implementation issues.

On properties of one functional used in software constructions for solving differential games

Nonlinear differential game (DG) is investigated; relaxations of the game problem of guidance are investigated also. The variant of the program iterations method realized in the space of position functions and delivering in limit the value function of the minimax-maximin DG for special functionals of a trajectory is considered. For every game position, this limit function realizes the least size of the target set neighborhood for which, under proportional weakening of phase constraints, the player interested in a guidance yet guarantees its realization. Properties of above-mentioned functionals and limit function are investigated. In particular, sufficient conditions for realization of values of given function under fulfilment of finite iteration number are obtained.

Nonlinear differential game “pursuit-evasion”: information aspect.

The article deals with the information aspects of the nonlinear differential game “pursuit-evasion”. The mini- max quality criterion (the price of the game), mathematical models of the movement of the same type of players are set. The main problem that arises in the practical consideration of a nonlinear differential game is the solution of a boundary value problem. It is proved that: 1) a differential game can be represented as an envelope of a family of singular curves (“instantaneous solutions”) constructed from each of its points, 2) the control of the players can be found on this family. The method of constructing instant solutions is justified. The principle of information dualism in the differential game “pursuit-evasion” is proved. Its use facilitates the solution of information and computational problems that arise during the implementation of the game.

$$\varepsilon $$-Capture in Nonlinear Differential Games Described by System of Order Two

$$\varepsilon $$-Capture in Nonlinear Differential Games Described by System of Order Two

Minimization of energy costs for UAV management in a conflict task

This article considers the method of developing an evader control strategy in the non-linear differential pursuit-evasion game problem. It is assumed that the pursuer resorts to the most probable control strategy in order to capture the evader and that at each moment the evader knows its own and the enemy’s physical capabilities. This assumption allows to bring the game problem down to the problem of a unilateral evader control, with the condition of reaching a saddle point not obligatory to be fulfilled. The control is realised in the form of synthesis and additionally ensures that the requirements for bringing the evader to a specified area with terminal optimization of certain state variables are satisfiedt. The solution of this problem will significantly reduce the energy losses for controlling an unmanned vehicle, the possible effect is to save 15-20 % of fuel with a probability of 0.98, to solve the problem of chasing the enemy.