Optimal Feedback Strategy Research Articles

Optimality of index policies for a sequential sampling problem

Consider the following sequential sampling problem: at each time, a choice must be made between obtaining an independent sample from one of a set of random reward variables or stopping the sampling. Sampling a random variable incurs a random cost at each time. The objective of the problem is to maximize the expected nett difference between the largest sample reward obtained before stopping and the accumulated costs incurred while sampling. In this paper, the authors prove that the optimal feedback strategies for this problem are index policies and provide an explicit expression for the optimal expected reward from any state. The problem is motivated by search methods for global optimization problems where the cost of computation is explicitly incorporated into the objective.

Real-Time Midcourse Guidance Robust to Missile Launch Conditions

In this paper, an real-time suboptimal midcourse guidance law robust to missile launch conditions, which is a neural network approximation of the optimal feedback strategy, is proposed. It is obtained by the appropriate representation of input data of the neural network, which guarantees robustness. Computer simulations show that the proposed guidance law allows the missile to intercept a given target almost exactly with only a limited number of the neural network weights.

Real-time midcourse guidance with intercept point prediction

In this paper, an on-line suboptimal midcourse guidance law, which is a neural-network approximation of the optimal feedback strategy, is proposed to eliminate the need for solving two-point boundary-value problems in real time. Moreover, for intercept point prediction, a fast converging, iterative algorithm based on a neural network time-to-go estimator is devised. Computer simulations confirm that the closed-loop behavior of the proposed guidance law is so close to the optimal trajectory that it outperforms nonoptimal guidance laws such as g-biased proportional navigation.

Complex differential games of pursuit-evasion type with state constraints, part 2: Numerical computation of optimal open-loop strategies

In Part 1 of this paper (Ref. 1), necessary conditions for optimal open-loop strategies in differential games of pursuit-evasion type have been developed for problems which involve state variable inequality constraints and nonsmooth data. These necessary conditions lead to multipoint boundary-value problems with jump conditions. These problems can be solved very efficiently and accurately by the well-known multiple-shooting method. By this approach, optimal open-loop strategies and their associated saddle-point trajectories can be computed for the entire capture zone of the game. This also includes the computation of optimal open-loop strategies and saddle-point trajectories on the barrier of the pursuit-evasion game. The open-loop strategies provide an open-loop representation of the optimal feedback strategies. Numerical results are obtained for a special air combat scenario between one medium-range air-to-air missile and one high-performance aircraft in a vertical plane. A dynamic pressure limit for the aircraft imposes a state variable inequality constraint of the first order. Special emphasis is laid on realistic approximations of the lift, drag, and thrust of both vehicles and the atmospheric data. In particular, saddle-point trajectories on the barrier are computed and discussed. Submanifolds of the barrier which separate the initial values of the capture zone from those of the escape zone are computed for two representative launch positions of the missible. By this way, the firing range of the pursuing missile is determined and visualized.

New general guidance method in constrained optimal control, part 2: Application to space shuttle guidance

The application and the performance of the neighboring optimal feedback scheme presented in Part 1 of this paper is demonstrated for the heating-constrained cross-range maximization problem of a space-shuttle-orbiter-type vehicle. This problem contains five state variables, two control variables, and a state variable inequality constraint of order zero.

Optimal, Time Consistent Robust Feedback Rules Under Parameter, Forecast and Behavioural Uncertainty

The treatment of uncertainty in large macroeconometric models is usually based on a local linearization of the nonlinear model. In this paper, We deviate from this approach and use a robust policy concept that is more related to sensitivity analysis for nonlinear models than stochastics. In the linear case, the method does, nevertheless, reduce to the simultaneous minimization of the mean and variance of the stochastic problem.The method involves parameterized feedback laws. The parameters of these feedbacks are determined to minimize the original objective function simultaneously with a sensitivity (variance) expression. This expression can either be chosen as the sensitivity (variance) of the original policy objective function or the sensitivity (variance) of the individual endogenous variables. The latter possibility is, in particular, feasible because of the feedback strategy.The time-consistency of the feedback laws under perfect foresight is ensured by considering a Nash game formulation. However, time-consistency suggests the use of the full optimal feedback strategy rather than its aggregate parameterization. In this approach, the policy maker determines the feedback policy and the second player attempts to estimate the expectation of future endogenous variables.

Game of two cars: Case of variable speed

A stochastic version of Isaacs's (Ref. 1) game of two cars is dealt with here. In this version, the pursuer, owing to thrust and drag forces, has a variable speed, whereas the evader's speed is constant. Also, the pursuer can maneuver as long as his speed is bounded by some lower and upper limits. The probability of interception, corresponding to optimal (saddle-point) feedback strategies, is computed and serves as a reference for evaluating the performance of four different versions of the proportional navigation pursuit law as well as two other strategies.

Locally optimal feedback strategy for non-linear systems with uncertain parameters

This paper broadens the classical optimal control approach to the optimal regulation of a non-measurable, but parametrized uncertainty. A locally optimal feedback strategy (LOFS) is derived by minimizing a measure of the performance index sensitivity with respect to the uncertainty parameter vector. In the state feedback case the LOFS can be characterized by an extension of the well-known Matrix-Riccati equation. Conditions for stability and finiteness of the resulting linear feedback law are derived for the stationary LQ case with uncertain parameters. Different numerical algorithms also covering the output feedback case are presented. Examples demonstrate the superior performance of the proposed feedback for problems with a significant parametric uncertainty.

Stochastic pursuit-evasion differential games in 3D

A stochastic version of Isaacs's homicidal chauffeur game in the (x, y, z)-space is considered. This is used to solve a pursuit-evasion problem in the (x, y, z)-space in which the pursuer has incomplete information on the evader motion. Optimal feedback strategies for the game, and optimal feedback guidance laws for the pursuer, which uses only the measurements available to the pursuer, are computed. A simple suboptimal guidance law for the pursuer is suggested.

A stochastic pursuit-evasion differential game in 3-D: Optimal evasion strategies

A stochastic pursuit-evasion differential game involving two players moving in the ( x, y, z)-space is considered. It is assumed that the evader receives the location of the pursuer relative to a coordinate system attached to the evader and moving with it. Optimal feedback strategies for the game, and optimal feedback evasion laws for the evader, who uses only the measurements available to him, are computed. A simple optimal evasion law for the evader is suggested.

Stochastic pursuit-evasion differential games in the plane

Four stochastic pursuit-evasion differential games involving two players, P and E, moving in the plane are considered. The difference between the games lies in their information structures. In each of the games, sufficient conditions on optimal feedback strategies, in the cases of complete information, and on weak optimal feedback strategies, in the cases of incomplete information, are derived. Optimal strategies are computed for the cases of complete information and weak suboptimal strategies for the cases of incomplete information. The results indicate that the correct measurement of the direction of the segment PE is more important than the measurement of the distance (P, E).

Optimal control problem for an automatic transmission

Attention is given to the control-theoretic aspects of the operation of an automatic transmission for a road vehicle. A differential game model for vehicle control is presented and analyzed; furthermore, the approach leads to a synthesis of the optimal feedback strategy for the selection of the transmission ratio of the automatic gearbox.

Strategies using an observer for steering a random motion of a point in a multitarget environment

Consider the random motion of a point M in the plane, and suppose that the velocity υ = ( υ 1, υ 2) of M is perturbed by an R 2-valued Gaussian white noise. It is assumed that l targets are presented in the plane, located at the points c 1,…, c l. The observations available to the point M consist of a sum of measurements, each measurement corresponding to a detection that originated from a target, and all detections arriving at random times; plus an R 2-valued Gaussian white noise. Using current and past values of these observations, via the introduction of a state observer, the point M wishes, by choosing an appropriate pair (v ∗, g ∗) (where v ∗ is a velocity law and g ∗ is the observer's gain), to steer itself into an ε-neighbourhood of the point c 1 before it leaves an open and bounded domain which contains all the points c 1,…, c l. Sufficient conditions on (v, g)-weak optimal feedback strategies are derived. An algorithm for computing (v, g)-weak suboptimal strategies is suggested, and these are computed for a variety of cases.

Optimization model for a wolf-ungulate system

Public outcries against predator control create a need to devise management policies that optimally balance the cost (managerial and environmental) of predator control against the benefit of ungulate harvesting. To address this problem, an optimization procedure utilizing stochastic dynamic programming is described. Through this approach, optimal feedback strategies for a wolf-ungulate system in Alaska are estimated. The dynamic predator-prey model used in the analysis is based on parameter estimates from data collected over an eight-year period in Denali (Mt. McKinley) National Park. Stability analysis of the system revealed that stability properties depend on predator search efficiency. The effects of random fluctuations in winter severity and alternative objective functions are considered in the estimation of optimal feedback strategies. Optimal moose harvesting strategies appear to be dependent on wolf control costs. If no wolf control cost is assessed, optimal moose harvest is independent of wolf density. Optimal wolf control strategies are completely insensitive to moose density. The strategies are compared to current and simplified management policies.

Predictive stochastic feedforward-feedback control of a heat exchanger-stirred tank system

An optimal stochastic feedforward-feedback control scheme is implemented on a heat exchanger-stirred tank system using an on-line minicomputer. Because variations in the measured disturbance variable affect the process output (controlled variable) before the compensating action can become effective, the feedforward action must be predictive in nature. Statistical time series models are used to model both the measured and unmeasured disturbances in the system. These stochastic disturbance models and the transfer-function models for the process are identified, fitted and checked using statistical model-building procedures on a set of data collected on-line using the minicomputer. The predictive feedforward-feedback controller is derived from these models. The performance of this control scheme is compared with that of an optimal feedback scheme and the actual on-line performance is shown to conform well to the theoretical predictions

Sufficient conditions for optimality in two-person zero-sum differential games with state and strategy constraints

A sufficiency theorem for optimal feedback strategies in two-person zero-sum differential games is given. The theorem is applicable to a wide class of such games for which strategies are Borel measurable functions on a subset of the state space. The theorem generalizes those of [1, 2, and 5].