State Variable Constraints Research Articles

Sufficient Conditions in Optimal Control Theory

During the last ten years or so a large number of papers in professional journals in economics dealing with dynamic optimization problems have been employing the modern version of the calculus of variations called optimal control theory. The central result in the theory is the well known Pontryagin maximum principle providing necessary conditions for optimality in very general dynamic optimization problems. These conditions are not, in general, sufficient for optimality. Of course, if an existence theorem can be applied guaranteeing that a solution exists, then by comparing all the candidates for optimality that the necessary conditions produce, we can in principle pick out an optimal solution to the problem. In several cases, however, there is a more convenient method that can be used. Suppose that a solution candidate suggests itself through an application of the necessary conditions, or possibly also by a process of informed guessing. Then, if we can prove that the solution satisfies suLfficiency conditions of the type considered in this paper, then these conditions will ensure the optimality of the solution. In such a case we need not go through the process of finding all the candidates for optimality, comparing them and finally appealing to all existence thieorem. In ordcle to get all idea of what types of conditions might be involved in such sufficiency theorems, it is natural to look at the corresponding problem in static optimization. Here it is well known that the first order calculus or Kulhn-TLcker conditions are sU11ficient for optimality, provided suitable concavity/convexity conditions are imposed on the functions involved. It is natural to expect that similar conditions might secure sufficiency also in dynamic optimization problems. Growth theorists were early aware of this and proofs of sufficienccy in particular problems were constructed; see, e.g., Uzawa's 1964 paper [19]. In the mathematical literature few and only rather special results were available until Mangasarian. in a 1966 paper [10] proved a rather general sufficiency theorem in which he was dealing with a nonlinear system, state and control variable constraints and a fixed time interval. In the maximization case, when there are no state space constraints, his result was, essentially, that the Pontryagin necessary conditions plus concavity of the Hamiltonian function with respect to the state and control variables, were sufficient for optimality. The Mangasarian concavity condition is rather strong and in many economic problems his theorem does not apply. Arrow [1] proposed an interesting partial

Study of linear discrete time systems with state variable constraints

In this paper we discuss the property of reachability and controllability for a linear discrete time system, with the trajectory constrained to satisfy a state variable inequality constraint of the form.

Design procedure for single input systems with state variable constraints

Design procedure for single input systems with state variable constraints

Shuttle Ascent Trajectory Optimization with Function Space Quasi-Newton Techniques

A Space Shuttle ascent trajectory optimization problem from lift-off to orbital insertion is solved with a function space version of a quasi-Newton parameter optimization method developed by Broyden. The problem includes five parameter and one bounded-function controls, two state-variable constraints, and four terminal conditions. The bounded controls are treated directly, while the remaining constraints are adjoined to the performance index (maximum payload) with penalty functions. The problem is formulated as a four-phase variational problem (liftoff, pitch-over, gravity-turn, linear tangent steering), and the appropriate gradients are developed by first variation theory. A projection operator is introduced to aid in the interpretation of the algorithm with mixed parameter and function controls.

Minimum-time rescue trajectories between spacecraft in circular orbits

Optimal space trajectories to be flown by a rescue vehicle so as to rendezvous with a distressed spacecraft in minimum time are investigated. Nonlinear equations describing the inverse square gravitational field are used, and large transfer angles are considered. The rocket thrust of the rescue vehicle is allowed to vary between zero and a specified upper limit. The distressed vehicle is passive. Both vehicles are initially in circular orbits, but not necessarily at the same altitude nor in the same plane. Atmospheric effects are neglected, but the minimum altitude of the rescue craft is restricted by introducing a state-variable constraint. It is found that the optimal trajectories may contain coasting arcs and that arcs lying along the state-variable constraint may be of intermediate thrust. A parametric study is made using a presently proposed space tug as the rescue vehicle. The results indicate that minimum-time trajectories could be satisfactorily flown by the tug in a low earth orbit rescue effort.

An efficient algorithm for solving optimal control problems with linear terminal constraints

The optimization of nonlinear systems subject to linear terminal state variable constraints is considered. A technique for solving this class of problems is proposed that involves a piecewise polynomial parameterization of the system variables. The optimal control problem is thereby reduced to a linearly constrained parameter optimization problem which can be solved efficiently using the quadratically convergent Gold-farb-Lapidus algorithm. Illustrative numerical examples are presented.

Application of the Conjugate Gradient Method to a Problem on Minimum Time Orbit Transfer

This correspondence considers the problem of optimally controlling the thrust steering angle of an ion-propelled spaceship so as to effect a minimum time coplanar orbit transfer from the mean orbital distance of Earth to mean Martian and Venusian orbital distances. This problem has been modelled as a free terminal time-optimal control problem with unbounded control variable and with state variable equality constraints at the final time. The problem has been solved by the penalty function approach, using the conjugate gradient algorithm. In general, the optimal solution shows a significant departure from earlier work. In particular, the optimal control in the case of Earth-Mars orbit transfer, during the initial phase of the spaceship's flight, is found to be negative, resulting in the motion of the spaceship within the Earth's orbit for a significant fraction of the total optimized orbit transfer time. Such a feature exhibited by the optimal solution has not been reported at all by earlier investigators of this problem.

Upper and lower bounds via penalty functions for control problems

A close relationship between penalty functions and duality is shown to exist for a certain class of convex optimal control problems with state and control variable constraints. This relationship can be used to generate upper and lower bounds on the value of the solution to a control problem of the prescribed class.

Short-time parameter optimization with flight control application

A design approach is presented for systems operating over a relatively short period of time about various operating points with state variable constraints. This class of problems is especially relevant to certain flight control problems. The design approach is applied to a simplified model of longitudinal dynamics of the F-4 aircraft operating in three widely separated flight conditions. A linear model is assumed about each operating point. Control is achieved via constrained state feedback. The basic problem is then to minimize a suitable integral quadratic performance measure subject to state variable constraints. The main theoretical result is theorem 1 which supplies the inequality constraints required to guarantee short-time stability. The short-time optimization problem is ultimately reduced to a nonlinear programming problem with inequality constraints.

Optimal Aircraft Go - Around and Flare Maneuvers

This paper analyzes in detail two of the critical aircraft maneuvers associated with approach and landing: the go-around maneuver and the flare maneuver. Optimal solutions that include state and control variable constraints are obtained for both problems. Two algorithms are given for computation of the minimum and maximum altitude loss associated with the pilot-controlled go-around maneuver. A matrix operator is obtained that can be used for in-flight computation of the altitude loss on a small general-purpose digital computer. The flare optimization presented is for a cost functional that includes both the longitudinal touchdown dispersion and the normal acceleration. A closed-loop mechanization is given that approximates the optimal trajectory. A second matrix operator which can be used for prediction of the longitudinal touchdown point is obtained. Uncertainties are also obtained for the purpose of establishing a prediction confidence level. It is proposed that these prediction techniques should be incorporated into a decision making performance monitor. This monitor could provide the pilot with a continuous assessment of the approach and could generate a preflare decision on whether or not to commit the aircraft to the flare maneuver.

Approximate optimal control of distributed parameter systems.

An approximation technique for determining the optimal control of distributed parameter systems is developed. Treated explicitly, to illustrate the basic algorithm, are systems with a single state variable and control variables which appear linearly. The system partial differential equation may be nonlinear. Extension can be made to systems with more than one state variable and/or nonlinear control variables. The basis of the approach involves the intro- duction of a finite Chebyshev series approximation for the state variable. State and control variable constraints are treated using penalty functions. Numerical results are obtained for a representative example involving the optimal heating of a metal slab. The computational results obtained compare favorably with results presented by other authors. vI approximation technique for determining the optimal control of distributed parameter systems is given in this paper. The technique represents an extension of one used by Johnson1 for a finite-thrust trajectory problem involving the solution of ordinary differential equations. The basis of the approach developed here involves the introduction of a finite Chebyshev series approximation for the state variable. The coefficients of the approximation are determined by insisting that the state variable approximation be equal to specified values of the true state variable at the finite set of nonuniformly distributed Chebyshev points. The values of the state variable at the Chebyshev points become the free parameters of the problem. Since a closed form analytic expression for the state variable is available it is possible to eliminate all numerical differentiation and integration. Consequently a problem with a functional performance index is transformed into one involving minimization of an ordinary function of many variables. Optimization of the performance index takes place when the values of the state variable are manipulated using one of the well known mathematical programming algorithms. Thus for the sake of computation, the roles of the state and control variable are reversed, from that prevalent in most common optimization techniques. To illustrate the basic algorithm to be developed, a one- dimensional heat conduction problem is considered. 2-3 It should be noted that this problem was chosen for illustrative purposes only and does not represent a limitation on the type of distributed parameter system for which the technique is applicable.

Discrete Approximations to Continuous Optimal Control Problems

It is demonstrated that if P is a continuous optimal control problem whose system of differential equations is linear in the control and the state variables, and whose control and state variable constraint sets are convex, a direct method of determining an optimal solution of P exists. It is demonstrated that such a “continuous” problem can be replaced by a sequence of finite-dimensional, “discrete” optimization problems in which the control and state variable constraints are treated directly. The approximation obtained relates the respective optimal solutions.

Applications of dynamic programming to the control of water resource systems

The complexity and expense of water system projects have made optimum operation and design by computer-based techniques of increasing interest in recent years. Dynamic programming offers a powerful approach to a wide variety of these problems. Most water system problems can be classed as one of the following three types: 1. (1) Optimum operation during a short period, such as 24 hours, when all quantities are known; 2. (2) Monthly or yearly policy optimization when some system parameters, such as stream inflows, are not known exactly; 3. (3) Long-range planning or resource allocation when demands may or may not be known exactly. Realistic water resource problems have many decision and state variable constraints. There are also nonlinearities or stochastic variations in both the state equations and the return function. This paper describes how dynamic programming can handle these difficulties. Several specialized dynamic programming techniques applicable to water system problems are also introduced. These include successive approximations, forward dynamic programming, dynamic programming for stochastic control, and iteration in policy space. Four examples are solved and discussed—short-term optimization of a two-reservoir system is solved with forward dynamic programming; short-term optimization of a four-reservoir system is treated by successive approximations; optimum operation over a year, when stream-flows are stochastic variables, is found by iteration in policy spaces; and optimum long-term planning of system additions given projected demand is treated by forward dynamic programming.

Linear smoothing using measurements containing correlated noise with an application to inertial navigation

Kalman and Bucy [1] derived the maximum likelihood filter for continuous linear dynamic systems where all measurements contain white noise, i.e., noise with short correlation times compared to response times of the dynamic system. The corresponding maximum likelihood smoother was described in [2]. The maximum likelihood filter was presented in [3] for the case in which some measurements contain either no noise or colored noise, i.e., noise with correlation times comparable to or larger than the response times of the dynamic system. In this paper the maximum likelihood smoother for this latter case is derived by formulating the estimation problem as a problem in the calculus of variations having state variable equality constraints. An application of the results is made to estimating gyro drift rates of an inertial navigation system.

Optimal programming problems with a bounded state space.

A new set of necessary conditions are presented for solutions to optimal programming problems with a state variable inequality constraint (SVIC). On a constrained arc, the dimension of the state space is reduced, and the influence functions associated with this reduced state space are shown to be unique and continuous across junctures with unconstrained arcs. It is also shown that unconstrained arcs must satisfy certain constraints at both ends of a constrained arc and that explicit use of this fact must be made if the SVIC is adjoined directly to the performance index. ^EVERAL investigators have discussed necessary condi^ tions for solutions to optimal programming problems with a state variable inequality constraint.15 Most of these investigators recognized that, at a point where the system enters onto a constrained arc (an entry point), all feasible unconstrained arcs passing through this point must satisfy certain tangency constraints, namely, these arcs must have zero values of the state variable constraint function and all of its time derivatives that do not involve the control variable (say p-1 of them). Since control of the constraint function is realized only by changing its pth time derivative, no finite control will keep the system on the constraint boundary if the path reaching the constraint boundary does not meet these tangency constraints. In this paper, we point out that the tangency constraints also apply to the unconstrained arc at a point where the system leaves a constrained arc (an exit point). These exitpoint tangency constraints are satisfied automatically if the necessary conditions of Ref. 3 are used. However, if one uses the direct adjoining approach of Ref. 4, we shall show that explicit use must be made of the tangency constraints at both ends.

Sufficient Conditions for the Optimal Control of Nonlinear Systems

It is well known that Pontryagin’s maximum principle furnishes neces, sary conditions for the optimality of the control of a dynamic system. In the present work sufficient conditions for the optimality of the control of a nonlinear system with state and control variable constraints and with fixed initial and terminal times are given. These conditions are essentially Pontryagin’s necessary conditions for the same problem, plus some convexity, negativity and strict negativity conditions. The present sufficient conditions subsume the recent results of Lee, wherein sufficient conditions for the optimality of a system, linear in the state variables, were given.

The time‐optimal control of discrete‐time linear systems with bounded controls

AbstractThe discrete‐time, time‐optimal control of high‐order, linear systems with bounded controls is formulated as a problem in linear programming. The solution obtained requires a certain minimum number of controls to be on their bounds. Numerical results are obtained for a system with controls bounded on one and both sides and extensions indicated for state variable constraints. The method proves to be extremely fast and simple to solve on a high‐speed digital computer.

Time optimal discrete control system with bounded state variable

This paper considers the minimum time regulator problem for a linear time-invariant discrete system with input and state variable constraints. The state space techniques are used throughout. Conditions for controllability of the system are examined in detail and an algorithm for generating optimal controls is given such that computor implementation is possible for higher-order systems.

A computational technique for optimal control problems with state variable constraint

where F, f, and g are piecewise C z in the (x, u) space. Physically, such problems arise from attempting to control a dynamic system from an initial state c to some terminal state x(T) so that some performance criterion F is minimized and the motion of the dynamic system lies within some clased surface g in the state space. Such problems have received considerable attention recently. Dreyfus and Gamkrelidze studied the necessary conditions for optimality for problem (P-l) [1, 2]. Berkovitz, Desoer, and Pontriagin et al. treated similar problems under the now well known heading of the Maximum Principle [3-5]. However, effective computational techniques for solving such problems were not developed until recently. Bryson and Kelley first proposed a successive approximation procedure for solving such problems without any inequality constraint [6, 7]. Dreyfus and Bryson then extended the technique to include (P-l) [8, 9]. Ho and Neustadt also developed computational techniques for the case of linear dynamic system with inequality constraint on the decision variable u [10, 11]. The present paper demonstrates another computational technique for (P-I) which is an extension of the method presented in ref. 10.