Algorithm For Optimal Control Problems Research Articles

Supplementary Optimality Properties of the Family of Gradient-Restoration Algorithms for Optimal Control Problems

The problem of minimizing a functional subject to differential constraints, nondifferential constraints, initial constraints, and final constraints is considered within the frame of the family of gradient-restoration algorithms for optimal control problems. This family includes sequential gradient-restoration algorithms (SGRA) and combined gradient-restoration algorithms (CGRA).The system of Lagrange multipliers associated with (i) the gradient phase of SGRA, (ii) the restoration phase of SGRA, and (iii) the combined gradient-restoration phase of CGRA is examined. It is shown that, in each case, the Lagrange multipliers are endowed with a suppiementary optimality property: they minimize a special functional, quadratic in the multipliers, subject to the multiplier differential equations and boundary conditions, for given state, control, and parameter.These supplementary optimality properties have considerable computational implications: they allow one to reduce the study of an iteration of (i), (ii), (iii) to a mathematical programming problem involving a finite number of parameters as unknowns.

A Class of Structured Quasi-Newton Algorithms for Optimal Control Problems

In this paper we consider a class of Quasi-Newton algorithms for the numerical solution of discrete-time optimal control problems. The approach followed here is based on a suitable augmented Lagrangian function whose unconstrained minimization yields both the solution of the problem and the associated multipliers. It is shown that structured Quasi-Newton algorithms can be constructed, which exploit the sparse structure of the Hessian matrix of the augmented Lagrangian and avoid the need of evaluating second order derivatives of the problem functions. Numerical examples are reported.

First-order strong variation algorithm for optimal control problems involving hyperbolic systems

This paper considers an optimal control problem involving linear, hyperbolic partial differential equations. A first-order strong variational technique is used to obtain an algorithm for solving the optimal control problem iteratively. It is shown that the accumulation points of the sequence of controls generated by the algorithm (if they exist) satisfy a necessary condition for optimality.

A control parametrization algorithm for optimal control problems involving linear systems and linear1

Computational schemes based on control parametrization techniques are known to be very efficient for solving optimal control problems. However, the convergence result is only available for the case in which the dynamic system is linear and without the terminal equality and inequality constraints. This paper is to improve this convergence result by allowing the presence of the linear terminal inequality. For illustration, an example arising in the study of optimally one-sided heating of a metal slab in a furnace is considered.

First-order strong variation algorithm for optimal control problems involving parabolic systems

This paper considers the optimal control of a system governed by a parabolic partial differential equation with first boundary conditions. For this system, a condition of extremality is defined, which is proven to be a necessary condition for optimality. For non-extremal controls, a method of constructing a new control that has an improved criterion value is discussed. It is shown that if a sequence of controls, each constructed from the previous control in the manner discussed, converges, then the limit is extremal.

Supplementary optimality properties of gradient-restoration algorithms for optimal control problems

In this paper, sequential gradient-restoration algorithms for optimal control problems are considered, and attention is focused on the gradient phase. It is shown that the Lagrange multipliers associated with the gradient phase not only solve the auxiliary minimization problem of the gradient phase, but are also endowed with a supplementary optimality property: they minimize the error in the optimality conditions, subject to the multiplier differential equations and boundary conditions, for given state, control, and parameter.

An exact penalty function algorithm for optimal control problems with control and terminal equality constraints, part 2

In Part 1 of this paper, implementable and conceptual versions of an algorithm for optimal control problems with control constraints and terminal equality constraints were presented. It was shown that anyL ∞ accumulation points of control sequences generated by the algorithms satisfy necessary conditions of optimality. Since such accumulation points need not exist, it is shown in this paper that control sequences generated by the algorithms always have accumulation points in the sense of control measure, and these accumulation points satisfy optimality conditions for the corresponding relaxed control problem.

An exact penalty function algorithm for optimal control problems with control and terminal equality constraints, part 1

The presence of control constraints, because they are nondifferentiable in the space of control functions, makes it difficult to cope with terminal equality constraints in optimal control problems. Gradient-projection algorithms, for example, cannot be employed easily. These difficulties are overcome in this paper by employing an exact penalty function to handle the cost and terminal equality constraints and using the control constraints to define the space of permissible search directions in the search-direction subalgorithm. The search-direction subalgorithm is, therefore, more complex than the usual linear program employed in feasible-directions algorithms. The subalgorithm approximately solves a convex optimal control problem to determine the search direction; in the implementable version of the algorithm, the accuracy of the approximation is automatically increased to ensure convergence.

Sequential conjugate gradient‐restoration algorithm for optimal control problems with non‐differential constraints and general boundary conditions, part 2

AbstractIn Reference 1, Wu and Miele developed the sequential conjugate gradient‐restoration algorithm for minimizing a functional subject to differential constraints, with or without non‐differential constraints, and general boundary conditions. In this paper, several numerical examples are presented. Some of these pertain to a quadratic functional subject to linear constraints, and some pertain to a non‐quadratic functional subject to non‐linear constraints. These examples demonstrate the feasibility as well as the convergence characteristics of the sequential conjugate gradient‐restoration algorithm.

Sequential conjugate gradient‐restoration algorithm for optimal control problems with non‐differential constraints and general boundary conditions, part I

AbstractIn this paper, a new member of the family of sequential gradient‐restoration algorithms for the solution of optimal control problems is presented. This is an algorithm of the conjugate gradient type, which is designed to solve two classes of optimal control problems, called Problem P1 and Problem P2 for easy indentification.Problem P1 involves minimizing a functional I subject to differential constraints and general boundary conditions. It consists of finding the state x(t), the control u(t), and the parameter pi so that the functional I is minimized, while the constraints and the boundary conditions are satisfied to a predetermined accuracy. Problem P2 extends Problem P1 to include non‐differential constraints to be satisfied everywhere along the interval of integration.The approach taken is a sequence of two‐phase cycles, composed of a conjugate gradient phase and a restoration phase. The conjugate gradient phase involves one iteration and is designed to decrease the value of the functional, while the constraints are satisfied to first order. The restoration phase involves one or more iterations; each restorative iteration is designed to force constraint satisfaction to first order, while the norm squared of the variations of the control, the parameter, and the missing components of the initial state is minimized. The resulting algorithm has several properties: (i) it produces a sequence of feasible solutions; (ii) each feasible solution is characterized by a value of the functional I which is smaller than that associated with any previous feasible solution; and (iii) for the special case of a quadratic functional subject to linear constraints, the variations of the state, control, and parameter produced by the sequence of conjugate gradient phases satisfy various orthogonality and conjugacy conditions.The algorithm presented here differs from those of References 1‐4, in that it is not required that the state vector be given at the initial point. Instead, the initial conditions can be absolutely general. In analogy with References 1‐4, the present algorithm is capable of handling general final conditions; therefore, it is suitable for the solution of optimal control problems with general boundary conditions.The importance of the present algorithm lies in that many optimal control problems either arise naturally in the present format or can be brought to such a format by means of suitable transformations.5 Therefore, a great variety of optimal control problems can be handled. This includes: (i) problems with control equality constraints, (ii) problems with state equality constraints, (iii) problems with state‐derivative equality constraints, (iv) problems with‐control inequality constraints, (v) problems with state inequality constraints, (vi) problems with state‐derivative inequality constraints, and (vii) Chebyshev minimax problems.Several numerical examples are presented in Part 2 (Reference 6) in order to illustrate the performance of the algorithm associated with Problem P1 and Problem P2. The numerical results show the feasibility as well as the convergence characteristics of the present algorithm.

Interactive Simulation Programs Simfor and Simhyb with Application to the Solution of Optimization Problems

Interactive continuous-system simulation programs SIMFOR and SIMHYB are presented. They were developed for the EAI PACER 600 hybrid computer, keeping in view the following requirements: a) Feasibility to solve two-point boundary-value problems; b) interactivity of simulation; c) easy programming; d) easy switchover from digital to hybrid simulation. Both programs are similar, but SIMFOR integrates the differential equations digitally, while SIMHYB performs the same task on the analog part of the hybrid. Neither of them has any compiler; they are merely program libraries including a main interactive control program, input/output procedures, function handling and integration procedures, etc. The problem to be solved is introduced through a user-supplied FORTRAN routine. Solution profiles can be directly observed on the PACER 1420 display terminal. It is possible to control a sequence of simulation runs by a user-written algorithm, so that more complex problems can be solved, too. To illustrate this, a SIMFOR realization of a gradient algorithm for optimal control problems is treated in the last section of the paper.

A transformation technique for optimal control problems with partially linear state inequality constraints

This paper considers optimal control problems involving the minimization of a functional subject to differential constraints, terminal constraints, and a state inequality constraint. The state inequality constraint is of a special type, namely, it is linear in some or all of the components of the state vector. A transformation technique is introduced, by means of which the inequality-constrained problem is converted into an equality-constrained problem involving differential constraints, terminal constraints, and a control equality constraint. The transformation technique takes advantage of the partial linearity of the state inequality constraint so as to yield a transformed problem characterized by a new state vector of minimal size. This concept is important computationally, in that the computer time per iteration increases with the square of the dimension of the state vector. In order to illustrate the advantages of the new transformation technique, several numerical examples are solved by means of the sequential gradient-restoration algorithm for optimal control problems involving nondifferential constraints. The examples show the substantial savings in computer time for convergence, which are associated with the new transformation technique.

Sequential conjugate gradient-restoration algorithm for optimal control problems with nondifferential constraints, † Part I

Abstract A sequential conjugate gradient-restoration algorithm is developed in order to solve optimal control problems involving a functional subject to differential constraints, nondifferential constraints, and terminal constraints. The algorithm is composed of a sequence of cycles, each cycle consisting of two phases, a conjugate gradient phase and a restoration phase. The conjugate gradient phase involves a single iteration and is designed to decrease the value of the functional, while satisfying the constraints to first order. During this iteration, the first variation of the functional is minimized, subject to the linearized constraints. The minimization is performed over the class of variations of the control and the parameter which are equidistant from some constant multiple of the corresponding variations of the previous conjugate gradient phase. For the special case of a quadratic functional subject to linear constraints, various orthogonality and conjugacy conditions hold. The restoration phase ...

Sequential conjugate gradient-restoration algorithm for optimal control problems with nondifferential constraints, Part II

Abstract In Part I, Miele et al. (1979) developed the sequential conjugate gradient-restoration algorithm for minimizing a functional subject to differential constraints, nondifferential constraints, and terminal constraints. In this paper, several numerical examples are presented, some pertaining to a quadratic functional subject to linear constraints and some pertaining to a nonquadratie functional subject to nonlinear constraints. These examples demonstrate the feasibility as well as the convergence characteristics of the sequential conjugate gradient-restoration algorithm.

Sequential gradient-restoration algorithm for optimal control problems with general boundary conditions

This paper considers the numerical solution of two classes of optimal control problems, called Problem P1 and Problem P2 for easy identification. Problem P1 involves a functionalI subject to differential constraints and general boundary conditions. It consists of finding the statex(t), the controlu(t), and the parameter π so that the functionalI is minimized, while the constraints and the boundary conditions are satisfied to a predetermined accuracy. Problem P2 extends Problem P1 to include nondifferential constraints to be satisfied everywhere along the interval of integration. Algorithms are developed for both Problem P1 and Problem P2. The approach taken is a sequence of two-phase cycles, composed of a gradient phase and a restoration phase. The gradient phase involves one iteration and is designed to decrease the value of the functional, while the constraints are satisfied to first order. The restoration phase involves one or more iterations and is designed to force constraint satisfaction to a predetermined accuracy, while the norm squared of the variations of the control, the parameter, and the missing components of the initial state is minimized. The principal property of both algorithms is that they produce a sequence of feasible suboptimal solutions: the functions obtained at the end of each cycle satisfy the constraints to a predetermined accuracy. Therefore, the values of the functionalI corresponding to any two elements of the sequence are comparable. The stepsize of the gradient phase is determined by a one-dimensional search on the augmented functionalJ, while the stepsize of the restoration phase is obtained by a one-dimensional search on the constraint errorP. The gradient stepsize and the restoration stepsize are chosen so that the restoration phase preserves the descent property of the gradient phase. Therefore, the value of the functionalI at the end of any complete gradient-restoration cycle is smaller than the value of the same functional at the beginning of that cycle. The algorithms presented here differ from those of Refs. 1 and 2, in that it is not required that the state vector be given at the initial point. Instead, the initial conditions can be absolutely general. In analogy with Refs. 1 and 2, the present algorithms are capable of handling general final conditions; therefore, they are suited for the solution of optimal control problems with general boundary conditions. Their importance lies in the fact that many optimal control problems involve initial conditions of the type considered here. Six numerical examples are presented in order to illustrate the performance of the algorithms associated with Problem P1 and Problem P2. The numerical results show the feasibility as well as the convergence characteristics of these algorithms.

A feasible directions algorithm for optimal control problems with control and terminal inequality constraints

Control constraints, because they are nondifferentiable in the space of control functions, are difficult to cope with in optimization algorithms not of the penalty function type. In feasible directions type algorithms a search direction is obtained by solving a subproblem which has a fixed constraint set S chosen a priori to make this subproblem simple to solve. In this paper control constraints are dealt with by using them to define, at the current control u , a constraint set S(u) for the search direction subproblem. This enables control constraints to be easily handled.

Function-space quasi-Newton algorithms for optimal control problems with bounded controls and singular arcs

Two existing function-space quasi-Newton algorithms, the Davidon algorithm and the projected gradient algorithm, are modified so that they may handle directly control-variable inequality constraints. A third quasi-Newton-type algorithm, developed by Broyden, is extended to optimal control problems. The Broyden algorithm is further modified so that it may handle directly control-variable inequality constraints. From a computational viewpoint, dyadic operator implementation of quasi-Newton methods is shown to be superior to the integral kernel representation. The quasi-Newton methods, along with the steepest descent method and two conjugate gradient algorithms, are simulated on three relatively simple (yet representative) bounded control problems, two of which possess singular subarcs. Overall, the Broyden algorithm was found to be superior. The most notable result of the simulations was the clear superiority of the Broyden and Davidon algorithms in producing a sharp singular control subarc.

Biased penalty function algorithms for optimal control problems with terminal restraints

Algorithms based on biasing data entering a square law terminal error penalty function are synthesized for the optimal control problem with terminal restraints at a fixed time. A development leads to basic, interpolative, and control compensated-bias updating strategies with progressively improved performance compared with existing methods demonstrated from computational results. The new algorithms are related to existing methods based on penalty weight updating, the modified method of multipliers, adjoint system and isoperimetric formulations.

Recent advances in gradient algorithms for optimal control problems

This paper summarizes recent advances in the area of gradient algorithms for optimal control problems, with particular emphasis on the work performed by the staff of the Aero-Astronautics Group of Rice University. The following basic problem is considered: minimize a functionalI which depends on the statex(t), the controlu(t), and the parameter π. Here,I is a scalar,x ann-vector,u anm-vector, and π ap-vector. At the initial point, the state is prescribed. At the final point, the statex and the parameter π are required to satisfyq scalar relations. Along the interval of integration, the state, the control, and the parameter are required to satisfyn scalar differential equations. First, the sequential gradient-restoration algorithm and the combined gradient-restoration algorithm are presented. The descent properties of these algorithms are studied, and schemes to determine the optimum stepsize are discussed. Both of the above algorithms require the solution of a linear, two-point boundary-value problem at each iteration. Hence, a discussion of integration techniques is given. Next, a family of gradient-restoration algorithms is introduced. Not only does this family include the previous two algorithms as particular cases, but it allows one to generate several additional algorithms, namely, those with alternate restoration and optional restoration. Then, two modifications of the sequential gradient-restoration algorithm are presented in an effort to accelerate terminal convergence. In the first modification, the quadratic constraint imposed on the variations of the control is modified by the inclusion of a positive-definite weighting matrix (the matrix of the second derivatives of the Hamiltonian with respect to the control). The second modification is a conjugate-gradient extension of the sequential gradient-restoration algorithm. Next, the addition of a nondifferential constraint, to be satisfied everywhere along the interval of integration, is considered. In theory, this seems to be only a minor modification of the basic problem. In practice, the change is considerable in that it enlarges dramatically the number and variety of problems of optimal control which can be treated by gradient-restoration algorithms. Indeed, by suitable transformations, almost every known problem of optimal control theory can be brought into this scheme. This statement applies, for instance, to the following situations: (i) problems with control equality constraints, (ii) problems with state equality constraints, (iii) problems with equality constraints on the time rate of change of the state, (iv) problems with control inequality constraints, (v) problems with state inequality constraints, and (vi) problems with inequality constraints on the time rate of change of the state. Finally, the simultaneous presence of nondifferential constraints and multiple subarcs is considered. The possibility that the analytical form of the functions under consideration might change from one subarc to another is taken into account. The resulting formulation is particularly relevant to those problems of optimal control involving bounds on the control or the state or the time derivative of the state. For these problems, one might be unwilling to accept the simplistic view of a continuous extremal arc. Indeed, one might want to take the more realistic view of an extremal arc composed of several subarcs, some internal to the boundary being considered and some lying on the boundary. The paper ends with a section dealing with transformation techniques. This section illustrates several analytical devices by means of which a great number of problems of optimal control can be reduced to one of the formulations presented here. In particular, the following topics are treated: (i) time normalization, (ii) free initial state, (iii) bounded control, and (iv) bounded state.

First-order strong variation algorithms for optimal control problems with terminal inequality constraints

This paper presents eight algorithms for solving optimal control problems with general constraints on the control and inequality constraints on the terminal state. These algorithms are of the differential dynamic programming type and bear a strong kinship to nonlinear programming algorithms of the feasible direction type.