Discrete Optimal Control Problem Research Articles

Optimal stationary control of discrete processes and a polynomial time algorithm for stochastic control problem on networks

Abstract The stochastic version of classical discrete optimal control problems with a finite set of states and infinite time horizon is studied. An approach for determining the optimal stationary control in discrete processes with deterministic and random states’ transitions of the dynamical system is described. The main problem is formulated on networks and a polynomial time algorithm for its solving is proposed. An application of the proposed algorithm for a class of stochastic decision problems is described.

Nonlinear dynamic systems and optimal control problems on time scales

This paper is mainly concerned with a class of optimal control problems of systems governed by the nonlinear dynamic systems on time scales. Introducing the reasonable weak solution of nonlinear dynamic systems, the existence of the weak solution for the nonlinear dynamic systems on time scales and its properties are presented. Discussing L1-strong-weak lower semicontinuity of integral functional, we give sufficient conditions for the existence of optimal controls. Using integration by parts formula and Hamiltonian function on time scales, the necessary conditions of optimality are derived respectively. Some examples on continuous optimal control problems, discrete optimal control problems, mathematical programming and variational problems are also presented for demonstration.

Inexact Restoration for Runge–Kutta Discretization of Optimal Control Problems

A numerical method is presented for Runge-Kutta discretization of unconstrained optimal control problems. First, general Runge-Kutta discretization is carried out to obtain a finite-dimensional approximation of the continous-time optimal control problem. Then a recent optimization technique, the inexact restoration (IR) method, due to Martinez and coworkers [E. G. Birgin and J. M. Martinez, J. Optim. Theory Appl., 127 (2005), pp. 229-247; J. M. Martinez and E. A. Pilotta, J. Optim. Theory Appl., 104 (2000), pp. 135-163; J. M. Martinez, J. Optim. Theory Appl., 111 (2001), pp. 39-58], is applied to the discretized problem to find an approximate solution. It is proved that, for optimal control problems, a key sufficiency condition for convergence of the IR method is readily satisfied. Under reasonable assumptions, the IR method for optimal control problems is shown to converge to a solution of the discretized problem. Convergence of a solution of the discretized problem to a solution of the continuous-time problem is also shown. It turns out that optimality phase equations of the IR method emanate from an associated Hamiltonian system, and so general Runge-Kutta discretization induces a symplectic partitioned Runge-Kutta scheme. A computational algorithm is described, and numerical experiments are made to demonstrate the working of the method for optimal control of the van der Pol system, employing the three-stage (order 6) Gauss-Legendre discretization.

Local Error Estimates for SUPG Solutions of Advection-Dominated Elliptic Linear-Quadratic Optimal Control Problems

We derive local error estimates for the discretization of optimal control problems governed by linear advection-diffusion partial differential equations (PDEs) using the streamline upwind/Petrov Galerkin (SUPG) stabilized finite element method. We show that if the SUPG method is used to solve optimization problems governed by an advection-dominated PDE, the convergence properties of the SUPG method is substantially different from the convergence properties of the SUPG method applied for the solution of an advection-dominated PDE. The reason is that the solution of the optimal control problem involves another advection-dominated PDE, the so-called adjoint equation, whose advection field is just the negative of the advection of the governing PDEs. For the solution of the optimal control problem, a coupled system involving both the original governing PDE as well as the adjoint PDE must be solved. We show that in the presence of a boundary layer, the local error between the solution of the SUPG discretized optimal control problem and the solution of the infinite dimensional problem is only of first order even if the error is computed locally in a region away from the boundary layer. In the presence of interior layers, we prove optimal convergence rates for the local error in a region away from the layer between the solution of the SUPG discretized optimal control problems and the solution of the infinite dimensional problem. Numerical examples are presented to illustrate some of the theoretical results.

Subgradients of the optimal value function in a parametric discrete optimal control problem

We study the first-order behavior of the optimal value function in a parametric discrete optimal control problem with linear constraints and a nonconvex cost function. By establishing a new result on the Frechet subdifferential of optimal value functions of parametric mathematical programming problems, we obtain some formulae on the Frechet subdifferential of optimal value functions in parametric discrete optimal control problems which complement results due to Kien et al. [3].

Discrete Hamilton–Jacobi Theory

We develop a discrete analogue of Hamilton-Jacobi theory in the framework of discrete Hamiltonian mechanics. The resulting discrete Hamilton-Jacobi equation is discrete only in time. We describe a discrete analogue of Jacobi's solution and also prove a discrete version of the geometric Hamilton-Jacobi theorem. The theory applied to discrete linear Hamiltonian systems yields the discrete Riccati equation as a special case of the discrete Hamilton-Jacobi equation. We also apply the theory to discrete optimal control problems, and recover some well-known results, such as the Bellman equation (discrete-time HJB equation) of dynamic programming and its relation to the costate variable in the Pontryagin maximum principle. This relationship between the discrete Hamilton-Jacobi equation and Bellman equation is exploited to derive a generalized form of the Bellman equation that has controls at internal stages.

Algorithms for solving discrete optimal control problems with infinite time horizon and determining minimal mean cost cycles in a directed graph as decision support tool

Time-discrete systems with a finite set of states are considered. Discrete optimal control problems with infinite time horizon for such systems are formulated. We introduce a certain graph-theoretic structure to model the transitions of the dynamical system. Algorithms for finding the optimal stationary control parameters are presented. Furthermore, we determine the optimal mean cost cycles. This approach can be used as a decision support strategy within such a class of problems; especially so-called multilayered decision problems which occur within environmental emission trading procedures can be modelled by such an approach.

Geometric structure-preserving optimal control of a rigid body

In this paper, we study a discrete variational optimal control problem for a rigid body. The cost to be minimized is the external torque applied to move the rigid body from an initial condition to a pre-specified terminal condition. Instead of discretizing the equations of motion, we use the discrete equations obtained from the discrete Lagrange---d'Alembert principle, a process that better approximates the equations of motion. Within the discrete-time setting, these two approaches are not equivalent in general. The kinematics are discretized using a natural Lie-algebraic formulation that guarantees that the flow remains on the Lie group SO(3) and its algebra $\mathfrak{s}\mathfrak{o}(3)$ . We use the Lagrange method for constrained problems in the calculus of variations to derive the discrete-time necessary conditions. We give a numerical example for a three-dimensional rigid body maneuver.

Solving discretized degenerate optimal control problems with state constraints

AbstractIn this paper, we consider a class of nonlinear optimization problems that arise from the discretization of optimal control problems with bounds on both state and control variables. We are particularly interested in degenerate cases, i.e. when the linear independence constraint qualification is not satisfied. For these problems, we analyse the basic global convergence properties and the numerical behaviour of a multiplier method that updates multipliers corresponding to inequality constraints instead of dealing with multipliers associated with equality constraints. Numerical results obtained for several instances of a discretized optimal control problem governed by a semi‐linear elliptic equation are included and indicate that this method is robust on degenerate cases, compared with other nonlinear optimization solvers. Copyright © 2009 John Wiley & Sons, Ltd.

Perturbation analysis of Lagrangian invariant subspaces of symplectic matrices

Lagrangian invariant subspaces for symplectic matrices play an important role in the numerical solution of discrete time, robust and optimal control problems. The sensitivity (perturbation) analysis of these subspaces, however, is a difficult problem, in particular, when the eigenvalues are on or close to some critical regions in the complex plane, such as the unit circle. We present a detailed perturbation analysis for several different cases of real and complex symplectic matrices. We analyze stability and conditional stability as well as the index of stability for these subspaces.

Asymptotic solution of a discrete optimal control problem for one class of weakly controllable systems

An asymptotic expansion in integer nonnegative powers of small parameter for a solution of a discrete optimal control problem for one class of weakly controllable systems is constructed by substituting an assumed asymptotic expansion into the problem conditions and obtaining a series of problems in the coefficients of the asymptotics. Conditions of existence of a solution to the perturbed problem for sufficiently small values of the parameter are found. Estimates of closeness of the approximate and exact solutions in terms of trajectory, control, and functional are obtained. The values of the minimized functional are proven not to increase when higher-order asymptotic approximations of the optimal control are used. The discussion is illustrated by examples.

An Algorithm for Solving Discrete Optimal Control Problems with Infinite Time Horizon - Determining the Minimal Mean Cost Cycles in a Directed Graph

Time-discrete systems with a finite set of states are considered. Discrete optimal control problems with infinite time horizon for such systems are formulated. We introduce a certain graph structure to model the transitions of the underlying dynamical system. Algorithms for finding the optimal stationary control parameters are presented: Furthermore we determine the optimal mean cost cycles. We elaborate this new intelligent technique in detail. The fractional linear programming representation can be regarded as an example for an intelligent optimization technique.

O решении дискретной линейно-квадратичной задачи оптимального управления

O решении дискретной линейно-квадратичной задачи оптимального управления

Optimal control of piecewise affine systems with piecewise affine state feedback

In this paper, we consider a class of optimal control problems involvingpiecewise affine (PWA) systems with piecewise affine state feedback. Wefirst show that if the piecewise affine state feedback control is assumed tobe continuous at the switching boundaries, then the number of switchingamongst PWA systems is finite. On this basis, this optimal control problemis transformed into a discrete valued optimal control problem. For thisdiscrete valued optimal control problem, we introduce the time scalingtransform to convert it into an equivalent constrained optimal parameterselection problem, for which it can be solved by existing optimal controltechniques for optimal parameter selection problems. A numerical example issolved so as to illustrate the proposed method.

A Discrete Dynamic Programming Approximation to the Multiobjective Deterministic Finite Horizon Optimal Control Problem

This paper addresses the problem of finding an approximation to the minimal element set of the objective space for the class of multiobjective deterministic finite horizon optimal control problems. The objective space is assumed to be partially ordered by a pointed convex cone containing the origin. The approximation procedure consists of a two-step discretization in time and state space. Following the first-order time discretization, the dynamic programming principle is used to find the multiobjective discrete dynamic programming equation equivalent to the resulting discrete multiobjective optimal control problem. The multiobjective discrete dynamic programming equation is finally discretized in the state space. The convergence of the approximation for both discretization steps is discussed.

Optimal Control of the Propagation of a Graph in Inhomogeneous Media

We study an optimal control problem for viscosity solutions of a Hamilton-Jacobi equation describing the propagation of a one-dimensional graph with the control being the speed function. The existence of an optimal control is proved together with an approximate controllability result in the $H^{-1}$-norm. We prove convergence of a discrete optimal control problem based on a monotone finite difference scheme and describe some numerical results.

Determination of functional gradient in an optimal control problem related to metal solidification

The gradient of the cost functional in the discrete optimal control problem of metal solidification in casting is exactly evaluated. The mathematical model describing the solidification process is based on a three-dimensional two-phase initial-boundary value problem of the Stefan type. Formulas determining exact gradient determination are derived using the fast automatic differentiation technique.

2-Regularity and 2-Normality Conditions in Discrete Optimal Control Problems

The paper is devoted to general optimal control problems for discrete-time systems with equality type of constraints on end points and control. We derive first-order necessary optimality conditions that are meaningful under the new nontriviality condition.

Second-order optimality conditions in a discrete optimal control problem

Discrete optimal control problems with variable endpoints and with inequality type constraints on control are considered. We derive first- and second-order necessary optimality conditions that are meaningful without a priori normality assumptions.

On the convergence of the extended conjugate gradient method for discrete optimal control problems (DOCP)

One striking factor that attracts problem solvers to use particular algorithm is its convergence behavior. In this paper, the convergence of the iterates generated by the ECGM for DOCP as proposed by [14] is examined. Journal of the Nigerian Association of Mathematical Physics Vol. 9 2005: pp. 493-500