Discrete Optimal Control Problem Research Articles

Necessary First-Order and Second-Order Optimality Conditions in Discrete-Time Stochastic Systems

In this paper, first-order and second-order necessary conditions for optimality for discrete-time stochastic optimal control problems governed by discrete-time Ito equations are established. A new discrete-time backward stochastic equation and discrete-time backward stochastic matrix equation are introduced. Based on the discrete-time backward stochastic Ito equation, a discrete-time stochastic maximum principle for the stochastic discrete optimal control problems is obtained. Moreover, using the discrete-time backward stochastic matrix equation the second-order necessary conditions for optimality are obtained.

Fractional spectral collocation method for optimal control problem governed by space fractional diffusion equation

In this paper, we mainly investigate the fractional spectral collocation discretization of optimal control problem governed by a space-fractional diffusion equation. Existence and uniqueness of the solution to optimal control problem is proved. The continuous first order optimality condition is derived. The eigenfunctions of two classes of fractional Strum–Liouville problems are used as basis functions to approximate state variable and adjoint state variable, respectively. The fractional spectral collocation scheme for the control problem is constructed based on ‘first optimize, then discretize’ approach. Note that the solutions of fractional differential equations are usually singular near the boundary, a generalized fractional spectral collocation scheme for the control problem is proposed based on ‘first optimize, then discretize’ approach. A projected gradient algorithm is designed based on the discrete optimality condition. Numerical experiments are carried out to verify the effectiveness of the proposed numerical schemes and algorithm.

A Note on Liao’s Recurrent Neural-Network Learning for Discrete Multi-stage Optimal Control Problems

The roots of neural-network backpropagation (BP) may be traced back to classical optimal-control gradient procedures developed in early 1960s. Hence, BP can directly apply to a general discrete N-stage optimal control problem that consists of N stage costs plus a terminal state cost. In this journal (Liao in Neural Process Lett 10:195–200, 1999), given such a multi-stage optimal control problem, Liao has turned it into a problem involving a terminal state cost only (via classical transformation), and then claimed that BP on the transformed problem leads to new recurrent neural network learning. The purpose of this paper is three-fold: First, the classical terminal-cost transformation yields no particular benefit for BP. Second, two simulation examples (with and without time lag) demonstrated by Liao can be regarded naturally as deep feed-forward neural-network learning rather than as recurrent neural-network learning from the perspective of classical optimal-control gradient methods. Third, BP can readily deal with a general history-dependent optimal control problem (e.g., involving time-lagged state and control variables) owing to Dreyfus’s 1973 extension of BP. Throughout the paper, we highlight systematic BP derivations by employing the recurrence relation of nominal cost-to-go action-value functions based on the stage-wise concept of dynamic programming.

Systematic Design of Optimal Low-Thrust Transfers for the Three-Body Problem

We develop a computational approach for the design of continuous low thrust transfers in the planar circular restricted three-body problem. The use of low thrust propulsion allows the spacecraft to depart from the natural dynamics and enables a wider range of transfers. We generate the reachable set of the spacecraft and use this to determine transfer opportunities, analogous to the intersection of control-free invariant manifolds. The reachable set is developed on a lower dimensional Poincare section and used to design transfer trajectories. This is solved numerically as a discrete optimal control problem using a variational integrator, which preserves the geometric structure of the motion in the three-body problem. We demonstrate our approach with two numerical simulations of transfers in the Earth-Moon three-body system.

Discrete Optimal Control of Interconnected Mechanical Systems

This article develops variational integrators for a class of underactuated mechanical systems using the theory of discrete mechanics. A discrete optimal control problem is then formulated for this class of system on the phase spaces of the actuated and unactuated subsystems separately. Exploiting the left-trivialization of the cotangent bundle, and assuming the time-step of discrete evolution is small enough to exploit the diffeomorphism feature of the exponential map in a neighbourhood of the identity of the Lie group, that enables a mapping of the group variables to the Lie algebra, a variational approach is adopted to obtain the first order necessary conditions that characterise optimal trajectories. The proposed approach is then demonstrated on two benchmark underactuated systems through numerical experiments.

LP Formulations of Discrete Time Long-Run Average Optimal Control Problems: The NonErgodic Case

We formulate and study the infinite dimensional linear programming (LP) problem associated with the deterministic discrete time long-run average criterion optimal control problem. Along with its du...

Flatness Based Optimal Control for Induction Machine Drives

Aim of this study is to derive an efficient method and problem formulation for optimal control of induction machines in variable speed drives, as used for example in electric vehicles. Induction machine dynamics are thus represented as a differential flat system and voltage source inverter losses are considered as an average value model. Based on the canonical form of the equivalent flat system, a nonlinear discrete finite horizon optimal control problem with a predefined torque and speed profile is introduced. If it is required to find a global solution, this problem can be efficiently solved by discrete dynamic programming or, when time is of essence, by means of sequential quadratic programming. The former is applied to a practical electric vehicle load cycle with limiting voltage constraints. The solution demonstrates how optimal control methods may be beneficial to drive design.

Strong Lagrange duality and the maximum principle for nonlinear discrete time optimal control problems

We examine discrete-time optimal control problems with general, possibly non-linear or non-smooth dynamic equations, and state-control inequality and equality constraints. A new generalized convexity condition for the dynamics and constraints is defined, and it is proved that this property, together with a constraint qualification constitute sufficient conditions for the strong Lagrange duality result and saddle-point optimality conditions for the problem. The discrete maximum principle of Pontryagin is obtained in a straightforward manner from the strong Lagrange duality theorem, first in a new form in which the Lagrangian is minimized both with respect to the state and to the control variables. Assuming differentiability, the maximum principle is obtained in the usual form. It is shown that dynamic systems satisfying a global controllability condition with convex costs, have the required convexity property. This controllability condition is a natural extension of the customary directional convexity condition applied in the derivation of the discrete maximum principle for local optima in the literature.

Deep learning as optimal control problems: Models and numerical methods

We consider recent work of [ 18 ] and [ 9 ], where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We review the first order conditions for optimality, and the conditions ensuring optimality after discretisation. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. The differential equation setting lends itself to learning additional parameters such as the time discretisation. We explore this extension alongside natural constraints (e.g. time steps lie in a simplex). We compare these deep learning algorithms numerically in terms of induced flow and generalisation ability.

Optimal control of time-discrete two-phase flow driven by a diffuse-interface model

We propose a general control framework for two-phase flows with variable densities in the diffuse interface formulation, where the distribution of the fluid components is described by a phase field. The flow is governed by the diffuse interface model proposed in Abelset al.[M3AS22(2012) 1150013]. On the basis of the stable time discretization proposed in Garckeet al.[Appl. Numer. Math.99(2016) 151] we derive necessary optimality conditions for the time-discrete and the fully discrete optimal control problem. We present numerical examples with distributed and boundary controls, and also consider the case, where the initial value of the phase field serves as control variable.

TO THE OPTIMALITY OF QUASI-SINGULAR CONTROL IN A DISCRETE STEPPED CONTROL PROBLEM WITH NONLOCAL BOUNDARY CONDITIONS

Consider one step discrete optimal control problem described by a system of nonlinear difference equations with nonlocal boundary conditions. Under the assumption of convexity of control domains, an analogue of the linearized maximum condition is proved. The case of degeneracy of the linearized maximum condition has been specially studied.

The computation method of rational multi-point forming of panel in the creep mode

The problems of modeling in the CAE-system of panels forming processes in the creep mode with the use of reconfigurable rod punch are considered. The geometrical and contact nonlinearity are taken into account in the model of the problem. For the formulation of rational deformation problems the damage parameter is taken as an optimization criterion. A discrete optimal control problem is formulated, which is solved by the method of dynamic programming with improvement by the method of local variations

Optimal stochastic sequential control with fractional rewards

In this paper we discuss the optimal stopping problem for multiparameter stochastic processes with the index set Nd, and in general, the optimal sequential control problem for stochastic processes with the index set consisting of increasing paths. We give the formulation of new control problems which combine the discrete time optimal sequential control problem with the criterion of a fractional reward. Using the standard mathematical argument in the theory of mathematical programming, we prove the existence of optimal stopping points and optimal controls for these problems under the certain assumption on the regularity of the stochastic processes. Further we discuss ε – optimal controls, finite time controls, and Markov case, respectively.

Identification of the thermal conductivity coefficient in two dimension case

In the paper the problem of determining the thermal conductivity coefficient that depends on temperature is studied. The consideration is based on the mixed boundary value problem for the two-dimensional unsteady-state heat equation. The inverse coefficient problem is reduced to a variation problem. The mean-root-square deviation of the heat flux obtained by solving initial-boundary value problem from the experimental data on the boundary of the considered domain is used as the objective functional. The optimal control problem is solved numerically. The objective functional is minimized using the gradient methods. It is well known that it is very important for the gradient methods to determine accurate values of the gradients. For this reason, in this paper we used the efficient fast automatic differentiation technique, which gives the exact functional gradient for the discrete optimal control problem. The examples of solving the inverse coefficient problem confirm the efficiency of the proposed algorithm.

Solution of Optimal Harvesting Problem by Finite Difference Approximations of Size-Structured Population Model

We solve numerically a forest management optimization problem governed by a nonlinear partial differential equation (PDE), which is a size-structured population model. The formulated problem is supplemented with a natural constraint for a solution to be non-negative. PDE is approximated by an explicit or implicit in time finite difference scheme, whereas the cost function is taken from the very beginning in the finite-dimensional form used in practice. We prove the stability of the constructed nonlinear finite difference schemes on the set of non-negative vectors and the solvability of the formulated discrete optimal control problems. The gradient information is derived by constructing discrete adjoint state equations. The projected gradient method is used for finding the extremal points. The results of numerical testing for several real problems show good agreement with the known results and confirm the theoretical statements.

The neuro-dynamic scheme for solving general form of discrete time optimal control problems

In this paper, we show that recently developed neural network methods for quadratic programming can be put to use in solving discrete time optimal control problems, with general pointwise constraints on states and controls. We describe a high performance recurrent neural network for a discrete time linear quadratic regulator problem with mixed state–control constraints. The equilibrium point of the proposed model is proved to be equivalent to the optimal solution of the discrete time problem. It is also shown that the proposed network model is stable in the Lyapunov sense and it is globally convergent to an exact optimal solution of the original problem. Several practical examples are provided to show the feasibility and the efficiency of the scheme.

Optimal Control of the Multiphase Stefan Problem

We consider the inverse multiphase Stefan problem, where information on the heat flux on the fixed boundary is missing and must be found along with the temperature and free boundaries. Optimal control framework is pursued, where boundary heat flux is the control, and the optimality criteria consist of the minimization of the $L_2$-norm declination of the trace of the solution to the Stefan problem from the temperature measurement on the fixed right boundary. The state vector solves multiphase Stefan problem in a weak formulation, which is equivalent to Neumann problem for the quasilinear parabolic PDE with discontinuous coefficient. Full discretization through finite differences is implemented and discrete optimal control problem is introduced. We prove well-posedness in a Sobolev space framework and convergence of discrete optimal control problems to the original problem both with respect to the cost functional and control. Along the way, the convergence of the method of finite differences for the weak solution of the multiphase Stefan problem is proved. The proof is based on achieving a uniform $L_{\infty}$ bound, and $W_2^{1,1}$-energy estimate for the discrete multiphase Stefan problem.

Turnpike Properties and Strict Dissipativity for Discrete Time Linear Quadratic Optimal Control Problems

We analyze the connection between turnpike behaviors and strict dissipativity properties for discrete time finite dimensional linear quadratic optimal control problems. We first use strict dissipativity as a sufficient condition for the turnpike property. Next, we characterize strict dissipativity and the newly introduced property of strict pre-dissipativity in terms of the system matrices related to the linear quadratic problem. These characterizations then lead to new necessary conditions for the turnpike properties under consideration, and thus eventually to necessary and sufficient conditions in terms of spectral criteria and matrix inequalities. One of the key novelties which distinguishes the results in the present paper from earlier ones on linear quadratic optimal control problems is the consideration of state and input constraints.

Optimal Control, Dichotomy, and Closed Range

The paper deals with optimality conditions of variational type for infinite horizon discrete time optimal control problems. We show that for unconstrained problems standard first order conditions e...

Variational multiscale method for the optimal control problems of convection--diffusion--reaction equations

In this paper, we analyze a projection-based variational multiscale (VMS) method for the optimal control problems governed by the convection-diffusion-reaction equations. We derive the first-order optimality conditions by the \emph{optimize-then-discretize} method. After expressing the discrete optimal control problem, we obtain the stability properties of state and adjoint variables. We also prove that the error in each variable is optimal. Through numerical examples, we show the efficiency of the stabilization for the solutions of the control, state, and adjoint variables.