Discrete-time Optimal Control Problem Research Articles

An Optimal Control Problem Related to the RSS Model

In this paper, we consider a discrete-time optimal control problem related to the model of Robinson, Solow and Srinivasan. We analyze this optimal control problem without concavity assumptions on a non-concave utility function which represents the preferences of the planner and establish the existence of good programs and optimal programs which are Stiglitz production programs.

A Smoothing Method for Ramp Metering

Ramp metering offers great potential to mitigate traffic congestion and improve freeway management efficiency under traffic congestion conditions. This paper proposes an optimization program for freeway dynamic ramp metering based on Cell Transmission Model (CTM). This problem has been formulated as a discrete time optimal control problem with smooth state equations and constraints to meter traffic inflow from on-ramps. In the proposed model, the ‘min’ operators in the primal CTM are non-differentiable and thus, the corresponding optimal control problem cannot be solved directly using conventional gradient based methods. In this paper, we introduce a smooth approximation to approximate the ‘min’ operators and then a unified computational approach is developed to solve the problem. Theoretical analysis is carried out, showing that the optimal solution obtained from the approximated problem converges to the optimal solution of the primal CTM. Compared to the classical inequality relaxation method, our method can resolve the flow holding-back problem and reduce under fundamental diagram phenomenon. Compared with the Big-M method, our method has better efficiency. To achieve the desired traffic response control in real application, a series of online optimal control problems are solved using Model Predictive Control (MPC). Simulation studies show that our method can significantly improve freeway traffic management efficiency.

Temporal deep unfolding for constrained nonlinear stochastic optimal control

This paper proposes a computational technique to solve discrete-time optimal control problems for nonlinear systems subject to stochastic disturbances, hard input constraints, and soft constraints. The approach employs the idea of deep unfolding, which is a recently developed model-based deep learning method that is applicable to iterative algorithms. By regarding each state transition of the dynamical system as an iteration step, these state transitions are unfolded into the layers of a deep neural network. This produces a computational graph that contains trainable parameters that determine control inputs. These parameters are optimized by training the deep neural network using the standard deep learning technique of backpropagation and incremental training. Then, the optimal control inputs are obtained from the optimized parameters. This provides a way to optimize the control inputs that otherwise would be difficult to obtain. The effectiveness of the proposed technique is demonstrated by numerical experiments for the maximum hands-off control problem and constrained model predictive control problems using a highly nonlinear model of a continuous stirred tank reactor.

Hidden invariant convexity for global and conic-intersection optimality guarantees in discrete-time optimal control

Non-convex discrete-time optimal control problems in, e.g., water or power systems, typically involve a large number of variables related through nonlinear equality constraints. The ideal goal is to find a globally optimal solution, and numerical experience indicates that algorithms aiming for Karush–Kuhn–Tucker points often find solutions that are indistinguishable from global optima. In our paper, we provide a theoretical underpinning for this phenomenon, showing that on a broad class of problems the objective can be shown to be an invariant convex function (invex function) of the control decision variables when state variables are eliminated using implicit function theory. In this way, optimality guarantees can be obtained, the exact nature of which depends on the position of the solution within the feasible set. In a numerical example, we show how high-quality solutions are obtained with local search for a river control problem where invexity holds.

Optimal harvesting strategy for hairtail, Trichiurus Lepturus, in Korea Sea using discrete-time age-structured model

Hairtail, Trichiurus Lepturus, is the major fishery resource in Korea and so has been occupying a large portion of total catches for a longtime. However, since the 1980s, the catch has fallen sharply. A climatic change and over-fishing of immature hairtails have decreased the resource in Korea Sea, which caused reduction in the catch. So, we should analyze this phenomenon and need to manage them. Thus, in this study, we first propose a discrete time age-structured model of hairtails from their biological features. Especially, since hairtail is the representative fish with characteristics of cannibalism in the marine ecosystem, we set the recruitment as Ricker type. Furthermore, for the environmental effect on its spawning, the sea surface temperature in the spawning ground is included in the model. Next, we find an optimal harvesting strategy for preserving the immature hairtails while simultaneously maximizing the corresponding profit of the fishing. The discrete-time optimal control problem is conducted as the extension of Pontryagin’s Maximum Principle to discrete system. Moreover, to suggest the optimal harvesting strategy in the future, the forecast of sea surface temperature should be accompanied. This is carried out by wavelet-ARIMA forecasting method. By using numerical simulations, we obtain that in the perspective of conservation, the limited fishing is the best for management of hairtails. On the other hand, in terms of balance between the conservation and economic profits, the optimal harvesting strategy is better than the limited fishing.

Online inverse optimal control for control-constrained discrete-time systems on finite and infinite horizons

In this paper, we consider the problem of computing parameters of an objective function for a discrete-time optimal control problem from state and control trajectories with active control constraints. We propose a novel method of inverse optimal control that has a computationally efficient online form in which pairs of states and controls from given state and control trajectories are processed sequentially without being stored or processed in batches. We establish conditions guaranteeing the uniqueness of the objective-function parameters computed by our proposed method from trajectories with active control constraints. We illustrate our proposed method in simulation.

Optimal Sizing and Control of Battery Energy Storage Systems for Hybrid Turboelectric Aircraft

<div class="section abstract"><div class="htmlview paragraph">Hybrid-electric gas turbine generators are considered a promising technology for more efficient and sustainable air transportation. The Ohio State University is leading the NASA University Leadership Initiative (ULI) <i>Electric Propulsion: Challenges and Opportunities</i>, focused on the design and demonstration of advanced components and systems to enable high-efficiency hybrid turboelectric powertrains in regional aircraft to be deployed in 2030. Within this large effort, the team is optimizing the design of the battery energy storage system (ESS) and, concurrently, developing a supervisory energy management strategy for the hybrid system to reduce fuel burn while mitigating the impact on the ESS life. In this paper, an energy-based model was developed to predict the performance of a battery-hybrid turboelectric distributed-propulsion (BHTeDP) regional jet. A study was conducted to elucidate the effects of ESS sizing and cell selection on the optimal power split between the turbogenerators (TGs) and ESS. To this extent, the supervisory energy management strategy is formulated into a discrete time optimal control problem and solved via dynamic programming. The performance of BHTeDP was compared to a turboelectric distributed-propulsion (TeDP) next-gen aircraft that assumes improvements in weight, drag, and engine efficiency consistent with regional jet entering operation in 2035.</div></div>

A simple proof of the discrete time geometric Pontryagin maximum principle on smooth manifolds

We establish a geometric Pontryagin maximum principle for discrete time optimal control problems on finite dimensional smooth manifolds under the following three types of constraints: a) constraints on the states pointwise in time, b) constraints on the control actions pointwise in time, c) constraints on the frequency spectrum of the optimal control trajectories. Our proof follows, in spirit, the path to establish geometric versions of the Pontryagin maximum principle on smooth manifolds indicated in Chang (2011) in the context of continuous-time optimal control.

Discrete Time Pontryagin Maximum Principle Under State-Action-Frequency Constraints

We establish a Pontryagin maximum principle for discrete time optimal control problems under the following three types of constraints: a) constraints on the states pointwise in time, b) constraints on the control actions pointwise in time, and c) constraints on the frequency spectrum of the optimal control trajectories. While the first two types of constraints are already included in the existing versions of the Pontryagin maximum principle, it turns out that the third type of constraints cannot be recast in any of the standard forms of the existing results for the original control system. We provide two different proofs of our Pontryagin maximum principle in this article, and include several special cases fine-tuned to control-affine nonlinear and linear system models. In particular, for minimization of quadratic cost functions and linear time invariant control systems, we provide tight conditions under which the optimal controls under frequency constraints are either normal or abnormal.

Discrete time optimal control with frequency constraints for non-smooth systems

We present a Pontryagin maximum principle for discrete time optimal control problems with (a) pointwise constraints on the control actions and the states, (b) smooth frequency constraints on the control and the state trajectories, and (c) nonsmooth dynamical systems. Pointwise constraints on the states and the control actions represent desired and/or physical limitations on the states and the control values; such constraints are important and are widely present in the optimal control literature. Constraints of type (b), while less standard in the literature, effectively serve the purpose of describing important properties of inertial actuators and systems. The conjunction of constraints of type (a) and (b) is a relatively new phenomenon in optimal control but is important for the synthesis control trajectories with a high degree of fidelity. The maximum principle established here provides first order necessary conditions for optimality that serve as a starting point for the synthesis of control trajectories corresponding to a large class of constrained motion planning problems that have high accuracy in a computationally tractable fashion. Moreover, the ability to handle a reasonably large class of nonsmooth dynamical systems that arise in practice ensures broad applicability of our theory.

The maximum principle for discrete-time control systems and applications to dynamic games

We study deterministic nonstationary discrete-time optimal control problems in both finite and infinite horizon. With the aid of Gâteaux differentials, we prove a discrete-time maximum principle in analogy with the well-known continuous-time maximum principle. We show that this maximum principle, together a transversality condition, is a necessary condition for optimality; we also show that it is sufficient under additional hypothesis. We use Gâteaux differentials as a natural setting to derive first-order conditions. Additionally, we use the discrete-time maximum principle to derive the discrete-time Euler equation and to characterize Nash equilibria for discrete-time dynamic games.

Strong Lagrange duality and the maximum principle for nonlinear discrete time optimal control problems

We examine discrete-time optimal control problems with general, possibly non-linear or non-smooth dynamic equations, and state-control inequality and equality constraints. A new generalized convexity condition for the dynamics and constraints is defined, and it is proved that this property, together with a constraint qualification constitute sufficient conditions for the strong Lagrange duality result and saddle-point optimality conditions for the problem. The discrete maximum principle of Pontryagin is obtained in a straightforward manner from the strong Lagrange duality theorem, first in a new form in which the Lagrangian is minimized both with respect to the state and to the control variables. Assuming differentiability, the maximum principle is obtained in the usual form. It is shown that dynamic systems satisfying a global controllability condition with convex costs, have the required convexity property. This controllability condition is a natural extension of the customary directional convexity condition applied in the derivation of the discrete maximum principle for local optima in the literature.

Optimal stochastic sequential control with fractional rewards

In this paper we discuss the optimal stopping problem for multiparameter stochastic processes with the index set Nd, and in general, the optimal sequential control problem for stochastic processes with the index set consisting of increasing paths. We give the formulation of new control problems which combine the discrete time optimal sequential control problem with the criterion of a fractional reward. Using the standard mathematical argument in the theory of mathematical programming, we prove the existence of optimal stopping points and optimal controls for these problems under the certain assumption on the regularity of the stochastic processes. Further we discuss ε – optimal controls, finite time controls, and Markov case, respectively.

A discrete-time Pontryagin maximum principle on matrix Lie groups

In this article we derive a Pontryagin maximum principle (PMP) for discrete-time optimal control problems on matrix Lie groups. The PMP provides first order necessary conditions for optimality; these necessary conditions typically yield two point boundary value problems, and these boundary value problems can then be solved to extract optimal control trajectories. Constrained optimal control problems for mechanical systems, in general, can only be solved numerically, and this motivates the need to derive discrete-time models that are accurate and preserve the non-flat manifold structures of the underlying continuous-time controlled systems. The PMPs for discrete-time systems evolving on Euclidean spaces are not readily applicable to discrete-time models evolving on non-flat manifolds. In this article we bridge this gap and establish a discrete-time PMP on matrix Lie groups. Our discrete-time models are derived via discrete mechanics, (a structure preserving discretization scheme) leading to the preservation of the underlying manifold under the dynamics, thereby resulting in greater numerical accuracy of our technique. This PMP caters to a class of constrained optimal control problems that includes point-wise state and control action constraints, and encompasses a large class of control problems that arise in various field of engineering and the applied sciences.

Symmetry reduction for dynamic programming

We present a method of exploiting symmetries of discrete-time optimal control problems to reduce the dimensionality of dynamic programming iterations. The results are derived for systems with continuous state variables, and can be applied to systems with continuous or discrete symmetry groups. We prove that symmetries of the state update equation and stage costs induce corresponding symmetries of the optimal cost function and the optimal policies. We then provide a general framework for computing the optimal cost function based on gridding a space of lower dimension than the original state space. This method does not require algebraic manipulation of the state update equations; it only requires knowledge of the symmetries that the state update equations possess. Since the method can be performed without any knowledge of the state update map beyond being able to evaluate it and verify its symmetries, this enables the method to be applied in a wide range of application problems. We illustrate these results on two six-dimensional optimal control problems that are computationally difficult to solve by dynamic programming without symmetry reduction.

The neuro-dynamic scheme for solving general form of discrete time optimal control problems

In this paper, we show that recently developed neural network methods for quadratic programming can be put to use in solving discrete time optimal control problems, with general pointwise constraints on states and controls. We describe a high performance recurrent neural network for a discrete time linear quadratic regulator problem with mixed state–control constraints. The equilibrium point of the proposed model is proved to be equivalent to the optimal solution of the discrete time problem. It is also shown that the proposed network model is stable in the Lyapunov sense and it is globally convergent to an exact optimal solution of the original problem. Several practical examples are provided to show the feasibility and the efficiency of the scheme.

Solving sparse polynomial optimization problems with chordal structure using the sparse bounded-degree sum-of-squares hierarchy

The sparse bounded degree sum-of-squares (sparse-BSOS) hierarchy of Weisser et al. (2017) constructs a sequence of lower bounds for a sparse polynomial optimization problem. Under some assumptions, it is proved by the authors that the sequence converges to the optimal value. In this paper, we modify the hierarchy to deal with problems containing equality constraints directly, without eliminating or replacing them by two inequalities. We also evaluate the sparse-BSOS hierarchy on a well-known bilinear programming problem, called the pooling problem, as well as a discrete-time optimal control problem.

Optimal Control, Dichotomy, and Closed Range

The paper deals with optimality conditions of variational type for infinite horizon discrete time optimal control problems. We show that for unconstrained problems standard first order conditions e...

Optimal control of the discrete-time fractional-order Cucker-Smale model

We obtain necessary optimality conditions for the discrete-time fractional-order Cucker-Smale optimal control problem. By using fractional order differences on the left side of nonlinear system we introduce memory effects to the considered problem.

Discrete-time interval optimal control problem

ABSTRACTThis paper proposes a new approach for the discrete-time interval optimal control problem. The optimal interval solution of discrete-time interval optimal control problem is obtained using the single-level constrained interval arithmetic coupled with a dynamic programming technique. Moreover, the optimal interval solution contains the real-valued optimal solutions. To illustrate an optimal solution for the interval case, we use the minimax regret criterion in the interval inventory control problem we present.