Terminal State Constraints Research Articles

A Necessary Condition for Optimal Control of Forward-Backward Stochastic Control System with Lévy Process in Nonconvex Control Domain Case

This paper analyzes one kind of optimal control problem which is described by forward-backward stochastic differential equations with Lévy process (FBSDEL). We derive a necessary condition for the existence of the optimal control by means of spike variational technique, while the control domain is not necessarily convex. Simultaneously, we also get the maximum principle for this control system when there are some initial and terminal state constraints. Finally, a financial example is discussed to illustrate the application of our result.

Nonlinear model predictive control for models in quasi‐linear parameter varying form

SummaryThis article presents a nonlinear model predictive control (NMPC) approach based on quasi‐linear parameter varying (quasi‐LPV) representations of the model and constraints. Stability of the proposed algorithm is ensured by the offline solution of an optimization problem with linear matrix inequality constraints in conjunction with an online terminal state constraint. Furthermore, an iterative approach is presented with which the NMPC optimization problem can be handled by solving a series of Quadratic Programs at each time step, this being highly computationally efficient. A practical and simple way of obtaining quasi‐LPV representations of the system using velocity‐based linearization is presented in two examples.

Continuous Viscosity Solutions to Linear-Quadratic Stochastic Control Problems with Singular Terminal State Constraint

This paper establishes the existence of a unique nonnegative continuous viscosity solution to the HJB equation associated with a linear-quadratic stochastic control problem with singular terminal state constraint and possibly unbounded cost coefficients. The existence result is based on a novel comparison principle for semi-continuous viscosity sub- and supersolutions for PDEs with singular terminal value. Continuity of the viscosity solution is enough to carry out the verification argument.

An improved solution method via the pole-transformation process for the maximum-crossrange problem

The maximum-crossrange problem is difficult to solve rapidly and stably because of highly nonlinear dynamics and nonconvex constraints. In this paper, an improved successive second-cone programming method via the pole-transformation process is proposed. The main contribution of this work is twofold. Firstly, the pole-transformation coordinate frame is defined, and the original problem is reformulated by the pole-transformation process to avoid the linearization of downrange and crossrange angles. The terminal state constraint is relaxed to an inequality constraint, which improves the stability performance of the solution method. Secondly, an adaptively adjustment method of the trust-region contraction coefficient is proposed to improve the convergence performance of the solution method. The optimal solution of the original problem is obtained by the improved successive second-order cone programming method. The numerical experiments verify the effectiveness of the pole-transformation process, and the convergence and stability performances of the improved solution method proposed in this paper.

Robust Economic Model Predictive Control Based on a Zonotope and Local Feedback Controller for Energy Dispatch in Smart-Grids Considering Demand Uncertainty

Electrical smart grids are complex MIMO systems whose operation can be noticeably affected by the presence of uncertainties such as load demand uncertainty. In this paper, based on a restricted representation of the demand uncertainty, we propose a robust economic model predictive control method that guarantees an optimal energy dispatch in a smart micro-grid. Load demands are uncertain, but viewed as bounded. The proposed method first decomposes control inputs into dependent and independent components, and then tackles the effect of demand uncertainty by tightening the system constraints as the uncertainty propagates along the prediction horizon using interval arithmetic and local state feedback control law. The tightened constraints’ upper and lower limits are computed off-line. The proposed method guarantees stability through a periodic terminal state constraint. The method is faster and simpler compared to other approaches based on Closed-loop min–max techniques. The applicability of the proposed approach is demonstrated using a smart micro-grid that comprises a wind generator, some photovoltaic (PV) panels, a diesel generator, a hydroelectric generator and some storage devices linked via two DC-buses, from which load demands can be adequately satisfied.

Exact penalty functions for optimal control problems II: Exact penalization of terminal and pointwise state constraints

SummaryThe second part of our study is devoted to an analysis of the exactness of penalty functions for optimal control problems with terminal and pointwise state constraints. We demonstrate that with the use of the exact penalty function method one can reduce fixed‐endpoint problems for linear time‐varying systems and linear evolution equations with convex constraints on the control inputs to completely equivalent free‐endpoint optimal control problems, if the terminal state belongs to the relative interior of the reachable set. In the nonlinear case, we prove that a local reduction of fixed‐endpoint and variable‐endpoint problems to equivalent free‐endpoint ones is possible under the assumption that the linearized system is completely controllable, and point out some general properties of nonlinear systems under which a global reduction to equivalent free‐endpoint problems can be achieved. In the case of problems with pointwise state inequality constraints, we prove that such problems for linear time‐varying systems and linear evolution equations with convex state constraints can be reduced to equivalent problems without state constraints, provided one uses the L∞ penalty term, and Slater's condition holds true, while for nonlinear systems a local reduction is possible, if a natural constraint qualification is satisfied. Finally, we show that the exact Lp‐penalization of state constraints with finite p is possible for convex problems, if Lagrange multipliers corresponding to the state constraints belong to Lp′, where p′ is the conjugate exponent of p, and for general nonlinear problems, if the cost functional does not depend on the control inputs explicitly.

Vertical hierarchical MPC for constrained linear systems

A hierarchical Model Predictive Control (MPC) formulation is presented for discrete-time linear systems with state and input constraints. A vertical hierarchical controller, with one controller per level, reduces the computational burden associated with the solution of centralized MPC having long prediction horizons and short time steps. To guarantee satisfaction of state and input constraints in the presence of both known and unknown disturbances, a robust MPC formulation is used at each level while waysets are used as a novel coordination mechanism between controllers at different levels. These waysets are implemented as terminal state constraints on lower-level controllers and are computed on-line based on the optimal state trajectory of upper-level controllers. To achieve the computational efficiency necessary for on-line calculation, waysets are represented as constrained zonotopes. State and input constraint satisfaction is proven for a hierarchical controller with an arbitrary number of levels and two numerical examples demonstrate the key features, performance, and scalability of the approach.

Mean-Field Leader-Follower Games with Terminal State Constraint

We analyze linear McKean-Vlasov forward-backward SDEs arising in leader-follower games with mean-field type control and terminal state constraints on the state process. We establish an existence and uniqueness of solutions result for such systems in time-weighted spaces as well as a {convergence} result of the solutions with respect to certain perturbations of the drivers of both the forward and the backward component. The general results are used to solve a novel single-player model of portfolio liquidation under market impact with expectations feedback as well as a novel Stackelberg game of optimal portfolio liquidation with asymmetrically informed players.

Optimal control with irregular performance

In this paper, we solve the long-standing fundamental problem of irregular linear--quadratic (LQ) optimal control, which has received significant attention since the 1960s. We derive the optimal controllers via the key technique of finding the analytical solutions to two different forward and backward differential equations (FBDEs). We give a complete solution to the finite-horizon irregular LQ control problem using a new `two-layer optimization' approach. We also obtain the necessary and sufficient condition for the existence of optimal and stabilizing solutions in the infinite-horizon case in terms of solutions to two Riccati equations and the stabilization of one specific system. For the first time, we explore the essential differences between irregular and standard LQ control, making a fundamental contribution to classical LQ control theory. We show that irregular LQ control is totally different from regular control as the irregular controller must guarantee the terminal state constraint of $P_1(T)x(T)=0$.

Solar-sail deep space trajectory optimization using successive convex programming

This paper presents a novel methodology for solving the time-optimal trajectory optimization problem for interplanetary solar-sail missions using successive convex programming. Based on the non-convex problem, different convexification technologies, such as change of variables, successive linearization, trust regions and virtual control, are discussed to convert the original problem into the formulation of successive convex programming. Because of the free final-time, successive linearization is performed iteratively for the nonconvex terminal state constraints. After the convexification process, each of problems becomes a convex problem, which can be solved effectively. An augmented objective function is introduced to ensure the convergence performance and effectiveness of our algorithm. After that, algorithms are designed to solve the discrete sub-problems in a successive solution procedure. Finally, numerical results demonstrate the effectiveness and accuracy of our algorithms.

Robust economic model predictive control based on a periodicity constraint

SummaryThis paper proposes robust economic model predictive control based on a periodicity constraint for linear systems subject to unknown‐but‐bounded additive disturbances. In this economic MPC design, a periodic steady‐state trajectory is not required and thus assumed unknown, which precludes the use of enforcing terminal state constraints as in other standard economic formulations. Instead, based on the desired periodicity of system operation, we optimize the economic performance over a set of periodic trajectories that include the current state. To achieve robust constraint satisfaction, we use a tube‐based technique in the economic MPC formulation. The mismatches between the nominal model and the closed‐loop system with perturbations are limited using a local control law. With the proposed robust tube‐based strategy, recursive feasibility is guaranteed. Moreover, under a convexity assumption, the closed‐loop convergence of the closed‐loop system is analyzed, and an optimality certificate is provided to check if the closed‐loop trajectory reaches a neighborhood of the optimal nominal periodic steady trajectory using Karush‐Kuhn‐Tucker optimality conditions. Finally, through numerical examples, we show the effectiveness of the proposed approach.

Super-replication with fixed transaction costs

We study super-replication of contingent claims in markets with fixed transaction costs. This can be viewed as a stochastic impulse control problem with a terminal state constraint. The first result in this paper reveals that in reasonable continuous time financial market models the super-replication price is prohibitively costly and leads to trivial buy-and-hold strategies. Our second result derives nontrivial scaling limits of super-replication prices for binomial models with small fixed costs.

Minimizing almost smooth control variation in nonlinear optimal control problems

In this paper, we consider an optimal control problem in which the control is almost smooth and the state and control are subject to terminal state constraints and continuous state and control inequality constraints. By introducing an extra set of differential equations for this almost smooth control, we transform this constrained optimal control problem into an equivalent problem involving both control function and system parameter vector as decision variables. Then, by the control parametrization technique and a time scaling transformation, the equivalent problem is approximated by a sequence of constrained optimal parameter selection problems, each of which is a finite dimensional optimization problem. For each of these constrained optimal parameter selection problems, a novel exact penalty function method is constructed by appending penalized constraint violations to the cost function. This gives rise to a sequence of unconstrained optimal parameter selection problems; and each of which can be solved by existing optimization algorithms or software packages. Finally, a practical container crane operation problem is solved, showing the effectiveness and applicability of the proposed approach.

Free finite horizon LQR: A bilevel perspective and its application to model predictive control

We consider linear-quadratic optimal control problems with free final time and terminal state constraints and propose a solution procedure that is particularly useful for online feedback control in a model-predictive control (MPC) framework. The procedure avoids the standard time transformation, which transforms the problem into an equivalent but non-convex optimal control problem on a fixed time horizon. The transformed problem typically suffers from many local minima, which might cause instabilities in online optimization tasks like LQR or model-predictive control. To avoid this drawback of the time transformation we develop a method from the viewpoint of bilevel optimal control, which is beneficial especially in online control tasks. The novelty of the approach is the optimal final time tracking procedure, for which we exploit a property of the Hamiltonian function. To this end we show that within the continuous-time closed-loop controlled system the optimal final time linearly decreases in time, as one could intuitively expect. Finally, numerical experiments support the effectiveness of the proposed algorithm.

Experimental validation of model predictive control stability for autonomous driving

This paper addresses the design of time-varying model predictive control of an autonomous vehicle in the presence of input rate constraints such that closed-loop stability is guaranteed. Stability is proved via Lyapunov techniques by adding a terminal state constraint and a terminal cost to the controller formulation. The terminal set is the maximum positive invariant set of a multi-plant description of the vehicle linear time-varying model. The terminal cost is an upper-bound on the infinite cost-to-go incurred by applying a linear–quadratic regulator control law. The proposed control design is experimentally tested and successfully stabilizes an autonomous Scania construction truck in an obstacle avoidance scenario.

Optimal-tuning of proportional-integral-derivative-like controller for constrained nonlinear systems and application to ship steering control

In this paper, we consider an optimal-tuning problem of proportional-integral-derivative-like (PID-like) controller for constrained nonlinear systems with continuous-time inequality constraints and a terminal state constraint. Due to the complexity of such constraints, it is difficult to solve this problem by using conventional optimization methods. To overcome this difficulty, a constraint transcription approach and a smoothing function are applied to these continuous–time inequality constraints. An augmented cost functional is obtained by appending these continuous-time inequality constraints to the cost functional. Then, this optimal–tuning problem can be converted into an approximate optimal parameter selection problem. The gradient formulae of the augmented cost function and the terminal constraint function are derived, and a gradient-based numerical algorithm is developed for solving this approximate problem. Convergence results show that there only exist a finite number of iterates such that the sequence obtained by using the algorithm is infeasible, and any optimal solution of the approximate problem is also an optimal solution of the original problem. Finally, a ship steering control problem is solved to illustrate the effectiveness of the approach proposed by us.

Integral and measure-turnpike properties for infinite-dimensional optimal control systems

We first derive a general integral-turnpike property around a set for infinite-dimensional non-autonomous optimal control problems with any possible terminal state constraints, under some appropriate assumptions. Roughly speaking, the integral-turnpike property means that the time average of the distance from any optimal solution to the turnpike set converges to zero, as the time horizon tends to infinity. Then, we establish the measure-turnpike property for strictly dissipative optimal control systems, with state and control constraints. The measure-turnpike property, which is slightly stronger than the integral-turnpike property, means that any optimal solution remains essentially, during the time frame, close to an optimal solution of an associated static optimal control problem, except during a subset of time frame that is of small relative Lebesgue measure as the time horizon is large. Motivated by a specific example of optimal control problem for the heat equation with control constraints, we next prove that strict strong duality, which is a classical notion in optimization, implies strict dissipativity, and measure-turnpike. Finally, we conclude the paper with several comments and open problems.

Maximum Principle of Optimal Stochastic Control with Terminal State Constraint and Its Application in Finance

This paper considers the optimal control problem for a general stochastic system with general terminal state constraint. Both the drift and the diffusion coefficients can contain the control variable and the state constraint here is of non-functional type. The author puts forward two ways to understand the target set and the variation set. Then under two kinds of finite-codimensional conditions, the stochastic maximum principles are established, respectively. The main results are proved in two different ways. For the former, separating hyperplane method is used; for the latter, Ekeland’s variational principle is applied. At last, the author takes the mean-variance portfolio selection with the box-constraint on strategies as an example to show the application in finance.

Linear Quadratic Stochastic Control Problems with Stochastic Terminal Constraint

We study a linear quadratic optimal control problem with stochastic coefficients and a terminal state constraint, which may be in force merely on a set with positive, but not necessarily full, probability. Under such a partial terminal constraint, the usual approach via a coupled system of a backward stochastic Riccati equation and a linear backward equation breaks down. As a remedy, we introduce a family of auxiliary problems parametrized by the supersolutions to this Riccati equation alone. The target functional of these problems dominates the original constrained one and allows for an explicit description of both the optimal control policy and the auxiliary problem's value in terms of a suitably constructed optimal signal process. This suggests that, for the minimal supersolution of the Riccati equation, the minimizers of the auxiliary problem coincide with those of the original problem, a conjecture that we see confirmed in all examples understood so far.

A Maximum Principle for Fully Coupled Forward-Backward Stochastic Control System Driven by Lévy Process with Terminal State Constraints

This paper is concerned with a fully coupled forward-backward stochastic optimal control problem where the controlled system is driven by Levy process, while the forward state is constrained in a convex set at the terminal time. The authors use an equivalent backward formulation to deal with the terminal state constraint, and then obtain a stochastic maximum principle by Ekeland’s variational principle. Finally, the result is applied to the utility optimization problem in a financial market.