A dynamic programming-inspired approach for Mixed Integer Optimal Control Problems with dwell time constraints

This thesis treats different aspects of the class of Mixed-Integer Optimal Control Problems (MIOCPs). These are optimization problems that combine the difficulties of underlying dynamic processes with combinatorial decisions. Typically, these combinatorial decisions are realized as switching decisions between the system’s different operations modes. During the last decades, direct methods emerged as the state-of-the-art solvers for MIOCPs. The formulation of a valid, tight and dependable integral relaxation, i.e., the formulation of a model for fractional values, plays an important role for these direct solution methods. We give detailed insight into several relaxation approaches for MIOCPs and compare them with regard to their respective structures. In particular, these are the typical solution’s structures and properties as convexity, problem size and numerical behavior. From these structural properties, we deduce some required specifications of a solver. Additionally, the modeling and subsequent limitation of the switching process directly tackle the class-specific typical issue of chattering solutions. One of the relaxation methods for MIOCPs is the outer convexification, where the binary variables only enter affinely. For the approximation of this relaxation’s solution, we took up on the control approximation problem in integral sense derived by Sager as part of a decomposition approach for MIOCPs with affine binary controls. This problem describes the optimal approximation of fractional controls with binary controls such that the corresponding dynamic process is changed as little as possible. For the multi-dimensional problem, we developed a new heuristic, which for the first time gives a bound that only depends on the control grid and not anymore on the number of the system’s controls. For the generalization of the control approximation problem with additional constraints, we derived a tailored branch-and-bound algorithm, which is based on the properties of the Lagrangian relaxation of the one-dimensional problem. This algorithm beats state-of-the-art commercial solvers for Mixed-Integer Linear Programs (MILPs) for this special approximation problem by several orders of magnitude. Overall, we present several, partially new modeling approaches for MIOCPs together with the accompanying structural properties. On this basis, we develop new theories for the approximation of certain relaxed solutions. We discuss the efficient implementation of the resulting structure exploiting algorithms. This leads to a deeper and better understanding of MIOCPs. We show the practicability of the theoretical observations with the help of four prototypical problems. The presented methods and algorithms allow on their basis the direct development of decision support and analysis tools in practice.

Tailored Mixed-Integer Optimal Control policies for real-world applications usually have to avoid very short successive changes of the active integer control. Minimum dwell time (MDT) constraints express this requirement and can be included into the combinatorial integral approximation decomposition, which solves mixed-integer optimal control problems (MIOCPs) to epsilon -optimality by solving one continuous nonlinear program and one mixed-integer linear program (MILP). Within this work, we analyze the integrality gap of MIOCPs under MDT constraints by providing tight upper bounds on the MILP subproblem. We suggest different rounding schemes for constructing MDT feasible control solutions, e.g., we propose a modification of Sum Up Rounding. A numerical study supplements the theoretical results and compares objective values of integer feasible and relaxed solutions.

Mixed integer optimal control problems are a generalization of ordinary optimal control problems that include additional integer valued control functions. The integer control functions are used to switch instantaneously from one system to another. We use a time transformation (similar as in [1]) to get rid of the integer valued functions. This allows to apply gradient based optimization methods to approximate the mixed integer optimal control problem. The time transformation from [1] is adapted such that problems with distinct state domains for each system can be solved and it is combined with the direct discretization method DMOC [2,3] (Discrete Mechanics and Optimal Control) to approximate trajectories of the underlying optimal control problems. Our approach is illustrated with the help of a first example, the hybrid mass oscillator. (© 2014 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

In this chapter we extend the problem class of continuous optimal control problems discussed in chapter 1 to include control functions that may at each point in time attain only a finite number of values from a discrete set. We briefly survey different approaches for the solution of the discretized Mixed–Integer Optimal Control Problem (MIOCP), such as Dynamic Programming, branching techniques, and Mixed–Integer Nonlinear Programming methods. These approaches however turn out to be computationally very demanding already for off–line optimal control, and this fact becomes even more apparent in a real–time on–line context. The introduction of an outer convexification and relaxation approach for the MIOCP class allows to obtain an approximation of the MIOCP’s solution by solving only one single but potentially much larger continuous optimal control problem using the techniques of chapter 1. We describe theoretical properties of this problem reformulation that provide bounds on the loss of optimality for the infinite dimensional MIOCP. Rounding schemes are presented for the discretized case that maintain these guarantees. We develop techniques for the introduction of switch costs into the class of MIOCPs and give reformulations that allow for a combination with direct multiple shooting and outer convexification.KeywordsOptimal Control ProblemSwitch CostControl TrajectoryInteger Nonlinear ProgramPath ConstraintThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

<abstract><p>This paper considers an optimal feedback control problem for a class of fed-batch fermentation processes. Our main contributions are as follows. Firstly, a dynamic optimization problem for fed-batch fermentation processes is modeled as an optimal control problem of switched dynamical systems, and a general state-feedback controller is designed for this dynamic optimization problem. Unlike the existing switched dynamical system optimal control problem, the state-dependent switching method is applied to design the switching rule, and the structure of this state-feedback controller is not restricted to a particular form. Then, this problem is transformed into a mixed-integer optimal control problem by introducing a discrete-valued function. Furthermore, each of these discrete variables is represented by using a set of 0-1 variables. By using a quadratic constraint, these 0-1 variables are relaxed such that they are continuous on the closed interval $ [0, 1] $. Accordingly, the original mixed-integer optimal control problem is transformed intoa nonlinear parameter optimization problem. Unlike the existing works, the constraint introduced for these 0-1 variables are at most quadratic. Thus, it does not increase the number of locally optimal solutions of the original problem. Next, an improved gradient-based algorithm is developed based on a novel search approach, and a large number of numerical experiments show that this novel search approach can effectively improve the convergence speed of this algorithm, when an iteration is trapped to a curved narrow valley bottom of the objective function. Finally, numerical results illustrate the effectiveness of this method developed by this paper.</p></abstract>

The solutions of mixed integer optimal control problems (MIOCPs) yield optimized trajectories for dynamical systems with instantly changing dynamical behavior. The instant change is caused by a changing value of the integer valued control function. For example, a changing integer value can cause a car to change the gear, or a mechanical system to close a contact. The direct discretization of a MIOCP leads to a mixed integer nonlinear program (MINLP) and can not be solved with gradient based optimization methods at once. We extend the work by Gerdts [1] and reformulate a MIOCP with integer dependent constraints as an ordinary optimal control problem (OCP). The discretized OCP can be solved using gradient based optimization methods. We show how the integer dependent constraints can be used to model systems with impact and present optimized trajectories of computational examples, namely of a lockable double pendulum and an acyclic telescope walker. (© 2016 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

We present a control problem for an electrical vehicle. Its motor can be operated in two discrete modes, leading either to acceleration and energy consumption, or to a recharging of the battery. Mathematically, this leads to a mixed-integer optimal control problem (MIOCP) with a discrete feasible set for the controls taking into account the electrical and mechanical dynamic equations. The combination of nonlinear dynamics and discrete decisions poses a challenge to established optimization and control methods, especially if global optimality is an issue. Probably for the first time, we present a complete analysis of the optimal solution of such a MIOCP: solution of the integer-relaxed problem both with a direct and an indirect approach, determination of integer controls by means of the sum up rounding strategy, and calculation of global lower bounds by means of the method of moments. As we decrease the control discretization grid and increase the relaxation order, the obtained series of upper and lower bounds converge for the electrical car problem, proving the asymptotic global optimality of the calculated chattering behavior. We stress that these bounds hold for the optimal control problem in function space, and not on an a priori given (typically coarse) control discretization grid, as in other approaches from the literature. This approach is generic and is an alternative to global optimal control based on probabilistic or branch-and-bound based techniques. The main advantage is a drastic reduction of computational time. The disadvantage is that only local solutions and certified lower bounds are provided with no possibility to reduce these gaps. For the instances of the electrical car problem, though, these gaps are very small. The main contribution of the paper is a survey and new combination of state-of-the-art methods for global mixed-integer optimal control and the in-depth analysis of an important, prototypical control problem. Despite the comparatively low dimension of the problem, the optimal solution structure of the relaxed problem exhibits a series of bang-bang, path-constrained, and sensitivity-seeking arcs.

Mixed-integer optimal control problems (MIOCPs) frequently arise in the domain of optimal control problems (OCPs) when decisions including integer variables are involved. However, existing state-of-the-art approaches for solving MIOCPs are often plagued by drawbacks such as high computational costs, low precision, and compromised optimality. In this study, we propose a novel multiphase scheme coupled with an iterative second-order cone programming (SOCP) algorithm to efficiently and effectively address these challenges in MIOCPs. In the first phase, we relax the discrete decision constraints and account for the terminal state constraints and certain path constraints by introducing them as penalty terms in the objective function. After formulating the problem as a quadratically constrained quadratic programming (QCQP) problem, we propose the iterative SOCP algorithm to solve general QCQPs. In the second phase, we reintroduce the discrete decision constraints to generate the final solution. We substantiate the efficacy of our proposed multiphase scheme and iterative SOCP algorithm through successful application to two practical MIOCPs in planetary exploration missions.

Optimal control problems with mixed integer control functions and logical implications, such as a state-dependent restriction on when a control can be chosen (so-called indicator or vanishing constraints) frequently arise in practice. A prominent example is the optimal cruise control of a truck. As every driver knows, admissible gear choices critically depend on the current velocity. A large variety of approaches has been proposed on how to numerically solve this challenging class of control problems. We present a computational study in which the most relevant of them are compared for a reference model problem, based on the same discretization of the differential equations. This comprehends dynamic programming, implicit formulations of the switching decisions, and a number of explicit reformulations, including mathematical programs with vanishing constraints in function spaces. We survey all of these approaches in a general manner, where several formulations have not been reported in the literature before. We apply them to a benchmark truck cruise control problem and discuss advantages and disadvantages with respect to optimality, feasibility, and stability of the algorithmic procedure, as well as computation time.

This thesis is concerned with numerical methods for Mixed-Integer Optimal Control Problems with Combinatorial Constraints. We establish an approximation theorem relating a Mixed-Integer Optimal Control Problem with Combinatorial Constraints to a continuous relaxed convexified Optimal Control Problems with Vanishing Constraints that provides the basis for numerical computations. We develop a a Vanishing- Constraint respecting rounding algorithm to exploit this correspondence computationally. Direct Discretization of the Optimal Control Problem with Vanishing Constraints yield a subclass of Mathematical Programs with Equilibrium Constraints. Mathematical Programs with Equilibrium Constraint constitute a class of challenging problems due to their inherent non-convexity and non-smoothness. We develop an active-set algorithm for Mathematical Programs with Equilibrium Constraints and prove global convergence to Bouligand stationary points of this algorithm under suitable technical conditions. For efficient computation of Newton-type steps of Optimal Control Problems, we establish the Generalized Lanczos Method for trust region problems in a Hilbert space context. To ensure real-time feasibility in Online Optimal Control Applications with tracking-type Lagrangian objective, we develop a Gaus-Newton preconditioner for the iterative solution method of the trust region problem. We implement the proposed methods and demonstrate their applicability and efficacy on several benchmark problems.

We describe a set of extensions to the AMPL modeling language to conveniently model mixed-integer optimal control problems for ODE or DAE dynamic processes. These extensions are realized as AMPL user functions and suffixes and do not require intrusive changes to the AMPL language standard or implementation itself. We describe and provide TACO, a Toolkit for AMPL Control Optimization that reads AMPL stub.nl files and detects the structure of the optimal control problem. This toolkit is designed to facilitate the coupling of existing optimal control software packages to AMPL. We discuss requirements, capabilities, and the current implementation. Using the example of the multiple shooting code for optimal control MUSCOD-II, a direct and simultaneous method for DAE-constrained optimal control, we demonstrate how the problem information provided by the TACO toolkit is interfaced to the solver. In addition, we show how the MS-MINTOC algorithm for mixed-integer optimal control can be used to efficiently solve mixed-integer optimal control problems modeled in AMPL. We use the AMPL extensions to model three control problem examples and we discuss how those extensions affect the representation of optimal control problems. Solutions to these problems are obtained by using MUSCOD-II and MS-MINTOC inside the AMPL environment. A collection of further AMPL control models is provided on the web site http://mintoc.de . MUSCOD-II and MS-MINTOC have been made available on the NEOS Server for Optimization, using the TACO toolkit to enable input of AMPL models.

This note discusses properties of parametric discrete-time Mixed-Integer Optimal Control Problems (MIOCPs) as they often arise in model predictive control with discrete controls. We argue that, in want for a handle on similarity properties of parametric MIOCPs, the turnpike phenomenon known in optimal control is helpful. We provide sufficient turnpike conditions based on a dissipativity notion of MIOCPs, and we prove that the turnpike phenomenon allows specific and accurate guesses for the discrete controls. We also derive an easily checkable sufficient condition for dissipativity of linear-quadratic MIOCPs. Moreover, we show how the turnpike property can be used to derive efficient node-weighted branch-and-bound schemes tailored to parametric MIOCPs. We draw upon numerical examples to illustrate our findings.

We consider mixed-integer optimal control problems with combinatorial constraints that couple over time such as minimum dwell times. We analyze a lifting and decomposition approach into a mixed-integer optimal control problem without combinatorial constraints and a mixed-integer problem for the combinatorial constraints in the control space. Both problems can be solved very efficiently with existing methods such as outer convexification with sum-up-rounding strategies and mixed-integer linear programming techniques. The coupling is handled using a penalty-approach. We provide an exactness result for the penalty which yields a solution approach that convergences to partial minima. We compare the quality of these dedicated points with those of other heuristics amongst an academic example and also for the optimization of electric transmission lines with switching of the network topology for flow reallocation in order to satisfy demands.

Modeling Mixed-Integer Constrained Optimal Control Problems in AMPL

Many engineering optimisation problems can be summarised as mixed-integer optimal control problems (MIOCPs) owing to the needs for mixed-integer dynamic control decisions. However, the convergence theory of Legendre-Gauss-Radau (LGR) approximation fails to apply to such non-smooth and discontinuous optimal control problems. Therefore, this paper develops an extended multi-interval LGR pseudospectral method (EMLGR), which has the following features: (i) the mixed-integer controls at the end of each interval and the interval intersections are added as two new controls to avoid the unrestrained control and shorten the switching time of integer control, and (ii) a smart adaptive collocation monitor (SACM) is provided to optimise the polynomial order and interval structure for further reducing computational complexity and improving approximation precision. The detailed solution procedure of EMLGR is given in this study, and experimental studies including five challenging practical engineering MIOCPs are taken to verify the superiorities of the proposed EMLGR in efficiency, accuracy and stability.