Discrete-time Nonlinear Optimal Control Problems Research Articles

Optimistic planning for control of hybrid-input nonlinear systems

We propose two branch-and-bound, optimistic planning algorithms for discrete-time nonlinear optimal control problems in which there is a continuous and a discrete action (input). The dynamics and rewards (negative costs) must be Lipschitz but can otherwise be general, as long as certain boundedness conditions are satisfied by the continuous action, reward, and Lipschitz constant of the dynamics. We start by investigating the structure of the space of hybrid-input sequences. Based on this structure, we propose for the first algorithm an optimistic selection rule that picks for refinement (branching) the subset with the largest upper bound on the value. At the price of a higher budget, the second method reduces the reliance on the Lipschitz constant, by refining all sets that are potentially optimistic. This effectively means that the Lipschitz constant is automatically optimized. The way to select the largest-impact action along which to refine the sets is the same for both algorithms, and still depends on the Lipschitz constant. We provide convergence rate guarantees for both methods, which link the computational budget to the near-optimality of the action sequences returned, in a way that depends on a problem complexity measure. We also give empirical results for a nonlinear problem, where the algorithms are applied in receding horizon, and depending on the budget either one or the other algorithm works better.

Dissipativity properties in constrained optimal control: A computational approach

In this paper, we consider discrete-time nonlinear optimal control problems with possibly non-convex cost subject to constraints on states and inputs. For these problems, we present a computational approach for the verification of strict dissipativity properties, which have recently been employed to study optimal system operation, turnpike phenomena, and closed-loop properties of economic model predictive control schemes. Based on a non-strict dissipation inequality, we provide necessary and sufficient conditions for strict dissipativity by explicitly computing the set w.r.t. which the system is strictly dissipative as well as the corresponding storage function. We focus on strict dissipativity w.r.t. periodic orbits and steady-states, being the most relevant cases of strict dissipativity, although our approach can be directly extended to strict dissipativity w.r.t. more general sets. For polynomial optimal control problems, the presented approach leads to a polynomial optimization problem, which is solved via sum-of-squares programming. The optimal periodic orbit can then be constructed, without a priori knowledge of its period length, as the set of points for which a suitable non-strict dissipation inequality holds with equality. In addition, we consider the important special case of indefinite linear quadratic optimal control problems subject to quadratic constraints, for which an optimal periodic orbit resulting from our construction is always located on the boundary of the constraint set.

On a New and Efficient Numerical Technique to Solve a Class of Discrete-time Nonlinear Optimal Control Problems

On a New and Efficient Numerical Technique to Solve a Class of Discrete-time Nonlinear Optimal Control Problems

Discrete-Time Nonlinear Stochastic Optimal Control Problem Based on Stochastic Approximation Approach

In this paper, a computational approach is proposed for solving the discrete-time nonlinear optimal control problem, which is disturbed by a sequence of random noises. Because of the exact solution of such optimal control problem is impossible to be obtained, estimating the state dynamics is currently required. Here, it is assumed that the output can be measured from the real plant process. In our approach, the state mean propagation is applied in order to construct a linear model-based optimal control problem, where the model output is measureable. On this basis, an output error, which takes into account the differences between the real output and the model output, is defined. Then, this output error is minimized by applying the stochastic approximation approach. During the computation procedure, the stochastic gradient is established, so as the optimal solution of the model used can be updated iteratively. Once the convergence is achieved, the iterative solution approximates to the true optimal solution of the original optimal control problem, in spite of model-reality differences. For illustration, an example on a continuous stirred-tank reactor problem is studied, and the result obtained shows the applicability of the approach proposed. Hence, the efficiency of the approach proposed is highly recommended.

Finite-horizon inverse optimal control for discrete-time nonlinear systems

In this note, we consider the problem of computing the parameters (or weights) of an optimal control objective function given optimal closed-loop state and control trajectories. We establish a method of inverse optimal control that exploits the discrete-time minimum principle. Under a testable matrix rank condition, our proposed method is guaranteed to recover the unknown objective-function parameters of finite-horizon discrete-time nonlinear optimal control problems.

Continuous Nonlinear Programming Techniques to Solve Scheduling Problems

One of the most known applications of Discrete Optimization is on scheduling. In contrast, one of the most known applications of Continuous Nonlinear Optimization is on the control of dynamic systems. In this paper, we combine both views, solving scheduling problems as dynamic systems, modeled as discrete-time nonlinear optimal control problems with state and control continuous variables subjected to upper and lower bounds. Complementarity constraints are used to represent scheduling decisions. One example we discuss in detail is the crude oil scheduling in ports, with numerical results presented.

Nonlinear Optimal Control Applied to Coordinated Ramp Metering

The goal of this paper is to describe a generic approach to the problem of optimal coordinated ramp metering control in large-scale motorway networks. In this approach, the traffic flow process is macroscopically modeled by use of a second-order macroscopic traffic flow model. The overall problem of coordinated ramp metering is formulated as a constrained discrete-time nonlinear optimal control problem, and a feasible-direction nonlinear optimization algorithm is employed for its numerical solution. The control strategy's efficiency is demonstrated through its application to the 32-km Amsterdam ring road. A number of adequately chosen scenarios along with a thorough analysis, interpretation, and suitable visualization of the obtained results provides a basis for the better understanding of some complex interrelationships of partially conflicting performance criteria. More precisely, the strategy's efficiency and equity properties as well as their tradeoff are studied and their partially competitive behavior is discussed. The results of the presented approach are very promising and demonstrate the efficiency of the optimal control methodology for motorway traffic control problems.

A set oriented approach to global optimal control

We describe an algorithm for computing the value function for all source, single desti- nation discrete-time nonlinear optimal control problems together with approximations of associated globally optimal control strategies. The method is based on a set oriented approach for the dis- cretization of the problem in combination with graph-theoretic techniques. The central idea is that a discretization of phase space of the given problem leads to an (all source, single destination) shortest path problem on a finite graph. The method is illustrated by two numerical examples, namely a single pendulum on a cart and a parametrically driven inverted double pendulum. 1991 Mathematics Subject Classification. 49M25,49J53,65K10,90C39.

Application of Evolutionary Computation to Optimal On-ramp Traffic Control

In this paper, the numerical solution method of an on-ramp traffic control problem is discussed from the viewpoint of the application of evolutionary computation. The problem is formulated as a discrete-time nonlinear optimal control problem. The optimal control problem is then transferred to an optimization problem with successive constraints. A parallel-adaptive genetic algorithm is proposed to obtain the numerical solution of the problem. A real-time practical control method is also proposed. Numerical simulations for assessing the performance of the proposed methods are performed using observed data of Hanshin Expressway. As a result, the proposed method obtains the solution at comparatively small generations. This means shortening the calculation time, and is useful for the real-time control. Moreover, it is shown that the proposed real-time control is effective to reduce heavy congestion on the expressway.

Nonlinear Optimal Control Applied to Coordinated Ramp Metering in the Amsterdam Ring-Road

The goal of this paper is to describe a generic approach to the problem of optimal coordinated ramp metering control in large-scale motorway networks. In this approach, the traffic flow process is macroscopically modelled by use of a second-order macroscopic traffic flow model. The overall problem of coordinated ramp metering is formulated as a constrained discrete-time nonlinear optimal control problem, and a feasible-direction nonlinear optimisation algorithm is employed for its numerical solution. The control strategy's efficiency is demonstrated through its application to the 32-km Amsterdam ring-road for a number of different scenarios

Optimal Coordinated Ramp Metering with Advanced Motorway Optimal Control

A generic approach to the problem of optimal coordinated ramp metering control in large-scale motorway networks is described that is implemented in the software tool Advanced Motorway Optimal Control. In this approach, the traffic flow process is modeled by use of a secondorder macroscopic traffic flow model. The overall problem of coordinated ramp metering is formulated as a constrained discrete-time nonlinear optimal control problem, and a feasible-direction nonlinear optimization algorithm is employed for its numerical solution. The control strategy’s efficiency is demonstrated through its application to the 32-km long Amsterdam ring road. A number of different scenarios with regard to the number of controlled ramps and the available storage space are discussed in some detail. The results of the presented approach are very promising and demonstrate the high efficiency and general applicability of the optimal control methodology for motorway traffic control problems.

Nonlinear Optimal Control Applied To Coordinated Ramp Metering

The goal of this paper is to describe a generic approach to the problem of optimal coordinated ramp metering control in large-scale motorway networks. In this approach, the traffic flow process is macroscopically modelled by use of a second-order macroscopic traffic flow model. The overall problem of coordinated ramp metering is formulated as a constrained discrete-time nonlinear optimal control problem, and a feasible-direction nonlinear optimisation algorithm is employed for its numerical solution. The control strategy's efficiency is demonstrated through its application to the 32km Amsterdam ring-road for a number of different scenarios. The results of the presented approach are very promising and demonstrate the efficiency of the optimal control methodology for motorway traffic control problems.

The PMSS approach to large-scale optimal control problems

Large-scale optimal control problems arise in several different contexts from a variety of applications. Many general computational algorithms have been proposed for the solution of these problems and a large literature on the subject abounds. To assimilate this material, it is necessary to overcome several difficulties. For example, the large-scale nature of the problems induces a complicated notation and specialized jargon that sometimes confuses the basic issues. Further, the ad hoc nature of the development of many algorithms tends to add an air of mystery that is difficult to dispel. Consequently, it is not always apparent that there are simple concepts and principles that underlie most, if not all, of these algorithms. In this paper, the general taxonomic scheme proposed by Geoffrion is exploited to provide a systematic framework for viewing computational algorithms for solving large-scale, optimal control problems. To demonstrate the utility of the viewpoint, a new algorithm for solving large-scale, discrete-time nonlinear optimal control problems is presented. The development is directed by the simple concepts and principles that underlie the proposed taxonomy.