Finite-horizon Optimal Control Problem Research Articles

A Gradient Flow Equation for Optimal Control Problems With End-point Cost

In this paper, we consider a control system of the form dot x = F(x)u, linear in the control variable u. Given a fixed starting point, we study a finite-horizon optimal control problem, where we want to minimize a weighted sum of an end-point cost and the squared 2-norm of the control. This functional induces a gradient flow on the Hilbert space of admissible controls, and we prove a convergence result by means of the Lojasiewicz-Simon inequality. Finally, we show that, if we let the weight of the end-point cost tend to infinity, the resulting family of functionals is Γ-convergent, and it turns out that the limiting problem consists in joining the starting point and a minimizer of the end-point cost with a horizontal length-minimizer path.

Coordinated yaw stability control for extreme path tracking of 4WIDEVs based on predictive control

To improve safety and stability of four-wheel independent drive electric vehicles (4WIDEVs) in extreme path tracking maneuvers, a hierarchical yaw stability control scheme based on predictive control is proposed in this work. The upper-level stability control strategy coordinates active front steering (AFS) and direct yaw moment control (DYC), yielding desired front steering angle and longitudinal force of each tire. The control task is formulated into a finite horizon optimal control problem, which is solved by hybrid model predictive control (hMPC) algorithm. The lower-level slip ratio control strategy calculates and tracks the optimal longitudinal slip ratio of each tire for implementation of the longitudinal tire force distributions in a wide range of road friction coefficient. Based on a qualified piecewise affine (PWA) approximation of lateral tire force models, the stable regions of vehicle states under multiple varying inputs are obtained by analyzing trajectories of fixed points. Furthermore, the control modes of the upper-level control strategy with different control inputs and state constraints are determined by judging bifurcation point of system. Finally, the proposed control approach is verified and compared through the dSPACE-based hardware-in-the-loop (HIL) experiments. The results indicate that the proposed yaw stability control strategy can improve yaw stability performance and reduce the tracking error significantly in extreme situations.

Learning of model-plant mismatch map via neural network modeling and its application to offset-free model predictive control

We propose an improved offset-free model predictive control (MPC) framework, which learns and utilizes the intrinsic model-plant mismatch map, to effectively exploit the advantages of model-based and data-driven control strategies and overcome the limitation of each approach. In this study, the model-plant mismatch map on steady-state manifold is approximated via artificial neural network (ANN) modeling based on steady-state data from the process. Though the learned model-plant mismatch map can provide the information at the equilibrium point (i.e., setpoint), it cannot provide model-plant mismatch information during transient state. To handle this, we additionally apply a supplementary disturbance variable which is updated from a revised disturbance estimator considering the disturbance value obtained from the learned model-plant mismatch map. Then, the learned and supplementary disturbance variables are applied to the target problem and finite-horizon optimal control problem of the offset-free MPC framework. By this, the control system can utilize both the learned model-plant mismatch information and the stabilizing property of the nominal disturbance estimator. The closed-loop simulation results demonstrate that the proposed offset-free MPC scheme utilizing the model-plant mismatch map learned via ANN modeling efficiently improves the closed-loop reference tracking performance of the control system. Additionally, the zero-offset tracking condition of the developed framework is mathematically examined.

A New Approach to Finite-Horizon Optimal Control for Discrete-Time Affine Nonlinear Systems via a Pseudolinear Method

In this article, a new time-varying adaptivedynamic programming (ADP) algorithm is developed to solve finite-horizon optimal control problems for a class of discrete-time affine nonlinear systems. Inspired by the pseudolinear method, the nonlinear system can be approximated by a series of time-varying linear systems. In each iteration of the time-varying ADP algorithm, the optimal control law for the time-varying linear system is obtained. For an arbitrary initial state, it is proven that states of the time-varying linear systems converge to the states of discrete-time affine nonlinear systems. It is also shown that the iterative value functions and the iterative control laws converge to the optimal value function and the optimal control law, respectively. Finally, numerical results are presented to verify the effectiveness of the present method.

DIRECT: A Differential Dynamic Programming Based Framework for Trajectory Generation

This letter introduces a differential dynamic programming (DDP) based framework for polynomial trajectory generation for differentially flat systems. In particular, instead of using a linear equation with increasing size to represent multiple polynomial segments as in literature, we take a new perspective from state-space representation such that the linear equation reduces to a finite horizon control system with a fixed state dimension and the required continuity conditions for consecutive polynomials are automatically satisfied. Consequently, the constrained trajectory generation problem (both with and without time optimization) can be converted to a discrete-time finite-horizon optimal control problem with inequality constraints, which can be approached by a recently developed interior-point DDP (IPDDP) algorithm. Furthermore, for unconstrained trajectory generation with preallocated time, we show that this problem is indeed a linear-quadratic tracking (LQT) problem (DDP algorithm with exact one iteration). All these algorithms enjoy linear complexity with respect to the number of segments. Both numerical comparisons with state-of-the-art methods and physical experiments are presented to verify and validate the effectiveness of our theoretical findings. The implementation code will be open-sourced. <xref ref-type="fn" rid="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">¹</xref> <fn id="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><label>¹</label> [Online] Available: <uri>https://github.com/ntu-caokun/DIRECT</uri> </fn>

On a probabilistic approach to synthesize control policies from example datasets

This paper is concerned with the design of control policies from example datasets. The case considered is when just a black box description of the system to be controlled is available and the system is affected by actuation constraints. These constraints are not necessarily fulfilled by the (possibly, noisy) example data and the system under control is not necessarily the same as the one from which these data are collected. In this context, we introduce a number of methodological results to compute a control policy from example datasets that: (i) makes the behavior of the closed-loop system similar to the one illustrated in the data; (ii) guarantees compliance with the constraints. We recast the control problem as a finite-horizon optimal control problem and give an explicit expression for its optimal solution. Moreover, we turn our findings into an algorithmic procedure. The procedure gives a systematic tool to compute the policy. The effectiveness of our approach is illustrated via a numerical example, where we use real data collected from test drives to synthesize a control policy for the merging of a car on a highway.

Sub-optimal fixed-finite-horizon spacecraft configuration control on SE(3)

For achieving the desired configuration of spacecraft at the desired fixed time, a sub-optimal fixed-finite-horizon configuration control method on the Lie group SE(3) is developed based on the Model Predictive Static Programming (MPSP). The MPSP technique has been widely used to solve finite-horizon optimal control problems and is known for its high computational efficiency thanks to the closed-form solution, but it cannot be directly applied to systems on SE(3). The methodological innovation in this paper enables that the MPSP technique is extended to the geometric control on SE(3), using the variational principle, the left-invariant properties of Lie groups, and the topology structure of Lie algebra space. Moreover, the energy consumption, which is crucial for spacecraft operations, is considered as the objective function to be optimized in the optimal control formulation. The effectiveness of the designed sub-optimal control method is demonstrated through an online simulation under disturbances and state measurement errors.

Inverse dynamics‐based formulation of finite horizon optimal control problems for rigid‐body systems

Inverse dynamics‐based formulation of finite horizon optimal control problems for rigid‐body systems

Online predictive connected and automated eco-driving on signalized arterials considering traffic control devices and road geometry constraints under uncertain traffic conditions

For energy-efficient Connected and Automated Vehicle (CAV) Eco-driving control on signalized arterials under uncertain traffic conditions, this paper explicitly considers traffic control devices (e.g., road markings, traffic signs, and traffic signals) and road geometry (e.g., road shapes, road boundaries, and road grades) constraints in a data-driven optimization-based Model Predictive Control (MPC) modeling framework. This modeling framework uses real-time vehicle driving and traffic signal data via Vehicle-to-Infrastructure (V2I) and Vehicle-to-Vehicle (V2V) communications. In the MPC-based control model, this paper mathematically formulates location-based traffic control devices and road geometry constraints using the geographic information from High-Definition (HD) maps. The location-based traffic control devices and road geometry constraints have the potential to improve the safety, energy, efficiency, driving comfort, and robustness of connected and automated driving on real roads by considering interrupted flow facility locations and road geometry in the formulation. We predict a set of uncertain driving states for the preceding vehicles through an online learning-based driving dynamics prediction model. We then solve a constrained finite-horizon optimal control problem with the predicted driving states to obtain a set of Eco-driving references for the controlled vehicle. To obtain the optimal acceleration or deceleration commands for the controlled vehicle with the set of Eco-driving references, we formulate a Distributionally Robust Stochastic Optimization (DRSO) model (i.e., a special case of data-driven optimization models under moment bounds) with Distributionally Robust Chance Constraints (DRCC) with location-based traffic control devices and road geometry constraints. We design experiments to demonstrate the proposed model under different traffic conditions using real-world connected vehicle trajectory data and Signal Phasing and Timing (SPaT) data on a coordinated arterial with six actuated intersections on Fuller Road in Ann Arbor, Michigan from the Safety Pilot Model Deployment (SPMD) project.

Solving infinite-horizon optimalcontrol problems of the time-delayedsystems by a feed forward neural network model

ABSTRACT A numerical method using neural network for solving infinite-horizon time-delayed optimal control problems is studied. The problem is first transformed, using a Páde approximation, to one without a time-delayed argument. By a suitable change of variable, the obtained non-delay infinite-horizon optimal control problem is converted to a finite-horizon nonlinear optimal control problem. We try to approximate the solution of Hamiltonian conditions based on the Pontryagin minimum principle (PMP). For this purpose, we introduce an error function that contains all PMP conditions. We then minimize the error function where weights and biases associated with all neurons are unknown. Substituting the optimal values of the weights and biases in the trial solutions, we obtain the optimal solution of the original problem. Several examples are given to show the efficiency of the method.

Optimal Control and Stabilization for Discrete-Time Markov Jump Linear Systems with Input Delay

This paper examines the finite horizon optimal control problem and infinite horizon stabilization problem for Markov jump linear system (MJLS) with input delay. The essential obstacle encountered i...

Performance Assessment of Different Cell-Transmission Models for Ramp-Metered Highway Networks

Throughout the past three decades, various versions of the the cell-transmission model (CTM) has been proposed. In spite of the increased attention to freeway traffic modelling via the CTM, an analysis of the currently available versions of the CTM has been missing. This study has the aim of filling this gap by comparing the performance of the popular versions of this model. To achieve this goal, four finite horizon optimal control problems with different underlying CTMs and cost functions are proposed. The traffic management control in all problems is the ramp metering control. The performance assessment is performed via simulation for a hypothetical network of highways, with different demand profiles along with the analysis of the equilibrium state in each case. The simulation results will provide a thorough comparison of the performance of these problems on different performance measures of the network and will highlight the advantages and disadvantages of each problem.

Data-driven Nonlinear MPC using Dynamic Response Surface Methodology

Abstract For many complex processes, it is desirable to use a nonlinear model in the MPC design, and the recently proposed Dynamic Response Surface Methodology (DRSM) is capable of accurately modeling nonlinear continuous processes over semi-infinite time horizons. We exploit the DRSM to identify nonlinear data-driven dynamic models that are used in an NMPC. We demonstrate the ability and effectiveness of the DRSM data-driven model to be used as the prediction model for a nonlinear MPC regulator. This DRSM model is efficiently used to solve a non-equally-spaced finite-horizon optimal control problem so that the number of decision variables is reduced. The proposed DRSM-based NMPC is tested on a representative nonlinear process, an isothermal CSTR in which a second-order irreversible reaction is taking place. It is shown that the obtained quadratic data-driven model accurately represents the open-loop process dynamics and that DRSM-based NMPC is an effective data-driven implementation of nonlinear MPC.

Classical System Theory Revisited for Turnpike in Standard State Space Systems and Impulse Controllable Descriptor Systems

The concept of turnpike connects the solution of long but finite time horizon optimal control problems with steady-state optimal controls. A key ingredient of the analysis of turnpike phenomena is ...

Multi-Commodity Traffic Signal Control and Routing With Connected Vehicles

A real-time traffic management policy that integrates traffic signal control and multi-commodity routing of connected vehicles in networks with multiple destinations is developed. The proposed policy is based on a multi-commodity formulation of the store-and-forward model and assumes all vehicles are able to exchange information with the infrastructure. Vehicles share information about their current location and final destination. Based on this information, the strategy determines both optimized signal timings at every intersection and vehicle-specific routing information at every link of the network. The control actions, i.e., signal times and routing information, are updated at every cycle and delivered by a finite horizon optimal control problem cast into a rolling horizon framework. The underlying optimization problem is convex, and thus the method is suitable for real-time operation in large networks. The method is validated via a micro-simulation study in networks with up to twenty intersections and, in all simulations, outperforms a real-time traffic-responsive signal control strategy that is based on a single-commodity store-and-forward model. The scalable computation effort for increasing network sizes and prediction horizon confirms the computational efficiency of the method.

Proportional-Integral Projected Gradient Method for Model Predictive Control

Recently there has been an increasing interest in primal-dual methods for model predictive control (MPC), which require minimizing the (augmented) Lagrangian at each iteration. We propose a novel first order primal-dual method, termed proportional-integral projected gradient method, for MPC where the underlying finite horizon optimal control problem has both state and input constraints. Instead of minimizing the (augmented) Lagrangian, each iteration of our method only computes a single projection onto the state and input constraint set. Our method ensures that, along a sequence of averaged iterates, both the distance to optimum and the constraint violation converge to zero at a rate of O(1/k) if the objective function is convex, where k is the iteration number. If the objective function is strongly convex, this rate can be improved to O(1/k <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) for the distance to optimum and O(1/k <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ) for the constraint violation. We compare our method against existing methods via a trajectory-planning example with convexified keep-out-zone constraints.

Economic model predictive control for transport-reaction systems with target profiles

For transport-reaction processes with certain general cost functions indicating comprehensive profit, the existing linear–quadratic optimal control or model predictive control (MPC) methods may be conservative. Due to their spatial characteristics, dynamic evolutions and final target static spatial distributions under different control strategies may result in remarkable variation of operation performance. This article presents a novel economic model predictive control (EMPC) synthesis for systems described by hyperbolic/parabolic partial differential equations (PDEs) with pre-optimized static spatially distributed profile, from both convergence and performance points of view. Instead of deriving approximate ordinary differential equations (ODEs) and applying finite-dimensional control strategies, this work addresses the optimization and stabilization problem on the basis of infinite dimensional model itself, which is realized by Cayley–Tustin (CT) transformation without any type of spatial approximation. The accumulated economic cost, input/output constraints are explicitly considered in the finite horizon optimal control problem (FHOCP), with an additional energy-like contractive constraint accounting for closed-loop convergence toward the target optimal steady-state profile. By directly solving Lyapunov functions associated with PDEs models, the parameters in the FHOCP are suitably defined such that the composite EMPC system admits algorithm feasibility and closed-loop stability. Finally, the proposed method is applied to typical examples; numerical results indicates the effectiveness and remarkable performance improvement of the EMPC method compared to standard MPC results.

Interior Point Differential Dynamic Programming

This brief introduces a novel differential dynamic programming (DDP) algorithm for solving discrete-time finite-horizon optimal control problems with inequality constraints. Two variants, namely feasible- and infeasible-IPDDP algorithms, are developed using a primal–dual interior-point methodology, and their local quadratic convergence properties are characterized. We show that the stationary points of the algorithms are the perturbed KKT points, and thus can be moved arbitrarily close to a locally optimal solution. Being free from the burden of the active-set methods, it can handle nonlinear state and input inequality constraints without a discernible increase in its computational complexity relative to the unconstrained case. The performance of the proposed algorithms is demonstrated using numerical experiments on three different problems: control-limited inverted pendulum, car-parking, and unicycle motion control and obstacle avoidance.

Finite-Horizon Optimal Control of Boolean Control Networks: A Unified Graph-Theoretical Approach.

This article investigates the finite-horizon optimal control (FHOC) problem of Boolean control networks (BCNs) from a graph theory perspective. We first formulate two general problems to unify various special cases studied in the literature: 1) the horizon length is a priori fixed and 2) the horizon length is unspecified but finite for given destination states. Notably, both problems can incorporate time-variant costs, which are rarely considered in existing work, and a variety of constraints. The existence of an optimal control sequence is analyzed under mild assumptions. Motivated by BCNs' finite state space and control space, we approach the two general problems intuitively and efficiently under a graph-theoretical framework. A weighted state transition graph and its time-expanded variants are developed, and the equivalence between the FHOC problem and the shortest-path (SP) problem in specific graphs is established rigorously. Two algorithms are developed to find the SP and construct the optimal control sequence for the two problems with reduced computational complexity, though technically, a classical SP algorithm in graph theory is sufficient for all problems. Compared with existing algebraic methods, our graph-theoretical approach can achieve state-of-the-art time efficiency while targeting the most general problems. Furthermore, our approach is the first one capable of solving Problem 2) with time-variant costs. Finally, a genetic network in the bacterium E. coli and a signaling network involved in human leukemia are used to validate the effectiveness of our approach. The results of two common tasks for both networks show that our approach can dramatically reduce the running time. Python implementation of our algorithms is available at GitHub https://github.com/ShuhuaGao/FHOC.

A Fixed-Point Characterization of the Optimal Costate in Finite-Horizon Optimal Control Problems

A fixed-point characterization of the optimal costate in finite-horizon optimal control problems for nonlinear systems is presented. It is shown that the optimal initial condition of the costate variable must be a fixed-point, for any time, of the composition of the forward and backward flows of the underlying Hamiltonian dynamics. Such an abstract property is then translated into a constructive condition by relying on a sequence of repeated Lie brackets involving the Hamiltonian dynamics and evaluated at a single point in the state space. This leads to a system of algebraic equations in the unknown initial optimal costate that allows achieving a desired degree of accuracy of the approximation while always consisting of a number of equations equal to the dimension of the state of the underlying system, regardless of the achieved accuracy. A dual characterization of the optimal terminal value of the state is also discussed, together with a few computational aspects of the proposed strategy.