Finite-horizon Optimal Control Research Articles

An exact convex relaxation of the freeway network control problem with controlled merging junctions

We consider the freeway network control problem where the aim is to optimize the operation of traffic networks modeled by the cell transmission model via ramp metering and partial mainline demand control. Optimal control problems using the cell transmission model are usually non-convex, due to the nonlinear fundamental diagram, but a convex relaxation in which demand and supply constraints are relaxed is often used. Previous works have established conditions under which solutions of the relaxation can be made feasible with respect to the original constraints. In this work, we generalize these conditions and show that the control of flows into merging junctions is sufficient to do so if the objective is to minimize the total time spent in traffic. We derive this result by introducing an alternative system representation. In the new representation, the system dynamics are concave and state-monotone. We show that exactness of the convex relaxation of finite horizon optimal control problems follows from these properties. Deriving the main result via a characterization of the system dynamics allows one to treat arbitrary monotone, concave fundamental diagrams and several types of control for merging junctions in a uniform manner. The derivation also suggests a straightforward method to verify if the results continue to hold for extensions or modifications of the models studied in this work.

Solutions to Constrained Optimal Control Problems with Linear Systems

This paper is devoted to present solutions to constrained finite-horizon optimal control problems with linear systems, and the cost functional of the problem is in a general form. According to the Pontryagin’s maximum principle, the extremal control of such problem is a function of the costate trajectory, but an implicit function. We here develop the canonical backward differential flows method and then give the extremal control explicitly with the costate trajectory by canonical backward differential flows. Moreover, there exists an optimal control if and only if there exists a unique extremal control. We give the proof of the existence of the optimal solution for this optimal control problem with Green functions.

Algebraic Approach to Nonlinear Optimal Control Problems with Terminal Constraints: Sufficient Conditions for Existence of Algebraic Solutions

This paper studies nonlinear finite-horizon optimal control problems with terminal constraints, where all nonlinear functions are rational or algebraic functions. We first extend a recursive elimin...

Designing a closed-loop guidance system to increase the accuracy of satellite-carrier boosters' landing point

A novel closed loop guidance method is provided in this paper to increase the accuracy of satellite-carrier boosters' landing point. The proposed method can be used in the first stage of flight vehicles that fly in atmosphere. In this case, solid motor propelled boosters would be able to more accurately land on the desired point. Moreover, in sub-optimal technique, the closed-loop guidance system would produce commands after booster separation, which guide second stage of satellite-carrier from initial conditions to the desired condition of orbit injection. In this method, sub-optimal integrated solution of control and guidance in closed-loop is developed. This sub-optimal technique named as Model Predictive Static Programming (MPSP) that is based on nonlinear optimal control theory and derived from combined philosophies of Model Predictive Control and Approximate Dynamic Programming; solves a class of finite horizon optimal control problems with terminal constraints. Furthermore because sensitivity matrices that are necessary for obtaining this solution can be computed recursively, this technique is computationally efficient and is appropriate for online implementation. In this paper, the dynamic equations of system are modeled in the presence of aerodynamic loading and the servo-mechanism dynamic. Moreover, by considering integrated guidance and control loops, a solution of the guidance and control system is proposed by three-degree of freedom spherical earth simulation model in atmosphere. Result show that proposed closed-loop guidance not only is able to remove aerodynamic and thrust modeling errors in first stage by flight data update that caused to more accurate of boosters' landing point, but also would still be able to guide second stage of satellite-carrier to the desired condition of placement in the orbit.

Deep reinforcement learning based finite-horizon optimal tracking control for nonlinear system

Abstract Reinforcement learning (RL) can be used to obtain an approximate numerical solution to the Hamilton-Jacobi-Bellman (HJB) equation. Recent advances in machine learning community enable the use of deep neural networks (DNNs) to approximate high-dimensional nonlinear functions as those that occur in RL, accurately without any domain knowledge. In the standard RL setting, both system and cost structures are unknown, and the amount of data needed to obtain an accurate approximation can be impractically large. Meanwhile, when the structures are known, they can be used to solve the HJB equation efficiently. Herein, the model-based globalized dual heuristic programming (GDHP) is proposed, in which the HJB equation is separated into value, costate, and policy functions. A particular class of interest in this research is finite horizon optimal tracking control (FHOC) problem. Additional issues that arise, such as time-varying functions, terminal constraints, and delta-input formulation, are addressed in the context of FHOC. The DNN structure and training algorithm suitable for FHOC are presented. A benchmark continuous reactor example is provided to illustrate the proposed approach.

Data-based predictive control via direct weight optimization

Abstract In this paper we propose a novel data-based predictive control scheme in which the prediction model is obtained from a linear combination of past system trajectories. The proposed controller optimizes the weights of this linear combination taking into account simultaneously performance and the variance of the estimation error. For unconstrained systems, dynamic programming is used to obtain an explicit linear solution of a finite or infinite horizon optimal control problem. When constraints are taken into account, the controller needs to solve online a quadratic optimization problem to obtain the optimal weights, possibly considering also local information to improve the performance and estimation. A simulation example of the application of the proposed controller to a quadruple-tank system is provided.

Robust adaptive gain higher order sliding mode observer based control-constrained nonlinear model predictive control for spacecraft formation flying

This work deals with the development of a decentralized optimal control algorithm, along with a robust observer, for the relative motion control of spacecraft in leader-follower based formation. An adaptive gain higher order sliding mode observer has been proposed to estimate the velocity as well as unmeasured disturbances from the noisy position measurements. A differentiator structure containing the Lipschitz constant and Lebesgue measurable control input, is utilized for obtaining the estimates. Adaptive tuning algorithms are derived based on Lyapunov stability theory, for updating the observer gains, which will give enough flexibility in the choice of initial estimates. Moreover, it may help to cope with unexpected state jerks. The trajectory tracking problem is formulated as a finite horizon optimal control problem, which is solved online. The control constraints are incorporated by using a nonquadratic performance functional. An adaptive update law has been derived for tuning the step size in the optimization algorithm, which may help to improve the convergence speed. Moreover, it is an attractive alternative to the heuristic choice of step size for diverse operating conditions. The disturbance as well as state estimates from the higher order sliding mode observer are utilized by the plant output prediction model, which will improve the overall performance of the controller. The nonlinear dynamics defined in leader fixed Euler-Hill frame has been considered for the present work and the reference trajectories are generated using Hill-Clohessy-Wiltshire equations of unperturbed motion. The simulation results based on rigorous perturbation analysis are presented to confirm the robustness of the proposed approach.

Optimization-Based Design of Bounded-Error Estimators Robust to Missing Data

Non-asymptotic bounded-error state estimators that provide hard bounds on the estimation error are crucial for safety-critical applications. This paper proposes a class of optimal bounded-error affine estimators to achieve a novel property we are calling Equalized Recovery that can be computed by leveraging ideas from the dual problem of affine finite horizon optimal control design. In particular, by using Q-parametrization, the estimator design problem is reduced to a convex optimization problem. An extension of this estimator to handle missing data (e.g., due to package drops or sensor glitches) is also proposed. These ideas are illustrated with a numerical example motivated by vehicle safety systems.

Formation Control of Multiple Unmanned Aerial Vehicles by Event-Triggered Distributed Model Predictive Control

This paper proposes an event-triggered model predictive control (MPC) scheme for the formation control of multiple unmanned aerial vehicles (UAVs). A distributed MPC framework is designed in which each UAV only shares the information with its neighbors, and the obtained local finite-horizon optimal control problem (FHOCP) can be solved by a swarm intelligent optimization algorithm. An event-triggered mechanism is proposed to reduce the computational burden for the distributed MPC scheme, which takes into consideration the predictive state errors as well as the convergence of cost function. Furthermore, a safe-distance-based strategy for no-fly zone avoidance is developed and integrated into the local cost function for each FHOCP. Numerical simulations show that the proposed event-triggered distributed MPC is more computationally efficient to achieve formation control of multiple UAVs in comparison with the traditional distributed MPC method.

Loss of Optimal Performance of the Finite-Horizon Continuous-Time Linear-Quadratic Controller Driven by a Reduced-Order Observer

In this paper, we derive an expression for the loss of optimal performance (compared to the corresponding linear-quadratic optimal performance with the instantaneous full-state feedback) when the continuous-time finite-horizon linear-quadratic optimal controller uses the estimates of the state variables obtained via a reduced-order observer. It was shown that the loss of optimal performance value can be found by solving the differential Lyapunov equation whose dimensions are equal to dimensions of the reduced-order observer. A proton exchange membrane fuel cell example is included to demonstrate the loss of optimal performance as a function of the final time. It can be seen from the simulation results that the loss of optimal performance value can be very large. The loss of optimal performance value can be drastically reduced by using the proposed least-square formulas for the choice of the reduced-order observer initial conditions.

Impact of Quantisation on Consistency of DMPC in Street Traffic with Dynamic Priority Rules

AbstractDistributed systems typically require communication to steer the overall system to a reference or target state and keep it there. We consider an intersection scenario as a spacial set with vehicles as multi‐agents with individual initial and target conditions. Utilising a Distributed Model Predictive Control scheme (DMPC), in every time step each vehicle solves an finite horizon optimal control problem. The resulting optimal state trajectory is projected onto a quantisation of the spatial set and the quantised trajectories are broadcasted to the other vehicles to ensure collision avoidance. The quantisation is mainly motivated to reduce the necessary communication effort among the agents. Here, we introduce prediction coherence as difference of two predictions in two successive time steps. We numerically explore the idea to utilise prediction coherence to derive a lower bound for the communication requirements among the vehicles. (© 2017 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Model checking optimal finite-horizon control for probabilistic gene regulatory networks

BackgroundProbabilistic Boolean networks (PBNs) have been proposed for analyzing external control in gene regulatory networks with incorporation of uncertainty. A context-sensitive PBN with perturbation (CS-PBNp), extending a PBN with context-sensitivity to reflect the inherent biological stability and random perturbations to express the impact of external stimuli, is considered to be more suitable for modeling small biological systems intervened by conditions from the outside. In this paper, we apply probabilistic model checking, a formal verification technique, to optimal control for a CS-PBNp that minimizes the expected cost over a finite control horizon.ResultsWe first describe a procedure of modeling a CS-PBNp using the language provided by a widely used probabilistic model checker PRISM. We then analyze the reward-based temporal properties and the computation in probabilistic model checking; based on the analysis, we provide a method to formulate the optimal control problem as minimum reachability reward properties. Furthermore, we incorporate control and state cost information into the PRISM code of a CS-PBNp such that automated model checking a minimum reachability reward property on the code gives the solution to the optimal control problem. We conduct experiments on two examples, an apoptosis network and a WNT5A network. Preliminary experiment results show the feasibility and effectiveness of our approach.ConclusionsThe approach based on probabilistic model checking for optimal control avoids explicit computation of large-size state transition relations associated with PBNs. It enables a natural depiction of the dynamics of gene regulatory networks, and provides a canonical form to formulate optimal control problems using temporal properties that can be automated solved by leveraging the analysis power of underlying model checking engines. This work will be helpful for further utilization of the advances in formal verification techniques in system biology.

Data-Driven Finite-Horizon Approximate Optimal Control for Discrete-Time Nonlinear Systems Using Iterative HDP Approach.

This paper presents a data-based finite-horizon optimal control approach for discrete-time nonlinear affine systems. The iterative adaptive dynamic programming (ADP) is used to approximately solve Hamilton-Jacobi-Bellman equation by minimizing the cost function in finite time. The idea is implemented with the heuristic dynamic programming (HDP) involved the model network, which makes the iterative control at the first step can be obtained without the system function, meanwhile the action network is used to obtain the approximate optimal control law and the critic network is utilized for approximating the optimal cost function. The convergence of the iterative ADP algorithm and the stability of the weight estimation errors based on the HDP structure are intensively analyzed. Finally, two simulation examples are provided to demonstrate the theoretical results and show the performance of the proposed method.

Sufficient optimality conditions for distributed, non-predictive ramp metering in the monotonic cell transmission model

We consider the ramp metering problem for a freeway stretch modeled by the Cell Transmission Model. Assuming perfect model knowledge and perfect traffic demand prediction, the ramp metering problem can be cast as a finite horizon optimal control problem with the objective of minimizing the Total Time Spent, i.e., the sum of the travel times of all drivers. For this reason, the application of Model Predictive Control (MPC) to the ramp metering problem has been proposed. However, practical tests on freeways show that MPC may not outperform simple, distributed feedback policies. Until now, a theoretical justification for this empirical observation was lacking. This work compares the performance of distributed, non-predictive policies to the optimal solution in an idealised setting, specifically, for monotonic traffic dynamics and assuming perfect model knowledge. To do so, we suggest a distributed, non-predictive policy and derive sufficient optimality conditions for the minimization of the Total Time Spent via monotonicity arguments. In a case study based on real-world traffic data, we demonstrate that these optimality conditions are only rarely violated. Moreover, we observe that the suboptimality resulting from such infrequent violations appears to be negligible. We complement this analysis with simulations in non-ideal settings, in particular allowing for model mismatch, and argue that Alinea, a successful, distributed ramp metering policy, comes close to the ideal controller both in terms of control behavior and in performance.

Second-Order Switching Time Optimization for Switched Dynamical Systems

Switching time optimization arises in finite-horizon optimal control for switched systems where, given a sequence of continuous dynamics, one minimizes a cost function with respect to the switching times. We propose an efficient method for computing the optimal switching times for switched linear and nonlinear systems. A novel second-order optimization algorithm is introduced where, at each iteration, the dynamics are linearized over an underlying time grid to compute the cost function, the gradient and the Hessian efficiently. With the proposed method, the most expensive operations at each iteration are shared between the cost function and its derivatives, thereby greatly reducing the computational burden. We implemented the algorithm in the Julia package SwitchTimeOpt allowing the user to easily solve switching time optimization problems. In the case of linear dynamics, many operations can be further simplified and benchmarks show that our approach is able to provide optimal solutions in just a few ms. In the case of nonlinear dynamics, two examples show that our method provides optimal solutions with up to two orders of magnitude time reductions over state-of-the-art approaches.

Thermal Management in Plug-In Hybrid Electric Vehicles: A Real-Time Nonlinear Model Predictive Control Implementation

A real-time nonlinear model predictive control (NMPC) for the thermal management (TM) of the electrical components cooling circuit in a Plug-In Hybrid Electric Vehicle (PHEV) is presented. The electrical components are highly temperature sensitive and, therefore, working out of the ranges recommended by the manufacturer can lead to their premature aging or even failure. Consequently, the goals for an accurate and efficient TM are to keep the main component, the Li-ion battery, within optimal working temperatures, and to consume the minimum possible electrical energy through the cooling circuit actuators. This multi-objective requirement is formulated as a finite-horizon optimal control problem (OCP) that includes a multi-objective cost function, several constraints, and a prediction model especially suitable for optimization. The associated NMPC is performed on real time by the optimization package MUSCOD-II and is validated in three different repeatable test-drives driven with a PHEV. Starting from identical conditions, each cycle is driven once being the cooling circuit controlled with NMPC and once with a conventional approach based on a finite-state machine. Compared to the conventional strategy, the NMPC proposed here results in a more accurate and healthier temperature performance, and at the same time, leads to reductions in the electrical consumption up to 8%.

Robust LQR for time-inhomogeneous Markov jump switched linear systems

Markov jump switched linear systems (MJSLSs) are switched linear systems, where a switching signal is governed by a Markov decision process. We consider a polytopic time-inhomogeneous setting, where the transition probabilities (between the operational modes of the system) associated to each discrete action are varying over time, with variations that are arbitrary within a polytopic set. We present and solve for this class of systems the finite horizon optimal control problem.

Real-Time Motion Planning for Aerial Videography With Dynamic Obstacle Avoidance and Viewpoint Optimization

We propose a method for real-time trajectory generation with applications in aerial videography. Taking framing objectives, such as position of targets in the image plane, as input, our method solves for robot trajectories and gimbal controls automatically and adapts plans in real time due to changes in the environment. We contribute a real-time receding horizon planner that autonomously records scenes with moving targets, while optimizing for visibility under occlusion and ensuring collision-free trajectories. A modular cost function, based on the reprojection error of targets, is proposed that allows for flexibility and artistic freedom and is well behaved under numerical optimization. We formulate the minimization problem under constraints as a finite horizon optimal control problem that fulfills aesthetic objectives, adheres to nonlinear model constraints of the filming robot and collision constraints with static and dynamic obstacles and can be solved in real time. We demonstrate the robustness and efficiency of the method with a number of challenging shots filmed in dynamic environments including those with moving obstacles and shots with multiple targets to be filmed simultaneously.

Forecast-Triggered Fault-Tolerant Model Predictive Control of Nonlinear Processes

In this work, we consider the problem of fault-tolerant stabilization of constrained nonlinear processes controlled over resource-constrained sensor-controller communication networks and subject to control actuator faults. A methodology for the design of resource-aware Lyapuonv-based model predictive control (MPC) systems that achieve the fault-tolerant stabilization objective with reduced sensor-controller communication is presented. In this approach, the control action is computed by solving on-line a finite-horizon optimal control problem based on an uncertain model of the plant subject to appropriate Lyapuonv-based stability constraints. The stability constraints are designed to ensure the desired closed-loop stability and performance properties in the presence of faults, and an explicit characterization of the state space region where fault-tolerant stabilization is guaranteed is obtained in terms of the fault size, the choice of the controller design parameters and the size of the plant-model mismatch. To keep sensor-controller communication to a minimum, a forecast-triggered communication strategy is used to determine when communication should be suspended or restored over the network. In this strategy, an update of the model state in the predictive controller using the sensor measurements at a given sampling time is triggered only when the Lyapunov function is forecasted to breach a certain threshold over the next sampling interval. The update-triggering threshold is derived using Lyapunov techniques and is explicitly parameterized in terms of the fault and a suitable fault accommodation parameter. Based on this characterization, fault accommodation strategies that guarantee closed-loop stability while simultaneously optimizing control and communication system resources are devised. Finally, the developed MPC formulation is illustrated using a chemical process example.

Receding horizon control for multiplicative noise stochastic systems with input delay

This paper is concerned with finite horizon stabilization control for a class of discrete time stochastic systems subject to multiplicative noise and input delay. By constructing a new cost function, a complete solution to the problem of finite horizon stabilization is given for the first time based on previous work Zhang et al. (2015). It is shown that the system can be stabilized in the mean square sense with the receding horizon control (RHC) if and only if two new inequalities on the terminal weighting matrices are satisfied. Moreover, the two inequalities can be solved by using iterative algorithm. The explicit stabilizing controller is derived by solving a finite horizon optimal control problem. Simulations demonstrate the effectiveness of the proposed method.