Finite-horizon Optimal Control Problem Research Articles

Algebraic Approach to Nonlinear Optimal Control Problems with Terminal Constraints: Sufficient Conditions for Existence of Algebraic Solutions

This paper studies nonlinear finite-horizon optimal control problems with terminal constraints, where all nonlinear functions are rational or algebraic functions. We first extend a recursive elimin...

Designing a closed-loop guidance system to increase the accuracy of satellite-carrier boosters' landing point

A novel closed loop guidance method is provided in this paper to increase the accuracy of satellite-carrier boosters' landing point. The proposed method can be used in the first stage of flight vehicles that fly in atmosphere. In this case, solid motor propelled boosters would be able to more accurately land on the desired point. Moreover, in sub-optimal technique, the closed-loop guidance system would produce commands after booster separation, which guide second stage of satellite-carrier from initial conditions to the desired condition of orbit injection. In this method, sub-optimal integrated solution of control and guidance in closed-loop is developed. This sub-optimal technique named as Model Predictive Static Programming (MPSP) that is based on nonlinear optimal control theory and derived from combined philosophies of Model Predictive Control and Approximate Dynamic Programming; solves a class of finite horizon optimal control problems with terminal constraints. Furthermore because sensitivity matrices that are necessary for obtaining this solution can be computed recursively, this technique is computationally efficient and is appropriate for online implementation. In this paper, the dynamic equations of system are modeled in the presence of aerodynamic loading and the servo-mechanism dynamic. Moreover, by considering integrated guidance and control loops, a solution of the guidance and control system is proposed by three-degree of freedom spherical earth simulation model in atmosphere. Result show that proposed closed-loop guidance not only is able to remove aerodynamic and thrust modeling errors in first stage by flight data update that caused to more accurate of boosters' landing point, but also would still be able to guide second stage of satellite-carrier to the desired condition of placement in the orbit.

Robust adaptive gain higher order sliding mode observer based control-constrained nonlinear model predictive control for spacecraft formation flying

This work deals with the development of a decentralized optimal control algorithm, along with a robust observer, for the relative motion control of spacecraft in leader-follower based formation. An adaptive gain higher order sliding mode observer has been proposed to estimate the velocity as well as unmeasured disturbances from the noisy position measurements. A differentiator structure containing the Lipschitz constant and Lebesgue measurable control input, is utilized for obtaining the estimates. Adaptive tuning algorithms are derived based on Lyapunov stability theory, for updating the observer gains, which will give enough flexibility in the choice of initial estimates. Moreover, it may help to cope with unexpected state jerks. The trajectory tracking problem is formulated as a finite horizon optimal control problem, which is solved online. The control constraints are incorporated by using a nonquadratic performance functional. An adaptive update law has been derived for tuning the step size in the optimization algorithm, which may help to improve the convergence speed. Moreover, it is an attractive alternative to the heuristic choice of step size for diverse operating conditions. The disturbance as well as state estimates from the higher order sliding mode observer are utilized by the plant output prediction model, which will improve the overall performance of the controller. The nonlinear dynamics defined in leader fixed Euler-Hill frame has been considered for the present work and the reference trajectories are generated using Hill-Clohessy-Wiltshire equations of unperturbed motion. The simulation results based on rigorous perturbation analysis are presented to confirm the robustness of the proposed approach.

Formation Control of Multiple Unmanned Aerial Vehicles by Event-Triggered Distributed Model Predictive Control

This paper proposes an event-triggered model predictive control (MPC) scheme for the formation control of multiple unmanned aerial vehicles (UAVs). A distributed MPC framework is designed in which each UAV only shares the information with its neighbors, and the obtained local finite-horizon optimal control problem (FHOCP) can be solved by a swarm intelligent optimization algorithm. An event-triggered mechanism is proposed to reduce the computational burden for the distributed MPC scheme, which takes into consideration the predictive state errors as well as the convergence of cost function. Furthermore, a safe-distance-based strategy for no-fly zone avoidance is developed and integrated into the local cost function for each FHOCP. Numerical simulations show that the proposed event-triggered distributed MPC is more computationally efficient to achieve formation control of multiple UAVs in comparison with the traditional distributed MPC method.

Data-based predictive control via direct weight optimization

Abstract In this paper we propose a novel data-based predictive control scheme in which the prediction model is obtained from a linear combination of past system trajectories. The proposed controller optimizes the weights of this linear combination taking into account simultaneously performance and the variance of the estimation error. For unconstrained systems, dynamic programming is used to obtain an explicit linear solution of a finite or infinite horizon optimal control problem. When constraints are taken into account, the controller needs to solve online a quadratic optimization problem to obtain the optimal weights, possibly considering also local information to improve the performance and estimation. A simulation example of the application of the proposed controller to a quadruple-tank system is provided.

Deep reinforcement learning based finite-horizon optimal tracking control for nonlinear system

Abstract Reinforcement learning (RL) can be used to obtain an approximate numerical solution to the Hamilton-Jacobi-Bellman (HJB) equation. Recent advances in machine learning community enable the use of deep neural networks (DNNs) to approximate high-dimensional nonlinear functions as those that occur in RL, accurately without any domain knowledge. In the standard RL setting, both system and cost structures are unknown, and the amount of data needed to obtain an accurate approximation can be impractically large. Meanwhile, when the structures are known, they can be used to solve the HJB equation efficiently. Herein, the model-based globalized dual heuristic programming (GDHP) is proposed, in which the HJB equation is separated into value, costate, and policy functions. A particular class of interest in this research is finite horizon optimal tracking control (FHOC) problem. Additional issues that arise, such as time-varying functions, terminal constraints, and delta-input formulation, are addressed in the context of FHOC. The DNN structure and training algorithm suitable for FHOC are presented. A benchmark continuous reactor example is provided to illustrate the proposed approach.

Impact of Quantisation on Consistency of DMPC in Street Traffic with Dynamic Priority Rules

AbstractDistributed systems typically require communication to steer the overall system to a reference or target state and keep it there. We consider an intersection scenario as a spacial set with vehicles as multi‐agents with individual initial and target conditions. Utilising a Distributed Model Predictive Control scheme (DMPC), in every time step each vehicle solves an finite horizon optimal control problem. The resulting optimal state trajectory is projected onto a quantisation of the spatial set and the quantised trajectories are broadcasted to the other vehicles to ensure collision avoidance. The quantisation is mainly motivated to reduce the necessary communication effort among the agents. Here, we introduce prediction coherence as difference of two predictions in two successive time steps. We numerically explore the idea to utilise prediction coherence to derive a lower bound for the communication requirements among the vehicles. (© 2017 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Sufficient optimality conditions for distributed, non-predictive ramp metering in the monotonic cell transmission model

We consider the ramp metering problem for a freeway stretch modeled by the Cell Transmission Model. Assuming perfect model knowledge and perfect traffic demand prediction, the ramp metering problem can be cast as a finite horizon optimal control problem with the objective of minimizing the Total Time Spent, i.e., the sum of the travel times of all drivers. For this reason, the application of Model Predictive Control (MPC) to the ramp metering problem has been proposed. However, practical tests on freeways show that MPC may not outperform simple, distributed feedback policies. Until now, a theoretical justification for this empirical observation was lacking. This work compares the performance of distributed, non-predictive policies to the optimal solution in an idealised setting, specifically, for monotonic traffic dynamics and assuming perfect model knowledge. To do so, we suggest a distributed, non-predictive policy and derive sufficient optimality conditions for the minimization of the Total Time Spent via monotonicity arguments. In a case study based on real-world traffic data, we demonstrate that these optimality conditions are only rarely violated. Moreover, we observe that the suboptimality resulting from such infrequent violations appears to be negligible. We complement this analysis with simulations in non-ideal settings, in particular allowing for model mismatch, and argue that Alinea, a successful, distributed ramp metering policy, comes close to the ideal controller both in terms of control behavior and in performance.

Thermal Management in Plug-In Hybrid Electric Vehicles: A Real-Time Nonlinear Model Predictive Control Implementation

A real-time nonlinear model predictive control (NMPC) for the thermal management (TM) of the electrical components cooling circuit in a Plug-In Hybrid Electric Vehicle (PHEV) is presented. The electrical components are highly temperature sensitive and, therefore, working out of the ranges recommended by the manufacturer can lead to their premature aging or even failure. Consequently, the goals for an accurate and efficient TM are to keep the main component, the Li-ion battery, within optimal working temperatures, and to consume the minimum possible electrical energy through the cooling circuit actuators. This multi-objective requirement is formulated as a finite-horizon optimal control problem (OCP) that includes a multi-objective cost function, several constraints, and a prediction model especially suitable for optimization. The associated NMPC is performed on real time by the optimization package MUSCOD-II and is validated in three different repeatable test-drives driven with a PHEV. Starting from identical conditions, each cycle is driven once being the cooling circuit controlled with NMPC and once with a conventional approach based on a finite-state machine. Compared to the conventional strategy, the NMPC proposed here results in a more accurate and healthier temperature performance, and at the same time, leads to reductions in the electrical consumption up to 8%.

Robust LQR for time-inhomogeneous Markov jump switched linear systems

Markov jump switched linear systems (MJSLSs) are switched linear systems, where a switching signal is governed by a Markov decision process. We consider a polytopic time-inhomogeneous setting, where the transition probabilities (between the operational modes of the system) associated to each discrete action are varying over time, with variations that are arbitrary within a polytopic set. We present and solve for this class of systems the finite horizon optimal control problem.

Real-Time Motion Planning for Aerial Videography With Dynamic Obstacle Avoidance and Viewpoint Optimization

We propose a method for real-time trajectory generation with applications in aerial videography. Taking framing objectives, such as position of targets in the image plane, as input, our method solves for robot trajectories and gimbal controls automatically and adapts plans in real time due to changes in the environment. We contribute a real-time receding horizon planner that autonomously records scenes with moving targets, while optimizing for visibility under occlusion and ensuring collision-free trajectories. A modular cost function, based on the reprojection error of targets, is proposed that allows for flexibility and artistic freedom and is well behaved under numerical optimization. We formulate the minimization problem under constraints as a finite horizon optimal control problem that fulfills aesthetic objectives, adheres to nonlinear model constraints of the filming robot and collision constraints with static and dynamic obstacles and can be solved in real time. We demonstrate the robustness and efficiency of the method with a number of challenging shots filmed in dynamic environments including those with moving obstacles and shots with multiple targets to be filmed simultaneously.

Forecast-Triggered Fault-Tolerant Model Predictive Control of Nonlinear Processes

In this work, we consider the problem of fault-tolerant stabilization of constrained nonlinear processes controlled over resource-constrained sensor-controller communication networks and subject to control actuator faults. A methodology for the design of resource-aware Lyapuonv-based model predictive control (MPC) systems that achieve the fault-tolerant stabilization objective with reduced sensor-controller communication is presented. In this approach, the control action is computed by solving on-line a finite-horizon optimal control problem based on an uncertain model of the plant subject to appropriate Lyapuonv-based stability constraints. The stability constraints are designed to ensure the desired closed-loop stability and performance properties in the presence of faults, and an explicit characterization of the state space region where fault-tolerant stabilization is guaranteed is obtained in terms of the fault size, the choice of the controller design parameters and the size of the plant-model mismatch. To keep sensor-controller communication to a minimum, a forecast-triggered communication strategy is used to determine when communication should be suspended or restored over the network. In this strategy, an update of the model state in the predictive controller using the sensor measurements at a given sampling time is triggered only when the Lyapunov function is forecasted to breach a certain threshold over the next sampling interval. The update-triggering threshold is derived using Lyapunov techniques and is explicitly parameterized in terms of the fault and a suitable fault accommodation parameter. Based on this characterization, fault accommodation strategies that guarantee closed-loop stability while simultaneously optimizing control and communication system resources are devised. Finally, the developed MPC formulation is illustrated using a chemical process example.

Receding horizon control for multiplicative noise stochastic systems with input delay

This paper is concerned with finite horizon stabilization control for a class of discrete time stochastic systems subject to multiplicative noise and input delay. By constructing a new cost function, a complete solution to the problem of finite horizon stabilization is given for the first time based on previous work Zhang et al. (2015). It is shown that the system can be stabilized in the mean square sense with the receding horizon control (RHC) if and only if two new inequalities on the terminal weighting matrices are satisfied. Moreover, the two inequalities can be solved by using iterative algorithm. The explicit stabilizing controller is derived by solving a finite horizon optimal control problem. Simulations demonstrate the effectiveness of the proposed method.

Receding horizon control for discrete‐time multiple input delay systems

SummaryThis paper is concerned with stabilization problem for discrete‐time systems with multiple input delay. By defining a new cost function, the stabilization condition is derived based on monotonically nonincreasing of the optimal cost. It is shown that if the 2 weighting matrices in the cost function satisfy the given linear matrix inequality, receding horizon controller can stabilize the input delay systems. Moreover, the explicit stabilization controller with the new cost function is obtained by solving a finite horizon optimal control problem. A numerical example illustrates the effectiveness of the proposed method.

On the Relation Between Turnpike Properties for Finite and Infinite Horizon Optimal Control Problems

We show that, under appropriate regularity conditions, a finite horizon optimal control problem exhibits the turnpike property, if and only if its infinite horizon counterpart does. We prove the result for both undiscounted and discounted problems and also provide a version which incorporates quantitative information about the convergence rates.

Finite-Horizon $H_{\infty }$ Tracking Control for Unknown Nonlinear Systems With Saturating Actuators.

In this paper, a neural network (NN)-based online model-free integral reinforcement learning algorithm is developed to solve the finite-horizon optimal tracking control problem for completely unknown nonlinear continuous-time systems with disturbance and saturating actuators (constrained control input). An augmented system is constructed with the tracking error system and the command generator system. A time-varying Hamilton-Jacobi-Isaacs (HJI) equation is formulated for the augmented problem, which is extremely difficult or impossible to solve due to its time-dependent property and nonlinearity. Then, an actor-critic-disturbance NN structure-based scheme is proposed to learn the time-varying solution to the HJI equation in real time without using the knowledge of system dynamics. Since the solution to the HJI equation is time-dependent, the form of NNs representation with constant weights and time-dependent activation functions is considered. Furthermore, an extra error is incorporated in order to satisfy the terminal constraints in the weight update law. Convergence and stability proofs are given based on the Lyapunov theory for nonautonomous systems. Two simulation examples are provided to demonstrate the effectiveness of the designed algorithm.

Neuro‐observer based online finite‐horizon optimal control for uncertain non‐linear continuous‐time systems

In this study, within framework of adaptive dynamic programming (ADP), a neuro-observer based online optimal control solution is developed for the finite-horizon optimal control problem of uncertain non-linear continuous-time systems. First, a neuro-observer is designed to estimate system states from the uncertain system without knowledge of system drift dynamics. Then based on the observed states, an online finite-horizon optimal control scheme is proposed by a single critic network to approximate the solution of the time-varying Hamilton–Jacobi–Bellman (HJB) equation and obtain the optimal control. In the design, a novel observer–critic architecture is presented for implementing the control scheme by using two neural networks (NNs): an observer NN is used to learn the uncertain system dynamics and a critic NN is employed not only to approximate the solution of time-varying HJB equation, but also to ensure the terminal constraint is satisfied. Besides, an additional adjusting term is introduced by Lyapunov theory in critic NN weight tuning algorithm, which relaxes the requirement for an initial stabilising control. Moreover, uniform ultimate bounded stability of the closed-loop system is guaranteed via Lyapunov's direct method. Finally, simulation results are shown to demonstrate the effectiveness of the proposed approach.

Support Vector Machine Informed Explicit Nonlinear Model Predictive Control Using Low-Discrepancy Sequences

In this paper, an explicit nonlinear model predictive controller (ENMPC) for the stabilization of nonlinear systems is investigated. The proposed ENMPC is constructed using tensored polynomial basis functions and samples drawn from low-discrepancy sequences. Solutions of a finite-horizon optimal control problem at the sampled nodes are used 1) to learn an inner and outer approximation of the feasible region of the ENMPC using support vector machines, and 2) to construct the ENMPC control surface on the computed feasible region using regression or sparse-grid interpolation, depending on the shape of the feasible region. The attractiveness of the proposed control scheme lies in its tractability to higher-dimensional systems with feasibility and stability guarantees, significantly small online computation times, and ease of implementation.

Improved MPSP Method-based Cooperative Re-entry Guidance for Hypersonic Gliding Vehicles

A computationally sufficient technique is used to solve the 3-D cooperative re-entry guidance problem for hypersonic gliding vehicles. Due to the poor surrounding adaptive ability of the traditional cooperative guidance methods, a novel methodology, named as model predictive static programming (MPSP), is used to solve a class of finite-horizon optimal control problems with hard terminal constraints. The main feature of this guidance law is that it is capable of hitting the target with high accuracy for each one of the cooperative vehicles at the same time. In addition, it accurately satisfies variable constraints. Performance of the proposed MPSP-based guidance is demonstrated in 3-D nonlinear dynamics scenario. The numerical simulation results show that the proposed cooperative re-entry guidance methodology has the advantage of computational efficiency and better robustness against the perturbations.

Robust economic Model Predictive Control using stochastic information

In this paper, we develop a new tube-based robust economic MPC scheme for linear time-invariant systems subject to bounded disturbances with given distributions. By using the error distribution in the predictions of the finite horizon optimal control problem, we can incorporate stochastic information in order to improve the expected performance while being able to guarantee strict feasibility. For this new framework, we can provide bounds on the asymptotic average performance of the closed-loop system. Moreover, a constructive approach is presented in order to find an appropriate terminal cost leading to a slight degradation of the bound on the guaranteed average performance.