Robust Optimal Control Problem Research Articles

Robust bi-objective optimal control of 1,3-propanediol microbial batch production process

This paper considers optimal control of glycerol producing 1,3-propanediol (1,3-PD) in batch process with uncertain time-delay and kinetic parameters. Based on a dynamic system with uncertain time-delay and kinetic parameters describing the batch process, we propose a robust bi-objective optimal control model, in which 1,3-PD productivity and glycerol consumption rate, and their semi-relative sensitivities with respect to the uncertain parameters are all involved in the vector objective. The control variables in this problem are the initial concentrations of biomass and glycerol and the terminal time of the process. Then, by introducing auxiliary time-delay systems and performing a time-scaling transformation, we transform the robust bi-objective optimal control problem into an equivalent bi-objective optimal control problem in standard form. Furthermore, we convert the equivalent optimal control problem into a sequence of solvable single-objective optimal control problems by using the normalized normal constraint method together with the constraint transcription technique. To solve each of the resulting optimal control problems, we develop a novel gradient-based solution method. Finally, numerical results are provided to verify the effectiveness of the proposed solution approach.

Decentralized robust zero-sum neuro-optimal control for modular robot manipulators in contact with uncertain environments: theory and experimental verification

This paper presents a decentralized robust zero-sum optimal control approach for modular robot manipulators (MRMs) in contact with uncertain environments based on the adaptive dynamic programming (ADP) algorithm. The dynamic model of MRMs is formulated via joint torque feedback technique that is deployed for each joint module to design the model compensation controller. An uncertainty decomposition-based robust control is developed to compensate the model uncertainties, and then, the robust optimal control problem of the MRM system is transformed into a two-player zero-sum optimal control one. According to the ADP algorithm, the Hamilton–Jacobi–Isaacs equation can be solved by establishing action and critic neural networks, thus making the derivation of the approximate optimal control policy feasible. Based on the Lyapunov theory, the closed-loop robotic system is proved to be asymptotic stable under the developed decentralized control method. Finally, experiments are conducted to verify the effectiveness and advantages of the proposed method.

Adaptive model predictive control for linear time varying MIMO systems

A robust, adaptive Model Predictive Control (MPC) approach for asymptotically stable, constrained linear time-varying (LTV) systems with multiple inputs and outputs is proposed. The approach consists of two-steps, carried out on-line with a receding horizon strategy. In the first one, a real-time Set Membership identification algorithm exploits the measured input–output data and the available prior knowledge to build and refine a set of admissible models of the plant (Feasible Parameter Set, FPS). This set is guaranteed to contain also the true system dynamics under the considered working assumptions. In the second step, a robust finite-horizon optimal control problem is formulated and solved. The variation of system dynamics is taken into account by inflating the FPS over the prediction horizon, according to worst-case bounds, assumed a priori, on the parameters’ rate of change. The resulting optimal control sequence guarantees that the outputs of all possible plants inside the FPS satisfy the operational constraints, also considering all possible future parameter changes. The main theoretical properties of the proposed approach are demonstrated and the method is showcased in numerical simulations, highlighting the fundamental improvement over previous approaches not designed for LTV systems.

Robust Optimal Control Scheme for Unknown Constrained-Input Nonlinear Systems via a Plug-n-Play Event-Sampled Critic-Only Algorithm

In this paper, a novel event-sampled robust optimal controller is proposed for a class of continuous-time constrained-input nonlinear systems with unknown dynamics. In order to solve the robust optimal control problem, an online data-driven identifier is established to construct the system dynamics, and an event-sampled critic-only adaptive dynamic programming method is developed to replace the conventional time-driven actor–critic structure. The designed online identification method runs during the solving process and is not applied as a priori part for the solutions, which simplifies the architecture and reduces computational load. The proposed robust optimal control algorithm tunes the parameters of critic-only neural network (NN) by event-triggering condition and runs in a plug-n-play framework without system functions, where fewer transmissions and less computation are required as all the measurements received simultaneously. Based on the novel design, the stability of system and the convergence of critic NN are demonstrated by Lyapunov theory, where the state is asymptotically stable and weight error is guaranteed to be uniformly ultimately bounded. Finally, the applications in a basic nonlinear system and the complex rotational–translational actuator problem demonstrate the effectiveness of the proposed method.

Reinforcement learning for linear continuous-time systems: an incremental learning approach

In this paper, we introduce a novel reinforcement learning &#x0028 RL &#x0029 scheme for linear continuous-time dynamical systems. Different from traditional batch learning algorithms, an incremental learning approach is developed, which provides a more efficient way to tackle the on-line learning problem in real-world applications. We provide concrete convergence and robust analysis on this incremental-learning algorithm. An extension to solving robust optimal control problems is also given. Two simulation examples are also given to illustrate the effectiveness of our theoretical result.

Robust optimal predictive control of heavy haul train under imperfect communication

With the rapid developments in communication technology, the bidirectional wireless communication channel is widely used to exchange information between the train and wayside control center. As an emerging technology, the communication based heavy haul train control (CBHHTC) system is becoming a better alternative for administrative department to provide the greater transport capacity. In order to put the latest automatic control methods into the practice, this paper investigates the robust optimal control problem of heavy haul train under CBHHTC. By formulating the CBHHTC based real-time control procedure as a networked control system (NCS) model, the possible data loss phenomenon in the wireless communications is considered, and its impact on the stability and performances of the controller design is elaborated. On the basis of Lyapunov stability theory and model predictive control (MPC) approach, a set of linear matrix inequalities (LMIs) is given as sufficient conditions to ensure the velocity tracking ability, energy-efficiency and operational safety with a prescribed H∞ disturbance attenuation level under the control constraints. Case studies are conducted to illustrate the effectiveness of the proposed controller.

Robust Optimal Control-based Design of Combined Chemo- and Immunotherapy Delivery Profiles

This paper addresses the problem of drug injection schedules design for cancer treatment, in the presence of model parametric uncertainties. It is commonly known that achieving optimal recovery performances under uncertainties is a complex task. Therefore, we propose to use a recent optimal control approach, based on the moment optimization framework. This method allows to formulate and solve robust optimal control problems by taking into account uncertain parameters and initial states, modeled as probability distributions. We analyse a two dimensional model that describes the interaction dynamics between tumor and immune cells. Furthermore, we derive statistically optimal combined strategies of chemo-and immunotherapy treatments, assuming the knowledge of probability distributions of some uncertain model parameters, namely, the tumor growth rate and the rate of immune cells influx. Numerical simulations are carried out in order to illustrate the effects of parametric uncertainties on dynamics, when using a nominal injection profile. Finally, we compare the recovery performance of nominal and robust schedules.

Robust optimal control of deterministic information epidemics with noisy transition rates

In this paper the robust optimal control of deterministic information epidemics is inspected taking into consideration the noisy transition rates. Distinct from conventional works, the heterogeneous susceptible–infected–susceptible (SIS) model is adopted where both the heterogeneities in the network topology and the individual diversity are considered. In light of the commonly existing noise in the transition processes, we address the robust optimal control problem aiming at maximizing the spreading performance at the finite time instant given a fixed budget. By using the distribution analysis techniques, the inspected problem is transformed to a constrained optimal control problem and solved by the Pontryagin Maximum Principle (PMP). A novel approach combining the forward–backward sweep method and the secant method is proposed to efficiently reduce the computation burden. The performance of the robust optimal control as well as the influence of the parameters is examined by numerical experiments in real social networks.

Robust optimal control during feedback disruption for nonlinear systems controlled by an observer-based output feedback controller

ABSTRACTIn this paper, a robust optimal control problem during feedback disruption is considered for a class of nonlinear systems which have been controlled by an observer-based output feedback controller. It is shown that during feedback disruption, there exists an optimal control input which keeps both system states and observer errors within a specified bound for the longest time. Then, it is shown that such an optimal control input can be practically implemented by using a bang-bang control input in terms of control performance. One numerical and one practical examples are given for clear illustration.

Optimal control with delayed information flow of systems driven by G-Brownian motion

In this paper, we study strongly robust optimal control problems under volatility uncertainty. In the G-framework, we adapt the stochastic maximum principle to find necessary and sufficient conditions for the existence of a strongly robust optimal control.

Robust Optimal Control of Nonlinear Systems With System Disturbance During Feedback Disruption

AbstractIn this paper, a robust optimal control problem of nonlinear systems with system disturbance during feedback disruption is considered. This is an extended work of previous time‐delay optimal control results, by adding external disturbance in the considered system. It is shown that there exists an optimal input signal which keeps the performance error within the specified bound for the longest time. Then, it is shown that such an optimal input signal can be approximated by an implementable bang‐bang input signal in terms of control performance. Two examples are given for illustration.

Towards rigorous robust optimal control via generalized high‐order moment expansion

SummaryThis study is concerned with the rigorous solution of worst‐case robust optimal control problems having bounded time‐varying uncertainty and nonlinear dynamics with affine uncertainty dependence. We propose an algorithm that combines existing uncertainty set‐propagation and moment‐expansion approaches. Specifically, we consider a high‐order moment expansion of the time‐varying uncertainty, and we bound the effect of the infinite‐dimensional remainder term on the system state, in a rigorous manner, using ellipsoidal calculus. We prove that the error introduced by the expansion converges to zero as more moments are added. Moreover, we describe a methodology to construct a conservative, yet more computationally tractable, robust optimization problem, whose solution values are also shown to converge to those of the original robust optimal control problem. We illustrate the applicability and accuracy of this approach with the robust time‐optimal control of a motorized robot arm.

Event-Trigger Based Robust-Optimal Control for Energy Harvesting Transmitter

This paper studies an online algorithm for an energy harvesting transmitter, where the transmission (completion) time is considered as the system performance. Unlike the existing online algorithms which more or less require the knowledge on the future behavior of the energy-harvesting rate, we consider a practical but significantly more challenging scenario where the energy-harvesting rate is assumed to be totally unknown. Our design is formulated as a robust-optimal control problem which aims to optimize the worst-case performance. The transmit power is designed only based on the current battery energy level and the data queue length directly monitored by the transmitter itself. Specifically, we apply an event-trigger approach in which the transmitter continuously monitors the battery energy and triggers an event when a significant change occurs. Once an event is triggered, the transmit power is updated according to the solution to the robust-optimal control problem, which is given in a simple analytic form. We present numerical results on the transmission time achieved by the proposed design and demonstrate its robust-optimality.

Robust optimal control with adjustable uncertainty sets

In this paper, we develop a unified framework for studying constrained robust optimal control problems with adjustable uncertainty sets. In contrast to standard constrained robust optimal control problems with known uncertainty sets, we treat the uncertainty sets in our problems as additional decision variables. In particular, given a finite prediction horizon and a metric for adjusting the uncertainty sets, we address the question of determining the optimal size and shape of the uncertainty sets, while simultaneously ensuring the existence of a control policy that will keep the system within its constraints for all possible disturbance realizations inside the adjusted uncertainty set. Since our problem subsumes the classical constrained robust optimal control design problem, it is computationally intractable in general. Nevertheless, we demonstrate that by restricting the families of admissible uncertainty sets and control policies, the problem can be formulated as a tractable convex optimization problem. We show that our framework captures several families of (convex) uncertainty sets of practical interest, and illustrate our approach on a demand response problem of providing control reserves for a power system.

Data-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control With Uncertain Dynamics

In this paper, the infinite-horizon robust optimal control problem for a class of continuous-time uncertain nonlinear systems is investigated by using data-based adaptive critic designs. The neural network identification scheme is combined with the traditional adaptive critic technique, in order to design the nonlinear robust optimal control under uncertain environment. First, the robust optimal controller of the original uncertain system with a specified cost function is established by adding a feedback gain to the optimal controller of the nominal system. Then, a neural network identifier is employed to reconstruct the unknown dynamics of the nominal system with stability analysis. Hence, the data-based adaptive critic designs can be developed to solve the Hamilton–Jacobi–Bellman equation corresponding to the transformed optimal control problem. The uniform ultimate boundedness of the closed-loop system is also proved by using the Lyapunov approach. Finally, two simulation examples are presented to illustrate the effectiveness of the developed control strategy.

Differential games, partial-state stabilization, and model reference adaptive control

In this paper, we develop a unified framework to find state-feedback control laws that solve two-player zero-sum differential games over the infinite time horizon and guarantee partial-state asymptotic stability of the closed-loop system. Partial-state asymptotic stability is guaranteed by means of a Lyapunov function that is positive definite and decrescent with respect to part of the system state. The existence of a saddle point for the system׳s performance measure is guaranteed by the fact that this Lyapunov function satisfies a partial differential equation that corresponds to a steady-state form of the Hamilton–Jacobi–Isaacs equation. In the second part of this paper, we show how our differential game framework can be applied not only to solve pursuit-evasion and robust optimal control problems, but also to assess the effectiveness of a model reference adaptive control law. Specifically, the model reference adaptive control architecture is designed to guarantee satisfactory trajectory tracking for uncertain nonlinear dynamical systems, whose matched nonlinearities are captured by the regressor vector. By modeling matched and unmatched nonlinearities, which are not captured by the regressor vector, as the pursuer׳s and evader׳s control inputs in a differential game, we provide an explicit characterization of the system׳s uncertainties that do not disrupt the trajectory tracking capabilities of the adaptive controller. Two numerical examples illustrate the applicability of our results.

Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design

This paper presents a novel non-model-based, data-driven adaptive optimal controller design for linear continuous-time systems with completely unknown dynamics. Inspired by the stochastic approximation theory, a continuous-time version of the traditional value iteration (VI) algorithm is presented with rigorous convergence analysis. This VI method is crucial for developing new adaptive dynamic programming methods to solve the adaptive optimal control problem and the stochastic robust optimal control problem for linear continuous-time systems. Fundamentally different from existing results, the a priori knowledge of an initial admissible control policy is no longer required. The efficacy of the proposed methodology is illustrated by two examples and a brief comparative study between VI and earlier policy-iteration methods.

Active set solver for min-max robust control with state and input constraints

This paper proposes an online active set strategy for computing the dynamic programming solution to a min-max robust optimal control problem with quadratic H1 stage cost for linear systems with linear state and input constraints in the presence of bounded disturbances. The solver determines the optimal active constraint set for a given plant state using an iterative procedure which computes the optimal sequence of feedback laws for a candidate active set and updates the active set by performing a line search in state space. The computational complexity of each iteration depends linearly on the length of the prediction horizon. The main contribution of the paper is its treatment of degeneracy caused by linearly dependent state and input constraints and its efficient handling is a crucial step in formulating the active set algorithm. The proposed approach ensures the continuity of optimal control laws along the line-of-search, thus enabling an efficient solution method based on homotopy. Conditions for global optimality are given and the convergence of the active set solver is established using the geometric properties of an associated multi-parametric programming problem. A receding horizon control strategy is proposed, which ensures a specified l2-gain from the disturbance input to the state and control inputs in the presence of linearly dependent constraints

Robust Optimal Control Design Using a Differential Game Approach for Open-Loop Linear Quadratic Descriptor Systems

This paper studies the robust optimal control problem for descriptor systems. We applied differential game theory to solve the disturbance attenuation problem. The robust control problem was converted into a reduced ordinary zero-sum game. Within a linear quadratic setting, we solved the problem for finite and infinite planning horizons.

Robust Control and Hot Spots in Spatiotemporal Economic Systems

We formulate stochastic robust optimal control problems, motivated by applications arising in interconnected economic systems, or spatially extended economies. We study in detail linear quadratic problems and nonlinear problems. We derive optimal robust controls and identify conditions under which concerns about model misspecification at specific site(s) could cause regulation to break down, to be very costly, or to induce pattern formation and spatial clustering. We call sites associated with these phenomena hot spots. We also provide an application of our methods by studying optimal robust control and the potential break down of regulation, due to hot spots, in a model where utility for in situ consumption is distance dependent.