Inverse Optimal Control Research Articles

Inverse Optimal Control for Dynamic Systems with Inequality Constraints

Inverse optimal control (IOC) algorithms can be used to reveal underlying objectives. Existing algorithms commonly estimate the objectives by assuming that the cost function can be represented as a weighted sum of features, and use optimality criteria to estimate the weights. However, the existing literature rarely discusses the recovery of cost functions in the presence of state or control constraints, which often exist due to the limited ranges of actuators and sensors. In this work, an optimisation problem is formulated to find the best values of weights and Lagrange multipliers of constraints to satisfy the optimality conditions, given a segment of an optimal trajectory. The maximum and minimum observed state and control variables are hypothesised as potential box constraints and validated by the associated Lagrange multipliers. In addition, this paper also introduces a method to dynamically choose the window size of the observation, or identify that not enough information was provided for an accurate estimation. The proposed approach is validated using simulated results generated with a two link serial arm. The results show the proposed approach can recover the cost function when box constraints are active, and the Lagrange multiplier value can indicate when and which constraints are present.

An Inverse Optimal Control Approach for Learning and Reproducing Under Uncertainties

This letter presents a novel inverse optimal control (IOC) approach that can account for uncertainties in measurements and system models. The proposed IOC approach aims to recover an objective function including a time-varying term, called variability, from a given demonstration. All uncertainties of the demonstration and the system model can be lumped into the variability such that the optimality condition violation is further reduced. The inferred objective function including the variability has two advantages over the objective function inferred by existing IOC approaches: first, the variability can enhance the capability of describing the given demonstration since it represents how the uncertainties of the system affect the objective function; and second, the proposed IOC approach can reproduce the trajectories such that we can predict the behavior of the system even with system modeling errors. We show that the variability exists and is unique under attainable assumptions. Illustrative numerical examples are presented to demonstrate the proposed method.

Observer-Based Adaptive Inverse Optimal Output Regulation for a Class of Uncertain Nonlinear Systems

This paper addresses the adaptive inverse optimal output regulation problem for a class of uncertain nonlinear systems driven by an exosystem. The unknown parameters, internal disturbances, and unmeasured states are contained in the nonlinear system. Firstly, the output regulation problem is decomposed into a feedforward control design problem which can be solved by the internal model based on the output regulation theory, and an adaptive inverse optimal stabilization problem. Then an auxiliary system is designed, and a new state observer related to the auxiliary system is given. By combining adaptive control technology and inverse optimal control method, a novel adaptive output feedback inverse optimal controller is developed to make the output of the system track the reference signal fast. With this control strategy, all the signals of the closed-loop system are uniformly ultimately bounded (UUB), and the newly well-defined cost functional which is connected with the auxiliary system and the controller can be minimized. Finally, a simulation case is put forward to verify the feasibility of the newly raised controller and the state observer.

Validation of Stochastic Optimal Control Models for Goal-Directed Human Movements on the Example of Human Driving Behavior

Stochastic Optimal Control models represent the state-of-the-art in modeling goal-directed human movements. The linear-quadratic sensorimotor (LQS) model based on signal-dependent noise processes in state and output equation is the current main representative. With our newly introduced Inverse Stochastic Optimal Control algorithm building upon two bi-level optimizations, we can identify its unknown model parameters, namely cost function matrices and scaling parameters of the noise processes, for the first time. In this paper, we use this algorithm to identify the parameters of a deterministic linear-quadratic, a linear-quadratic Gaussian and a LQS model from human measurement data to compare the models’ capability in describing goal-directed human movements. Human steering behavior in a simplified driving task shown to posses similar features as point-ot-point human hand reaching movements serves as our example movement. The results show that the identified LQS model outperforms the others with statistical significance. Particularly, the average human steering behavior is modeled significantly better by the LQS model. This validates the positive impact of signal-dependent noise processes on modeling human average behavior.

Inverse Optimal Control of Korteweg-de Vries-Burgers Equation*

We consider inverse optimal adaptive control for Korteweg-de Vries Burgers (KdVB) equation. First, we prove that the boundary control design in Balogh and Krstic (2000) is inverse optimal for a meaningful functional. Next, we achieve an adaptive boundary control design for KdVB equation with an unknown dissipation coefficient. Moreover, we prove that it is also inverse optimal for a meaningful functional. Two examples are given to show the validity of the proposed design.

Input and state constrained inverse optimal control with application to power networks

We study input and state constrained inverse optimal control problems starting from a stabilizing controller with a control Lyapunov function, where the goal is to make the controller an explicit solution of the resulting constrained optimal control problem. For an appropriate cost design and initial states for which a sublevel set of the Lyapunov function is contained in the state constraint set and the initial input lies on an ellipsoid inside the input constraint set, we show that the stabilizing controller solves the constrained optimal control problem. Compared to the state-of-the-art, we avoid solving nonlinear optimization problems evaluated pointwise, i.e., for every state, or in a repetitive fashion, i.e., at each time step. We apply our theoretical results to study the angular droop control studied in (Jouini et al., 2022) of an inverter-based power network. For this, we accommodate the constraints on the angle and power generation and exemplify our approach through a two-inverter case study.

Disturbance-Observer-Based Adaptive Fuzzy Tracking Control for Unmanned Autonomous Helicopter With Flight Boundary Constraints

In this article, a disturbance-observer-based adaptive fuzzy tracking control scheme is proposed for a medium-scale unmanned helicopter of six degrees of freedom in the presence of system uncertainties, flight boundary constraints, and external disturbances. A flight boundary protection algorithm is proposed to ensure its flight trajectory within the given safety range. A fuzzy logic system is utilized to estimate the system uncertainties and a nonlinear disturbance observer is adopted to handle the unknown compound terms of the external disturbances and the estimation errors resulting from the fuzzy logic system. An inverse optimal control approach is then used to avoid solving the Hamilton–Jacobi–Bellman equation in minimizing a cost function in the attitude loop. It is shown via the Lyapunov method that the desired safe tracking performance of the position loop and attitude loop of the controlled unmanned helicopter can be achieved. Simulations are provided to illustrate the effectiveness of the proposed control scheme.

Adaptive inverse optimal backstepping control strategy for longitudinal vibration of high-speed elevator system based on fuzzy observer

To effectively suppress the longitudinal vibration of the car under the conditions of high-speed operation and emergency braking, and improve the ride comfort of the car system, this paper proposes an adaptive fuzzy inverse optimal output feedback control strategy. Firstly, the dynamic model of the high-speed elevator system is established and the nonlinear dynamic model is approximated by the fuzzy logic system, and the auxiliary system model is established. Fuzzy state observer is designed to estimate the unmeasurable state. Furthermore, an adaptive inverse optimal output feedback controller based on fuzzy observer is designed by using adaptive backstepping technology and inverse optimal control principle. The stability analysis shows that the proposed adaptive fuzzy inverse optimal output feedback control strategy not only ensures the stability of the car attitude of high-speed elevator but also realizes the inverse optimization of the target cost function. Finally, the acceleration time–frequency response analysis of the two typical stages of high-speed elevator uniform running and emergency braking is carried out, and the numerical results are compared with the linear quadratic controller optimized by stepping quantum genetic algorithm (GA-LQR) and controller based on the state-dependent Ricatti equation (SDRE). The analysis verifies the effectiveness of the controller.

Impulsive Neural Control to Schedule Antivirals and Immunomodulators for COVID-19.

New SARS-CoV-2 variants escaping the effect of vaccines are an eminent threat. The use of antivirals to inhibit the viral replication cycle or immunomodulators to regulate host immune responses can help to tackle the viral infection at the host level. To evaluate the potential use of these therapies, we propose the application of an inverse optimal neural controller to a mathematical model that represents SARS-CoV-2 dynamics in the host. Antiviral effects and immune responses are considered as the control actions. The variability between infected hosts can be large, thus, the host infection dynamics are identified based on a Recurrent High-Order Neural Network (RHONN) trained with the Extended Kalman Filter (EKF). The performance of the control strategies is tested by employing a Monte Carlo analysis. Simulation results present different scenarios where potential antivirals and immunomodulators could reduce the viral load.

An analysis method for continuous psychophysics based on Bayesian inverse optimal control

An analysis method for continuous psychophysics based on Bayesian inverse optimal control

Inverse linear-quadratic discrete-time finite-horizon optimal control for indistinguishable homogeneous agents: A convex optimization approach

The inverse linear-quadratic optimal control problem is a system identification problem whose aim is to recover the quadratic cost function and hence the closed-loop system matrices based on observations of optimal trajectories. In this paper, the discrete-time, finite-horizon case is considered, where the agents are also assumed to be homogeneous and indistinguishable. The latter means that the agents all have the same dynamics and objective functions and the observations are in terms of “snap shots” of all agents at different time instants, but what is not known is “which agent moved where” for consecutive observations. This absence of linked optimal trajectories makes the problem challenging. We first show that this problem is globally identifiable. Then, for the case of noiseless observations, we show that the true cost matrix, and hence the closed-loop system matrices, can be recovered as the unique global optimal solution to a convex optimization problem. Next, for the case of noisy observations, we formulate an estimator as the unique global optimal solution to a modified convex optimization problem. Moreover, the statistical consistency of this estimator is shown. Finally, the performance of the proposed method is demonstrated by a number of numerical examples.

Robust Inverse Q-Learning for Continuous-Time Linear Systems in Adversarial Environments.

This article proposes robust inverse Q -learning algorithms for a learner to mimic an expert's states and control inputs in the imitation learning problem. These two agents have different adversarial disturbances. To do the imitation, the learner must reconstruct the unknown expert cost function. The learner only observes the expert's control inputs and uses inverse Q -learning algorithms to reconstruct the unknown expert cost function. The inverse Q -learning algorithms are robust in that they are independent of the system model and allow for the different cost function parameters and disturbances between two agents. We first propose an offline inverse Q -learning algorithm which consists of two iterative learning loops: 1) an inner Q -learning iteration loop and 2) an outer iteration loop based on inverse optimal control. Then, based on this offline algorithm, we further develop an online inverse Q -learning algorithm such that the learner mimics the expert behaviors online with the real-time observation of the expert control inputs. This online computational method has four functional approximators: a critic approximator, two actor approximators, and a state-reward neural network (NN). It simultaneously approximates the parameters of Q -function and the learner state reward online. Convergence and stability proofs are rigorously studied to guarantee the algorithm performance.

Neural Inverse Optimal Control of a Regenerative Braking System for Electric Vehicles

This paper presents the development of a neural inverse optimal control (NIOC) for a regenerative braking system installed in electric vehicles (EVs), which is composed of a main energy system (MES) including a storage system and an auxiliary energy system (AES). This last one is composed of a supercapacitor and a buck–boost converter. The AES aims to recover the energy generated during braking that the MES is incapable of saving and using later during the speed increase. To build up the NIOC, a neural identifier has been trained with an extended Kalman filter (EKF) to estimate the real dynamics of the buck–boost converter. The NIOC is implemented to regulate the voltage and current dynamics in the AES. For testing the drive system of the EV, a DC motor is considered where the speed is controlled using a PID controller to regulate the tracking source in the regenerative braking. Simulation results illustrate the efficiency of the proposed control scheme to track time-varying references of the AES voltage and current dynamics measured at the buck–boost converter and to guarantee the charging and discharging operation modes of the supercapacitor. In addition, it is demonstrated that the proposed control scheme enhances the EV storage system’s efficacy and performance when the regenerative braking system is working. Furthermore, the mean squared error is calculated to prove and compare the proposed control scheme with the mean squared error for a PID controller.

Voltage Regulation in Second-Order Dc-Dc Converters Via the Inverse Optimal Control Design with Proportional-Integral Action

This article addresses the problem regarding power regulation in classical DC-DC second-order converters by means of a nonlinear control technique based on inverse optimal control theory. There are few papers that describe inverse optimal control for DC-DC converters in the literature. Therefore, this study constitutes a contribution to the state of the art on nonlinear control techniques for DC-DC converters. In this vein, the main objective of this research was to implement inverse optimal control theory with integral action to the typical DC-DC conversion topologies for power regulation, regardless of the load variations and the application. The converter topologies analyzed were: (i) Buck; (ii) Boost; (iii) Buck-Boost; and (iv) Non-Inverting Buck-Boost. A dynamical model was proposed as a function of the state variable error, which helped to demonstrate that the inverse optimal control law with proportional-integral action implemented in the different converters ensures stability in each closed-loop operation via Lyapunov’s theorem. Numerical validations were carried out by means of simulations in the PSIM software, comparing the established control law, the passivity-based PI control law, and an open-loop control. As a conclusion, it was possible to determine that the proposed model is easier to implement and has a better dynamical behavior than the PI-PBC, ensuring asymptotic stability from the closed-loop control design.

Inverse Optimal Control for Maximizing Harvested Energy From Dynamical Systems

Abstract This paper formulates and solves a new problem of inverse optimal control to maximize harvested energy from dynamical systems induced by external excitations. An inverse optimal control law is designed to minimize a cost function penalizing negativeness of the harvested energy and positiveness of the control and system (partial) states, where it does not require to solve a Hamilton–Jacobi–Belman (HJB) equation or a Hamilton–Jacobi–Isaacs (HJI) equation. An application to a point absorber wave energy converter (PAWEC) is included to illustrate the effectiveness of the proposed theory.

Efficient Broadband Frequency Conversion via Shortcuts to Adiabaticity

AbstractThe method of adiabatic frequency conversion, in analogy with a two‐level atomic system, has been put forward recently and verified experimentally to achieve robust frequency mixing processes such as sum and difference frequency generation. Here a comparative study of efficient frequency mixing using various techniques of shortcuts to adiabaticity such as counter‐diabatic driving and invariant‐based inverse engineering is presented. It is shown here that it is possible to perform sum frequency generation by properly designing the poling structure of a periodically poled crystal and the coupling between the input lights and the crystal. The required crystal length for frequency conversion significantly decreases beyond the adiabatic limit. This approach significantly improves the robustness of the process against the variation in temperature as well as the signal frequency. By introducing a single parameter control technique with constant coupling and combining with the inverse engineering, perturbation theory, and optimal control, it is shown that the phase mismatch can be further optimized with respect to the fluctuations of input wavelength and crystal temperature that results into a novel experimentally realizable mixing scheme.

Observer-Based Finite-Time Inverse Optimal Output Regulation for Uncertain Nonlinear Systems

This paper deals with the finite-time inverse optimal output regulation problem for a class of uncertain nonlinear systems that are subjected to an exosystem. The nonlinear systems not only contain external disturbances in the exosystem but also take the unknown nonlinear functions and unmeasured states into account. The output regulation problem is first converted into a stabilization problem through the internal model. Then, fuzzy logic systems (FLSs) are employed to approximate the unknown nonlinear functions. An auxiliary system is constructed and, based on the auxiliary system, a fuzzy state observer is designed to estimate the unmeasured states. Furthermore, a novel adaptive fuzzy finite-time inverse optimal output feedback controller and adaptive law are designed by combining backstepping technology, adaptive control technology, finite-time stability theory, and the inverse optimal control method. The control algorithm ensures that all the signals of the closed-loop system are semiglobally practical finite-time stable (SGPFS), so the newly well-defined cost function that is connected with the auxiliary system can be minimized. Finally, the validity of the approach is confirmed by virtue of the simulation results.

Putting perception into action with inverse optimal control for continuous psychophysics.

Psychophysical methods are a cornerstone of psychology, cognitive science, and neuroscience where they have been used to quantify behavior and its neural correlates for a vast range of mental phenomena. Their power derives from the combination of controlled experiments and rigorous analysis through signal detection theory. Unfortunately, they require many tedious trials and preferably highly trained participants. A recently developed approach, continuous psychophysics, promises to transform the field by abandoning the rigid trial structure involving binary responses and replacing it with continuous behavioral adjustments to dynamic stimuli. However, what has precluded wide adoption of this approach is that current analysis methods do not account for the additional variability introduced by the motor component of the task and therefore recover perceptual thresholds that are larger compared to equivalent traditional psychophysical experiments. Here, we introduce a computational analysis framework for continuous psychophysics based on Bayesian inverse optimal control. We show via simulations and previously published data that this not only recovers the perceptual thresholds but additionally estimates subjects' action variability, internal behavioral costs, and subjective beliefs about the experimental stimulus dynamics. Taken together, we provide further evidence for the importance of including acting uncertainties, subjective beliefs, and, crucially, the intrinsic costs of behavior, even in experiments seemingly only investigating perception.

Online Learning Human Behavior for a Class of Human-in-the-Loop Systems via Adaptive Inverse Optimal Control

To enhance the machines’ intelligence, it is important for them to learn how humans perform tasks. In this article, the issue of online adaptive learning human behavior is addressed for a class of human-in-the-loop (HiTL) systems using the state measurement only. The hypothesis underlying our study is that human behavior can be described by a linear quadratic optimal control model with an unknown weighting matrix for the quadratic cost function. In this model, the weighting matrix depicts the human tradeoff of various objectives. Our aim is thus to only use the system state measurement for learning the weighting matrix under the condition that human feedback gain matrix is unknown. A novel adaptive inverse optimal control approach to online learning human behavior is proposed for the HiTL system, which integrates adaptive estimation and linear matrix inequality (LMI) optimization techniques. Our approach consists of two steps: First, an adaptive law is developed to learn the human feedback gain matrix online using the system state measurement only, and second, the weighting matrix of human cost function is retrieved by solving an LMI optimization problem with the learned feedback gain matrix. Finally, simulation and experiment results on a steering assist system of intelligent vehicles are presented to illustrate the effectiveness of the proposed method.

H ∞ inverse optimal attitude tracking on the special orthogonal group SO(3)

The problem of attitude tracking in the presence of disturbances is addressed by combining inverse optimality and disturbance attenuation. Conditions are provided which ensure that an inverse optimal nonlinear attitude control problem is solved and a meaningful cost function is minimised. The approach results in a PD-type attitude control law on the Special Orthogonal Group which guarantees that, for almost all initial conditions, the energy gain from an exogenous disturbance to a specified error signal respects a given upper bound. For numerical simulations, a satellite attitude tracking problem from literature is considered. The controller gains are tuned using a decoupled linearised single-axis model and the structured control synthesis method. Results indicate that the proposed controller has good tracking and disturbance attenuation capability. In particular, for the problem under consideration, the inverse optimal controller is seen to have better transient performance than its continuous-time quaternion counterpart. Moreover, for initial errors up to radians, it has comparable performance to the shortest-path quaternion PD controller.