Inverse Optimal Control Approach Research Articles

Adaptive Inverse Nonlinear Optimal Control Based on Finite-Time Concurrent Learning and Semidefinite Programming.

This article investigates the problem of inverse optimal control (IOC) for a class of nonlinear affine systems. An adaptive IOC approach is proposed to recover the cost functional using only the system state data, which integrates the finite-time concurrent learning (FTCL) technique and the semidefinite programming (SDP) technique. First, an identifier neural network (NN) is employed to approximate the unknown nonlinear control policy, and an FTCL-based update law is proposed to estimate the weights of the identifier NN online, which removes the traditional persistent excitation (PE) condition. Moreover, the finite-time convergence as well as the uniformly ultimately boundness (UUB) of estimation error of the identifier NN weights are analysed according to whether or not there exists the identifier NN approximation error. Then, with the help of a value NN for approximating the value function, an SDP problem with a quadratic objective function can be set up for determining the weighting matrices of the cost functional. Finally, simulation results are presented to validate the proposed method.

Online Inverse Optimal Control for Time-Varying Cost Weights.

Inverse optimal control is a method for recovering the cost function used in an optimal control problem in expert demonstrations. Most studies on inverse optimal control have focused on building the unknown cost function through the linear combination of given features with unknown cost weights, which are generally considered to be constant. However, in many real-world applications, the cost weights may vary over time. In this study, we propose an adaptive online inverse optimal control approach based on a neural-network approximation to address the challenge of recovering time-varying cost weights. We conduct a well-posedness analysis of the problem and suggest a condition for the adaptive goal, under which the weights of the neural network generated to achieve this adaptive goal are unique to the corresponding inverse optimal control problem. Furthermore, we propose an updating law for the weights of the neural network to ensure the stability of the convergence of the solutions. Finally, simulation results for an example linear system are presented to demonstrate the effectiveness of the proposed strategy. The proposed method is applicable to a wide range of problems requiring real-time inverse optimal control calculations.

Inverse optimal control for autonomous carrier landing with disturbances

This work addresses the optimal control of carrier landing affected by deck motion and airwake disturbances. Starting from aircraft dynamics written under an affine form, a novel inverse optimal control approach is designed and then used as the main control technique to provide fast and accurate tracking of the aircraft reference trajectory, as well as to improve the convergence performances and the success rate of the automatic carrier landing. A robust automatic control landing system is proposed by designing a novel inverse optimal control algorithm based on disturbance observers (for the estimation and compensation of airwake), a deck motion compensation block, and fixed-time command filters for computing the reference signals and reducing the tracking errors. The proposed optimal scheme includes a guidance controller, an attitude controller (involving the control of the heading angle, attitude angles, and angular rates), and an approach power compensation system. The proposed architecture is proved to be globally stable by using the Lyapunov theory. Finally, the comparative simulation results prove both the reliability of the control scheme and the superiority of the novel inverse optimal control technique over other nonlinear control approaches employed in the design of automatic control landing systems.

Composite adaptive online inverse optimal control approach to human behavior learning

There have been broad ranges of human-in-the-loop (HiTL) control systems, like fly-by-wire aircraft, share driving, and energy management. To develop advanced HiTL control systems with augmented intelligence, it is desirable that the machine is empowered to understand how a skilled human operator determines the control actions. It is common to model the human operator as an optimal controller with an unknown weighting matrix which depicts the tradeoff between multiple control objectives. Our aim is thus to learn the weighting matrix of the human objective function using only the system state measurement. Accordingly, for the HiTL system, a composite adaptive inverse optimal control (IOC) approach, which integrates concurrent learning (CL) based composite adaptive estimation and linear matrix inequality (LMI) optimization techniques, is proposed to learn human behavior online. The proposed composite adaptive IOC method consists of two steps: first, a composite adaptive update law is proposed to online estimate the human feedback gain matrix, which removes the persistent excitation condition required by traditional adaptive estimation methods, and then, an LMI optimization problem is solved to retrieve the weighting matrix of the human objective function with the learned feedback gain. The applicability and effectiveness of the proposed method are demonstrated with a steering control simulation.

Prescribed performance-based robust inverse optimal control for spacecraft proximity operations with safety concern

This paper explores and analyzes the robust proximity control problem for spacecraft in the presence of external disturbance, model uncertainty, and safety concern. To fulfill the safety proximity requirement, a novel two-stage time-depended performance function is constructed to guarantee the transient and steady-state proximity performance. Then, by assimilating the advantages of the prescribed performance control methodology and inverse optimal control approach, a neuroadaptive prescribed performance-based inverse optimal control scheme is proposed, which ensures all system signals comply with the preassigned performance function. The developed controller shows optimality concerning convergence performance and energy consumption, whilst satisfying the well-designed safety corridor. Finally, simulation scenarios are provided to demonstrate the effectiveness of the presented control scheme.

An Inverse Optimal Control Approach for Learning and Reproducing Under Uncertainties

This letter presents a novel inverse optimal control (IOC) approach that can account for uncertainties in measurements and system models. The proposed IOC approach aims to recover an objective function including a time-varying term, called variability, from a given demonstration. All uncertainties of the demonstration and the system model can be lumped into the variability such that the optimality condition violation is further reduced. The inferred objective function including the variability has two advantages over the objective function inferred by existing IOC approaches: first, the variability can enhance the capability of describing the given demonstration since it represents how the uncertainties of the system affect the objective function; and second, the proposed IOC approach can reproduce the trajectories such that we can predict the behavior of the system even with system modeling errors. We show that the variability exists and is unique under attainable assumptions. Illustrative numerical examples are presented to demonstrate the proposed method.

Disturbance-Observer-Based Adaptive Fuzzy Tracking Control for Unmanned Autonomous Helicopter With Flight Boundary Constraints

In this article, a disturbance-observer-based adaptive fuzzy tracking control scheme is proposed for a medium-scale unmanned helicopter of six degrees of freedom in the presence of system uncertainties, flight boundary constraints, and external disturbances. A flight boundary protection algorithm is proposed to ensure its flight trajectory within the given safety range. A fuzzy logic system is utilized to estimate the system uncertainties and a nonlinear disturbance observer is adopted to handle the unknown compound terms of the external disturbances and the estimation errors resulting from the fuzzy logic system. An inverse optimal control approach is then used to avoid solving the Hamilton–Jacobi–Bellman equation in minimizing a cost function in the attitude loop. It is shown via the Lyapunov method that the desired safe tracking performance of the position loop and attitude loop of the controlled unmanned helicopter can be achieved. Simulations are provided to illustrate the effectiveness of the proposed control scheme.

Online Learning Human Behavior for a Class of Human-in-the-Loop Systems via Adaptive Inverse Optimal Control

To enhance the machines’ intelligence, it is important for them to learn how humans perform tasks. In this article, the issue of online adaptive learning human behavior is addressed for a class of human-in-the-loop (HiTL) systems using the state measurement only. The hypothesis underlying our study is that human behavior can be described by a linear quadratic optimal control model with an unknown weighting matrix for the quadratic cost function. In this model, the weighting matrix depicts the human tradeoff of various objectives. Our aim is thus to only use the system state measurement for learning the weighting matrix under the condition that human feedback gain matrix is unknown. A novel adaptive inverse optimal control approach to online learning human behavior is proposed for the HiTL system, which integrates adaptive estimation and linear matrix inequality (LMI) optimization techniques. Our approach consists of two steps: First, an adaptive law is developed to learn the human feedback gain matrix online using the system state measurement only, and second, the weighting matrix of human cost function is retrieved by solving an LMI optimization problem with the learned feedback gain matrix. Finally, simulation and experiment results on a steering assist system of intelligent vehicles are presented to illustrate the effectiveness of the proposed method.

Online Model Predictive Motion Cueing With Real-Time Driver Prediction

In this article a motion cueing algorithm (MCA) based on model predictive control (MPC) for a hexapod-based dynamic driving simulator is derived. The design objective of the MCA is to reproduce the real world accelerations and angular velocities for a test person in the simulator while respecting its actuator limitations. This results in an underlying nonlinear, state-constrained optimal control problem. In order to exploit the full predictive potential of the described algorithm, a method to predict the future driver behavior and thus the future desired values for the MPC is derived by modeling the driver as an optimal controller. The OCP weights that mainly influence the predicted driving actions are learned from demonstration using an inverse optimal control (IOC) approach. Furthermore, the direct incorporation of human perception models into the motion planning process is considered. An online driver-in-the-loop experiment with the Daimler driving simulator shows the high potential of the derived MPC scheme compared to the commonly used filter-based approach as well as the efficiency of the underlying optimization method.

Attitude Tracking Control With Constraints for Rigid Spacecraft Based on Control Barrier Lyapunov Functions

This article investigates the attitude tracking control problem with constraints for rigid spacecraft subject to parameter uncertainty and external disturbance. Based on the concept of the safety control problem, the investigated problem is formulated as the stabilization of the corresponding attitude tracking error system of the rigid spacecraft. By using the inverse optimal control approach, a control Lyapunov-barrier function (CLBF)-based controller is proposed for the stabilization of the spacecraft attitude tracking error system via combining a designed control barrier function and an existing control Lyapunov function. For ensuring the robustness of the proposed CLBF-based controller in the presence of parameter uncertainty and external disturbance, a sliding-mode control part is added in the presented controller. It is proven through the Lyapunov stability theory that the closed-loop system under the proposed controller is practical finite-time stable. Simulation results are given to show the effectiveness of the developed CLBF-based attitude tracking controller.

Distributed Inverse Optimal Control for Discrete-Time Nonlinear Multi-Agent Systems

This letter describes a robust distributed inverse optimal control framework for a multi-agent discrete-time nonlinear system, where the dynamics of each agent is directly affected by terms that depend on the state and input of the neighborhood agents and other disturbance signals. The individual local cost is formulated and a control solution for each agent is derived considering an inverse optimal control approach. To address the interaction between the agents, a coordination method based on a non-cooperative game is proposed. Using Lyapunov and Input-to-State Stability (ISS) arguments, we derive conditions under which the proposed game converges to a fixed point and the overall multi-agent system is ISS with respect to the disturbance signals. Simulation results for a coupled pendula system are presented.

Modified three‐dimensional true proportional navigation and its inverse optimal form

AbstractA novel three‐dimensional modified true proportional navigation (MTPN) guidance and its inverse optimal form are proposed for the interception of the nonmaneuvering and maneuvering targets, and the nonlinear dynamic characteristics of pursuit situations and target maneuvers are taken into full account. The proposed approach is feasible without knowing any prior information or estimate of the target maneuvers, which are regarded as the unknown disturbance input. Therefore, the TPN‐type guidance is extended to intercept the maneuvering targets with bounded unknown accelerations. Moreover, the explicit physical significance is given for each component of the navigation gain expression of MTPN. The inverse optimal control approach and the input‐to‐state stability are introduced to the MTPN design. Using the MTPN, the homing missile can conduct the terminal homing guidance phase in any particular direction with respect to their nonmaneuvering or maneuvering targets through choosing an appropriate inertial reference coordinate system in practice. Using the proposed schemes, the line‐of‐sight (LOS) rate is globally exponentially stable for nonmaneuvering targets, and is input‐to‐state stability (ISS) for maneuvering targets with bounded unknown accelerations. Performed simulation results have confirmed the effectiveness of the proposed scheme.

Distributed inverse optimal control

This paper develops a distributed approach for inverse optimal control (IOC) in multi-agent systems. Here each agent can only communicate with certain nearby neighbors and only accesses segments of system’s trajectory, which is not sufficient for the agent to solve the IOC problem alone. By introducing the concept of the data effectiveness and bridging the connection between each segment and its contribution to solving IOC, we formulate the IOC problem as a problem of achieving least-square solutions via a distributed algorithm. Simulations are provided to validate the proposed distributed IOC approach.

Learning Occupancy Priors of Human Motion From Semantic Maps of Urban Environments

Understanding and anticipating human activity is an important capability for intelligent systems in mobile robotics, autonomous driving, and video surveillance. While learning from demonstrations with on-site collected trajectory data is a powerful approach to discover recurrent motion patterns, generalization to new environments, where sufficient motion data are not readily available, remains a challenge. In many cases, however, semantic information about the environment is a highly informative cue for the prediction of pedestrian motion or the estimation of collision risks. In this work, we infer occupancy priors of human motion using only semantic environment information as input. To this end, we apply and discuss a traditional Inverse Optimal Control approach, and propose a novel approach based on Convolutional Neural Networks (CNN) to predict future occupancy maps. Our CNN method produces flexible context-aware occupancy estimations for semantically uniform map regions and generalizes well already with small amounts of training data. Evaluated on synthetic and real-world data, it shows superior results compared to several baselines, marking a qualitative step-up in semantic environment assessment.

Consensusability and Global Optimality of Discrete-Time Linear Multiagent Systems.

The consensusability and global optimality problems are solved for the discrete-time linear multiagent system (MAS) with marginally stable and strictly unstable dynamics. A unified framework is proposed by capturing the maximal disc-guaranteed gain margin (GGM) of the discrete-time linear quadratic regulator (LQR). Sufficient and necessary conditions on consensusability are established. Two bounds of the consensus region are derived only in terms of the unstable eigenvalues of the agent' dynamics. For the single-input MAS, by proving that the radius of the consensus region exactly equals the reciprocal of the Mahler measure of the agent' dynamics, we incidentally reveal the relation between the maximal GGM and the intrinsic entropy rate of the system dynamics for single-input discrete-time linear systems. By employing the inverse optimal control approach, it is proved that the globally optimal consensus is achieved, if and only if the associated Laplacian matrix is a simple matrix and all its nonzero eigenvalues can be radially projected into a specific subset of the consensus region. Moreover, the limitation on the eigenvalues vanishes for the marginally stable MAS. As an application of the global optimality, the minimum-energy-distributed consensus control problem is solved for the marginally stable MAS. Finally, a numerical example is given to demonstrate the effectiveness of the obtained results.

Distributed optimal consensus with obstacle avoidance algorithm of mixed-order UAVs–USVs–UUVs systems

This article addresses the formation control with obstacle avoidance for a mixed-order linear multi-agent in the ocean environment via a distributed optimal control approach. The considered system is comprised of unmanned aerial vehicles (UAVs), unmanned surface vehicles (USVs) and unmanned underwater vehicles (UUVs). A consensus control law for horizontal direction and height direction is then designed uniformly, and the stability is ensured using block Kronecker product and matrix transformation theory. A linear quadratic regulator method is further designed to achieve distributed optimization. Moreover, for obstacle avoidance in the UUVs, a non-quadratic penalty function is constructed based on the seamount threat model by adopting an inverse optimal control approach. Simulation results confirm the efficiency of the consensus and obstacle avoidance.

Inverse optimal control with time-varying objectives: application to human jumping movement analysis

Analysis of complex human movements can provide valuable insights for movement rehabilitation, sports training, humanoid robot design and control, and human–robot interaction. To accomplish complex movement, the central nervous system must coordinate the musculo-skeletal system to achieve task and internal (e.g., effort minimisation) objectives. This paper proposes an inverse optimal control approach for analysing complex human movement that does not assume that the control objective(s) remains constant throughout the movement. The movement trajectory is assumed to be optimal with respect to a cost function composed of the sum of weighted basis cost functions, which may be time varying. The weights of the cost function are recovered using a sliding window. To illustrate the proposed approach, a dataset consisting of standing broad jump to targets at three different distances is collected. The method can be used to extract control objectives that influence task success, identify different motion strategies/styles, as well as to observe how control strategy changes during the motor learning process. Kinematic analysis confirms that the identified control objectives, including centre-of-mass takeoff vector and foot placement upon landing are important to ensure that a given participant lands on the target. The dataset, including nearly 800 jump trajectories from 22 participants is also provided.

An inverse optimal approach to ship course-keeping control

Abstract This paper deals with the ship course tracking control problem in a novel inverse optimal control approach. The inverse optimal stabilization problem and inverse optimal gain assignment problem are firstly extended to general systems affine in the control with unknown control gain. It is shown that a sufficient condition to solve the inverse optimal control problem is the existence of a stabilization control law in a special form for a corresponding auxiliary system. Then, by employing backstepping technique, control laws are designed which solve the inverse optimal stabilization, inverse optimal adaptive stabilization and inverse optimal adaptive gain assignment problem of ship course control system, respectively. Simulations are included to illustrate the effectiveness of the proposed control algorithms.

Optimal Control and Inverse Optimal Control with Continuous Updating for Human Behavior Modeling

The theory of optimal control has received considerable attention to model motion behavior or decision making of humans. Most approaches are based on a fixed (or infinite) time horizon which implies that all information is available at the beginning of the time interval. Nevertheless, it is reasonable to believe that the human uses information defined by a continuously moving information horizon at each time instant and adapts accordingly. Therefore, in this paper, we propose an optimal feedback control approach based on the paradigm of continuous updating. The model parameters which define individual human behavior consist of the cost function parameters and the length of the information horizon, which can be identified via a corresponding inverse optimal control approach. We show the applicability of the approach with simulations of a potential application example of human behavior identification from the point of view of a driving assistance system.

IOC Based Trajectory Generation to Increase Human Acceptance of Robot Motions in Collaborative Tasks

Abstract Collaboration between humans and robots is an important aspect of Industry 4.0. It can be improved by incorporating human-like characteristics into robot motion planning. It is assumed that humans move optimal with respect to a certain objective or cost function. To find this function, also for a robot, we use an inverse optimal control approach identifying what linear weighted combination of physically interpretable cost functions best mimics human point-to-point motions. A bi-level optimization is used, where the upper level compares the optimal robot result of the lower level with human reference motions. Two depth cameras are combined in a setup to record these reference motions. The resulting weighted cost functions are then used to generate new motions for a seven degrees of freedom robot arm. The resulting optimized motions are compared to standard robot motions based on linear interpolation in joint or task space. The comparison is performed by means of a small experiment where preliminary observations show that humans experience these motions as more anthropomorphic and feel at least equally comfortable and safe compared to existing motion planning strategies.