Inverse Optimal Control Approach Research Articles

Inverse Optimal Control for Multiphase Cost Functions

In this paper, we consider a dynamical system whose trajectory is a result of minimizing a multiphase cost function. The multiphase cost function is assumed to be a weighted sum of specified features (or basis functions) with phase-dependent weights that switch at some unknown phase transition points. A new inverse optimal control approach for recovering the cost weights of each phase and estimating the phase transition points is proposed. The key idea is to use a length-adapted window moving along the observed trajectory, where the window length is determined by finding the minimal observation length that suffices for a successful cost weight recovery. The effectiveness of the proposed method is first evaluated on a simulated robot arm, and then, demonstrated on a dataset of human participants performing a series of squatting tasks. The results demonstrate that the proposed method reliably retrieves the cost function of each phase and segments each phase of motion from the trajectory with a segmentation accuracy above 90%.

Fusion of inverse optimal and model predictive control strategies

In this study, model predictive control (MPC) and inverse optimal control (IOC) approaches are merged with each other and a new control strategy is evolved. The key feature in this strategy is to solve the IOC problem repeatedly for each receding horizon of the model predictive control approach. From another perspective, MPC structure is inserted to IOC problem and thus, IOC problem is solved repeatedly using different initial conditions at the beginning of each receding horizon. In the solution phase of IOC, the parameters of the candidate control Lyapunov function matrix are estimated using the global evolutionary Big Bang-Big Crunch (BB-BC) optimization algorithm in an on-line manner. Thus, the proposed control structure solves the optimal control problem in classical MPC approach to the search of an appropriate candidate control Lyapunov function matrix for each control horizon. The comparison of the proposed method with the other related control methods are performed on the ball and beam system via simulations and real-time applications.

Invariant-based optimal composite stimulated Raman exact passage

Shortcuts to adiabaticity have been put forward for accelerating slow adiabatic passages in various quantum systems with tremendous applications for performing quantum information processing tasks. In this paper, we propose a hybrid protocol to achieve stimulated Raman exact passage (STIREP) by combining invariant-based inverse engineering, optimal control, and composite pulse approaches. We first derive the general solution and the corresponding pulse shapes by invariant-based inverse engineering. Counterintuitive and optimal (intuitive) pulse sequences are formulated in this context and incorporated into composite sequences. Such composite STIREP not only features robustness against the fluctuation of laser intensity, but also reduces the operation time and energy cost.

Field-of-view constrained two-stage guidance law design for three-dimensional salvo attack of multiple missiles via an optimal control approach

This study considers the cooperative guidance problem for multiple missiles against a stationary target with the field-of-view constraint. A two-stage guidance scheme is proposed to realize salvo attack of multiple missiles without using any information on time-to-go or its estimation. In the first guidance phase, this study integrates the cooperative guidance problem, state tracking problem and field-of-view constraint into a unified optimal control framework. A non-quadratic field-of-view constraint cost is innovatively constructed via an inverse optimal control approach, which leads to an analytical, distributed, and optimal guidance law to provide favored initial conditions for the latter phase. The stability and optimality of the closed-loop system is also proven. With respect to the second guidance phase, missiles disconnect from each other and are governed by the typical pure proportional navigation guidance law to eventually realize the salvo attack mission. Numerical simulations are performed to validate the theoretical results.

Inverse optimal control for unmanned aerial helicopters with disturbances

SummaryThis paper proposes an optimal control method of an unmanned aerial helicopter (UAH) with unknown disturbances. Solving the Hamilton‐Jacobi‐Bellman (HJB) equation is considered as the common approach to design an optimal controller under a meaningful cost function when facing the nonlinear optimal control problem. However, the HJB equation is hard to solve even for a simple problem. The inverse optimal control method that avoids the difficulties of solving the HJB equation has been adopted. In this inverse optimal control approach, a stabilizing optimal control law and a particular cost function that are obtained by a control Lyapunov function are required. An integrator backstepping method is used in designing the optimal control law of the UAH. Furthermore, a disturbance‐observer–based control (DOBC) approach has been adopted in the optimal control law for dealing with the unknown disturbances of the UAH system. Simulation results have been given to certify the stability of the nonlinear UAH system and the validity of this developed control method.

An improved optimal control strategy for hybrid AC/DC power system

The additional modulation function of high voltage direct current (HVDC) links can improve the system stability while the unavoidable random external disturbance and sudden changes seriously affect the controller performance and the system stability. In this paper, an improved control scheme based on inverse optimal adaptive control is proposed for the hybrid AC/DC system which considers the system parameter error, external disturbance and other random perturbation. The proposed control scheme combines the inverse optimal control approach and the adaptive control method to account for the modelling errors and uncertainty, which aims at the disturbance attenuation and the optimal operating of the hybrid AC/DC power system. Besides, the improved PI (Proportional-Integral) controller based on the lead-lag link for the current control of HVDC system is proposed to further improve system dynamic performance. Simulations conducted on a typical hybrid AC/DC system show the correctness and validity of the proposed control method.

Optimal formation control and collision avoidance in environment with multiple rectangle obstacles

This paper addresses formation control problem with collision avoidance for general linear multi-agent systems via an optimal control strategy. In the proposed optimal control strategy, a novel potential function is designed to accomplish formation of multi-agent systems (MASs) with obstacle/collision avoidance capability, which can avoid rectangle obstacles accurately. In this potential function, a novel relative velocity based self-adaptive detection region is proposed to avoid collisions with adjacent agents. Moreover, a non-quadratic avoidance performance index is constructed based on inverse optimal control approach. Then, the optimal control strategy is designed to guarantee the asymptotic stability of the closed-loop system and optimality of the proposed performance index. Finally, a simulation example is given to illustrate the efficiency of the proposed approach.

Humanoid gait generation in complex environments based on template models and optimality principles learned from human beings

In this paper, we present an inverse optimal control-based transfer of motions from human experiments to humanoid robots and apply it to walking in constrained environments. To this end, we introduce a 3D template model, which describes motion on the basis of center-of-mass trajectory, foot trajectories, upper-body orientation, and phase duration. Despite its abstract architecture, with prismatic joints combined with damped series elastic actuators instead of knees, the model (including dynamics and constraints) is suitable for describing both human and humanoid locomotion with appropriate parameters. We present and apply an inverse optimal control approach to identify optimality criteria based on human motion capture experiments. The identified optimal strategy is then transferred to a humanoid robot template model for gait generation by solving an optimal control problem, which takes into account the properties of the robot and differences in the environment. The results of this step are the center-of-mass trajectory, the foot trajectories, the torso orientation, and the single and double support phase durations for a sequence of steps, allowing the humanoid robot to walk within a new environment. In a previous paper, we have already presented one computational cycle (from motion capture data to an optimized robot template motion) for the example of walking over irregular stepping stones with the aim of transferring the motion to two very different humanoid robots (iCub@Heidelberg and HRP-2@LAAS). This study represents an extension, containing an entirely new part on the transfer of the optimized template motion to the iCub robot by means of inverse kinematics in a dynamic simulation environment and also on the real robot.

An ensemble inverse optimal control approach for robotic task learning and adaptation

This paper contributes a novel framework to efficiently learn cost-to-go function representations for robotic tasks with latent modes. The proposed approach relies on the principle behind ensemble methods, where improved performance is obtained by aggregating a group of simple models, each of which can be efficiently learnedq. The maximum-entropy approximation is adopted as an effective initialization and the quality of this surrogate is guaranteed by a theoretical bound. Our approach also provides an alternative perspective to view the popular mixture of Gaussians under the framework of inverse optimal control. We further propose to enforce a dynamics on the model ensemble, using Kalman estimation to infer and modulate model modes. This allows robots to exploit the demonstration redundancy and to adapt to human interventions, especially in tasks where sensory observations are non-Markovian. The framework is demonstrated with a synthetic inverted pendulum example and online adaptation tasks, which include robotic handwriting and mail delivery.

An Inverse Optimal Control Approach to Explain Human Arm Reaching Control Based on Multiple Internal Models

Human motor control is highly efficient in generating accurate and appropriate motor behavior for a multitude of tasks. This paper examines how kinematic and dynamic properties of the musculoskeletal system are controlled to achieve such efficiency. Even though recent studies have shown that the human motor control relies on multiple models, how the central nervous system (CNS) controls this combination is not fully addressed. In this study, we utilize an Inverse Optimal Control (IOC) framework in order to find the combination of those internal models and how this combination changes for different reaching tasks. We conducted an experiment where participants executed a comprehensive set of free-space reaching motions. The results show that there is a trade-off between kinematics and dynamics based controllers depending on the reaching task. In addition, this trade-off depends on the initial and final arm configurations, which in turn affect the musculoskeletal load to be controlled. Given this insight, we further provide a discomfort metric to demonstrate its influence on the contribution of different inverse internal models. This formulation together with our analysis not only support the multiple internal models (MIMs) hypothesis but also suggest a hierarchical framework for the control of human reaching motions by the CNS.

A Hybrid Framework for Understanding and Predicting Human Reaching Motions.

Robots collaborating naturally with a human partner in a confined workspace need to understand and predict human motions. For understanding, a model-based approach is required as the human motor control system relies on the biomechanical properties to control and execute actions. The model-based control models explain human motions descriptively, which in turn enables predicting and analyzing human movement behaviors. In motor control, reaching motions are framed as an optimization problem. However, different optimality criteria predict disparate motion behavior. Therefore, the inverse problem—finding the optimality criterion from a given arm motion trajectory—is not unique. This paper implements an inverse optimal control (IOC) approach to determine the combination of cost functions that governs a motion execution. The results indicate that reaching motions depend on a trade-off between kinematics and dynamics related cost functions. However, the computational efficiency is not sufficient for online prediction to be utilized for HRI. In order to predict human reaching motions with high efficiency and accuracy, we combine the IOC approach with a probabilistic movement primitives formulation. This hybrid model allows an online-capable prediction while taking into account motor variability and the interpersonal differences. The proposed framework affords a descriptive and a generative model of human reaching motions which can be effectively utilized online for human-in-the-loop robot control and task execution.

Toward optimal mapping of human dual-arm motion to humanoid motion for tasks involving contact with the environment

In this article, we explore human motion skills in the dual-arm manipulation tasks that include contact with equipment with the final aim to generate human-like humanoid motion. Human motion is analyzed using the optimization approaches starting with the assumption that human motion is optimal. A combination of commonly used optimization criteria in the joint space with the weight coefficients is considered: minimization of kinetic energy, minimization of joint velocities, minimization of the distance between the current and ergonomic positions, and maximization of manipulability. The contribution of each criterion for seven different dual-arm manipulation tasks to provide the most accurate imitation of the human motion is given via suggested inverse optimization approach calculating values of weight coefficients. The effects on actors’ body characteristics and the characteristics of the environment (involved equipment) on the choice of criterion functions are additionally analyzed. The optimal combination of weight coefficients calculated by the inverse optimization approach is used in our inverse kinematics algorithm to transfer human motion skills to the motion of the humanoid robots. The results show that the optimal combination of weight coefficients is able to generate human-like humanoid motions rather than individual one of the considered criterion functions. The recorded human motion and the motion of the humanoid robot ROMEO, obtained with the strategy used by human and defined by our inverse optimal control approach, for the tasks “opening/closing a drawer” are assessed visually and quantitatively.

Germinal Center Optimization Applied to Neural Inverse Optimal Control for an All-Terrain Tracked Robot

Nowadays, there are several meta-heuristics algorithms which offer solutions for multi-variate optimization problems. These algorithms use a population of candidate solutions which explore the search space, where the leadership plays a big role in the exploration-exploitation equilibrium. In this work, we propose to use a Germinal Center Optimization algorithm (GCO) which implements temporal leadership through modeling a non-uniform competitive-based distribution for particle selection. GCO is used to find an optimal set of parameters for a neural inverse optimal control applied to all-terrain tracked robot. In the Neural Inverse Optimal Control (NIOC) scheme, a neural identifier, based on Recurrent High Orden Neural Network (RHONN) trained with an extended kalman filter algorithm, is used to obtain a model of the system, then, a control law is design using such model with the inverse optimal control approach. The RHONN identifier is developed without knowledge of the plant model or its parameters, on the other hand, the inverse optimal control is designed for tracking velocity references. Applicability of the proposed scheme is illustrated using simulations results as well as real-time experimental results with an all-terrain tracked robot.

Gait analysis using optimality criteria imputed from human data

This study proposes an efficient and automatic tool to understand and analyze the human natural and fast gait tasks. First, the gait tasks are modeled as a nonlinear optimal control problem along with a nonlinear model predictive control, usually used in the humanoid robots control. Second, under the assumption that the walking motions are the result of an optimization process, the identification of plausible optimality criteria weight values is achieved with an inverse optimal control (IOC) approach. Our IOC walking scheme is performed on ten subjects with ten trials for each gait task considered. Results show that the variability observed in the experimental data are exhibited by our proposed scheme. Finally, an easy distinguish can be made between the gait tasks only by checking out the exhibited criteria weight values.

Inverse optimal controller based on extended Kalman filter for discrete‐time nonlinear systems

SummaryIn this study, we present an inverse optimal control approach based on extended Kalman filter (EKF) algorithm to solve the optimal control problem of discrete‐time affine nonlinear systems. The main aim of inverse optimal control is to circumvent the tedious task of solving the Hamilton‐Jacobi‐Bellman equation that results from the classical solution of a nonlinear optimal control problem. Here, the inverse optimal controller is based on defining an appropriate quadratic control Lyapunov function (CLF) where the parameters of this candidate CLF were estimated by adopting the EKF equations. The root mean square error of the system states is used as the observed error in the case of classical EKF algorithm application, whereas, here, the EKF tries to eliminate the same root mean square error defined over the parameters by generating a CLF matrix with appropriate elements. The performance and the applicability of the proposed scheme is illustrated through both simulations performed on a nonlinear system model and a real‐time laboratory experiment. Simulation study demonstrate the effectiveness of the proposed method in comparison with 2 other inverse control approaches. Finally, the proposed controller is implemented on a professional control board to stabilize a DC‐DC boost converter and minimize a meaningful cost function. The experimental results show the applicability and effectiveness of the proposed EKF‐based inverse optimal control even in real‐time control systems with a very short time constant.

Using Optimal Control to Disambiguate the Effect of Depression on Sensorimotor, Motivational and Goal-Setting Functions.

Differentiating the ability from the motivation to act is of central importance to psychiatric disorders in general and depression in particular. However, it has been difficult to develop quantitative approaches to relate depression to poor motor performance in goal-directed tasks. Here, we use an inverse optimal control approach to provide a computational framework that can be used to infer and factorize performance deficits into three components: sensorimotor speed, goal setting and motivation. Using a novel computer-simulated driving experiment, we found that (1) severity of depression is associated with both altered sensorimotor speed and motivational function; (2) moderately to severely depressed individuals show an increased distance from the stop sign indicating aversive learning affecting goal setting functions. Taken together, the inverse optimal control framework can disambiguate on an individual basis the sensorimotor from the motivational dysfunctions in depression, which may help to develop more precisely targeted interventions.

Optimal formation control with limited communication for multi-unmanned aerial vehicle in an obstacle-laden environment

This paper investigates the formation control problem of multiple unmanned aerial vehicles (UAVs) with limited communication in a known and realistic obstacle-laden environment. In order to deal with the limited communication constraints, the leader–follower strategy and the virtual leader strategy are integrated into an optimal control framework to formulate this formation control problem. This combination formation framework can be achieved by integrating a redefined directed graph and a proposed information vector. In more practical applications, an obstacle/collision avoidance strategy is achieved by constructing a non-quadratic cost function innovatively using a virtual flow field approach. The proposed optimal control laws, which derive from the local information rather than the global information, are proved to guarantee the stability of the close-loop system by an inverse optimal control approach. The simulation results demonstrate the effectiveness of the formation flight of multiple UAVs with limited communication in an obstacle-laden environment.

Adaptive learning of human motor behaviors: An evolving inverse optimal control approach

Understanding human behaviors has received considerable attention in neuroscience literature. Existing research addressed the main question: given measurements of a certain human movement, what is the underlying optimality criteria that human has optimized to fulfill this movement? Inverse Optimal Control (IOC) is a well-established approach to understand the biological movements in terms of the optimal control theory. IOC learns the criterion that best describes the demonstrated human behavior. Thus far, gradient-based techniques have been used to obtain the unknown behavior cost. However, these techniques are limited by locating only local optimum parameters. In this paper, behavior learning is modeled as an Inverse Linear Quadratic Regulator (ILQR) problem, where linear behavior dynamics and a quadratic cost are assumed. An efficient meta-heuristic technique, Particle Swarm Optimization (PSO), is used to retrieve the unknown cost in the proposed ILQR problem. Moreover, an evolving-ILQR algorithm is proposed to refine the learned cost once new unseen demonstrations exist to overcome the over-fitting problem. The reach-to-grasp behavior is studied to quantify the proposed approaches. Results are encouraging and show consistency with that in neuroscience literature. Meanwhile, the evolving-ILQR algorithm has quantified in successive scenarios, where the so far retrieved behavior cost has incrementally refined once new unseen demonstrations are available.

Why Don't We Move Slower? The Value of Time in the Neural Control of Action.

Movement time is a fundamental characteristic of neural motor control, but the principles underlying its formation remain little known. This work addresses that question within the inverse optimal control framework where the challenge is to uncover what optimality criterion underlies a system's behavior. Here we rely on the "cost of time" theory that finds its roots into the brain's tendency to discount the actual value of future reward. It asserts that the time elapsed until action completion entails a cost, thereby making slow moves nonoptimal. By means of a thorough theoretical analysis, the present article shows how to sample the infinitesimal values of the time cost without prior assumption about its hypothetical nature and emphasizes its sigmoidal shape for reaching.

Control óptimo inverso como alternativa para la regulación de un convertidor DC-DC elevador

El presente artículo aplica los conceptos del control óptimo inverso para regular la tensión de salida de un circuito convertidor de potencia DC-DC elevador. El control óptimo inverso define a priori una señal de control óptima y busca a posteriori el funcional que esta última minimiza, facilitando la solución del problema de control óptimo tras evitar resolver explícitamente la ecuación de Hamilton Jacobi Bellman. Se plantean las ideas generales del método, definiendo las ecuaciones necesarias para calcular la ley de control óptima inversa. Posteriormente se realiza verificación numérica en MATLAB® para la acción de control, operando bajo la influencia de perturbaciones en el modelo del sistema. De los resultados de simulación obtenidos se confirma la pertinencia de la técnica para manipular el comportamiento dinámico del circuito. Trabajos futuros incluyen la validación en laboratorio para las predicciones teóricas propuestas.