General Optimal Control Problem Research Articles

Mainardi smoothing homotopy method for solving nonlinear optimal control problems

This paper proposes a new type of smoothing homotopy method for solving general nonlinear optimal control problems via the indirect method, leveraging the Mainardi kernel as the smoothing kernel. The Mainardi kernel is firstly derived from the fundamental solution of the time-fractional diffusion-wave equation, representing a generalized form of the Gaussian kernel. By altering the fractional derivative order, the kernel can seamlessly switch between non-Gaussian and Gaussian forms. Then, parts of the two-point boundary value problems are convolved with the smoothing kernel, and the resulting surrogates are incorporated into the necessary conditions, replacing the terminal state and costate variables. The homotopy process is used for the smoothing parameter: increasing this parameter enhances the smoothing effect, simplifying the homotopy problems, while a parameter value of zero implies no smoothing, reverting to the original problems. Additionally, explicit expressions for the Mainardi kernel at specific derivative orders are derived, avoiding the inefficiencies associated with fractional derivatives. Simulation examples demonstrate that the proposed method offers significant advantages in flexibility and convergence compared to the traditional Gaussian smoothing homotopy method.

A general time-inconsistent stochastic optimal control problem with delay

A general time-inconsistent stochastic optimal control problem with delay

Error analysis of Haar wavelet-based Galerkin numerical method with application to various nonlinear optimal control problems

ABSTRACT First, this paper defines a general nonlinear optimal control problem with state/control constraints and its approximation problem as the Haar wavelet Galerkin optimal control problem (HWGOCP). Then, a Haar wavelet-based Galerkin numerical method has been developed, which converts it to a nonlinear optimization problem. We theoretically prove that a Haar wavelet feasible solution of HWGOCP will exist. We also show that the approximate solutions of HWGOCP are consistent and converge to the optimal solution of the problem. A variety of application problems have been considered, which include optimal control of tumour growth using Chemotherapy drugs, optimal control of infection via the SIS model using treatment, the Brachistochrone problem in mechanics, optimal control of mold using a fungicide, optimal control of pH value of a chemical reaction to determine the quality of a product, etc.

Optimal control for both forward and backward discrete-time systems

Forward discrete-time systems use past information to update the current state, while backward discrete-time systems use future information to update the current state. This study focuses on optimal control problems within the context of forward and backward discrete-time systems. We begin by investigating a general optimal control problem for both forward and backward discrete-time systems. Leveraging the inherent properties of these systems and the Bellman optimality principle, we derive recursive equations as a means to solve such optimal control problems. Using these recursive equations, we obtain analytical expressions for both the optimal controls and optimal values of bang–bang and linear quadratic optimal control problems. Finally, we present a numerical example and an industrial wastewater treatment problem to illustrate and demonstrate our findings.

Sushchestvovanie sublorentsevykh dlinneyshikh

Sufficient conditions for the existence of optimal trajectories in general optimalcontrol problems with free terminal time as well as in sub-Lorentzian problems are obtained.

A necessary optimality condition for extended weighted generalized fractional optimal control problems

Using the recent weighted generalized fractional order operators of Hattaf, a general fractional optimal control problem without constraints on the values of the control functions is formulated and a corresponding (weak) version of Pontryagin’s maximum principle is proved. As corollaries, necessary optimality conditions for Caputo–Fabrizio, Atangana–Baleanu and weighted Atangana–Baleanu fractional dynamic optimization problems are trivially obtained. As an application, the weighted generalized fractional problem of the calculus of variations is investigated and a new more general fractional Euler–Lagrange equation is given.

ON NORMALITY IN OPTIMAL CONTROL PROBLEMS WITH STATE CONSTRAINTS

A general optimal control problem with endpoint, mixed and state constraints is considered. The question of normality of the known necessary optimality conditions is investigated. Normality stands for the positiveness of the Lagrange multiplier λ0 corresponding to the cost functional. In order to prove the normality condition, an appropriate derivative operator for the state constraints is constructed, which acts in a specific Hilbert space and has the properties of surjection and continuity.

Smoothing Homotopy Methods for Solving Nonlinear Optimal Control Problems

Inspired by the widely used smoothing techniques in image processing and statistical analysis, a novel smoothing homotopy paradigm is proposed in this paper for solving general optimal control problems with the indirect method. In contrast to the existing approaches, the sensitivity associated with the two-point-boundary-value problem (TPBVP) in optimal control is reduced by convoluting appropriate part of the TPBVP with a smoothing kernel. The homotopic process is applied to the bandwidth of the smoothing: a larger bandwidth produces a stronger smoothing effect, rendering the TPBVP better behaved and much easier to solve; a zero bandwidth results in no smoothing and thus leads to the original TPBVP. Depending on whether the Gaussian smoothing kernel or a non-Gaussian kernel is used in the particular implementation, two different smoothing homotopy methods are developed, each with distinctive features of its own. Challenging numerical examples are provided to demonstrate the working and effectiveness of the methods.

Stochastic Framework for Optimal Control of Planetary Reentry Trajectories Under Multilevel Uncertainties

We present a novel stochastic optimal control framework that accounts for various types of uncertainties, with application to reentry trajectory planning. The formulation of the optimal trajectory control problem is presented in the context of an indirect method where a functional objective associated with the terminal vehicle speed is to be minimized. Uncertain input parameters in the optimal trajectory control model, including aerodynamic parameters and initial and terminal conditions, are modeled as aleatory random variables, while the statistical parameters of these aleatory distributions are themselves random variables. The parametric and model uncertainties are simultaneously propagated through an extended polynomial chaos expansion (EPCE) formalism. Several metrics are described to evaluate response statistics and presented as insightful tools for robust decision making. Specifically, the response probability density function (PDF) reflecting influence of both epistemic and aleatory uncertainties is obtained. By sampling over the random variables representing model error, an ensemble of response PDFs is generated and the associated failure probability is estimated as a random variable with its own polynomial chaos expansion. Besides, the sensitivity index functions of response PDF with respect to the statistical parameters are evaluated. Coupling parametric and model uncertainties within the EPCE framework leads to a robust and efficient paradigm for multilevel uncertainty propagation and PDF characterization in general optimal control problems.

GOPS: A general optimal control problem solver for autonomous driving and industrial control applications

Solving optimal control problems serves as the basic demand of industrial control tasks. Existing methods like model predictive control often suffer from heavy online computational burdens. Reinforcement learning has shown promise in computer and board games but has yet to be widely adopted in industrial applications due to a lack of accessible, high-accuracy solvers. Current Reinforcement learning (RL) solvers are often developed for academic research and require a significant amount of theoretical knowledge and programming skills. Besides, many of them only support Python-based environments and limit to model-free algorithms. To address this gap, this paper develops General Optimal control Problems Solver (GOPS), an easy-to-use RL solver package that aims to build real-time and high-performance controllers in industrial fields. GOPS is built with a highly modular structure that retains a flexible framework for secondary development. Considering the diversity of industrial control tasks, GOPS also includes a conversion tool that allows for the use of Matlab/Simulink to support environment construction, controller design, and performance validation. To handle large-scale problems, GOPS can automatically create various serial and parallel trainers by flexibly combining embedded buffers and samplers. It offers a variety of common approximate functions for policy and value functions, including polynomial, multilayer perceptron, convolutional neural network, etc. Additionally, constrained and robust algorithms for special industrial control systems with state constraints and model uncertainties are also integrated into GOPS. Several examples, including linear quadratic control, inverted double pendulum, vehicle tracking, humanoid robot, obstacle avoidance, and active suspension control, are tested to verify the performances of GOPS.

Regularity of optimal solutions and the optimal cost for hybrid dynamical systems via reachability analysis

For a general optimal control problem for dynamical systems with hybrid dynamics, we study the dependency of the optimal cost on perturbations to problem constraints and mappings. The former limits the allowable values for the state, input, initial and terminal condition, and the terminal time, while the latter is given by the right-hand sides of the differential/difference inclusions defining the dynamics, as well as the functions defining the cost functional We show that upper and lower semicontinuous dependence of solutions on initial conditions – properties that are captured by outer and inner well-posedness, respectively – lead to the existence of a solution to the optimal control problem and upper/lower semicontinuity of the optimal cost. In particular, by exploiting properties of finite horizon reachable sets for hybrid systems, we show that the optimal cost varies upper semicontinuously when the hybrid system is outer well-posed, and lower semicontinuously when it is inner well-posed and an additional assumption requiring partial knowledge of solutions holds. Consequently, when the system is both inner and outer well-posed and the aforementioned assumption holds, the optimal cost varies continuously, and optimal solutions vary outer/upper semicontinuously. We further show that even in the absence of this solution-based assumption, the optimal cost (respectively, solutions) can be continuously (respectively, outer/upper semicontinuously) approximated. Results are demonstrated by examples, theoretically and numerically.

A geometric optimal control approach for imitation and generalization of manipulation skills

Daily manipulation tasks are characterized by regular features associated with the task structure, which can be described by multiple geometric primitives related to actions and object shapes. Only using Cartesian coordinate systems cannot fully represent such geometric descriptors. In this article, we consider other candidate coordinate systems and propose a learning approach to extract the optimal representation of an observed movement/behavior from these coordinates. This is achieved by using an extension of Gaussian distributions on Riemannian manifolds, which is used to analyze a small set of user demonstrations statistically represented in different coordinate systems. We formulate the skill generalization as a general optimal control problem based on the (iterative) linear quadratic regulator ((i)LQR), where the Gaussian distribution in the proper coordinate systems is used to define the cost function. We apply our approach to object grasping and box-opening tasks in simulation and on a 7-axis Franka Emika robot using open-loop and feedback control, where precision matrices result in the automatic determination of feedback gains for the controller from very few demonstrations represented in multiple coordinate systems. The results show that the robot can exploit several geometries to execute the manipulation task and generalize it to new situations. The results show high variation along the do-not-matter direction, while maintaining the invariant characteristics of the task in the coordinate system(s) of interest. We then tested the approach in a human–robot shared control task. Results show that the robot can modify its grasping strategy based on the geometry of the object that the user decides to grasp.

Optimal feedback controls of stochastic linear quadratic control problems in infinite dimensions with random coefficients

It is a longstanding unsolved problem to characterize the optimal feedback controls for general linear quadratic optimal control problem of stochastic evolution equation with random coefficients. A solution to this problem is given in [22] under some assumptions which can be verified for interesting concrete models, such as controlled stochastic wave equations, controlled stochastic Schrödinger equations, etc. More precisely, the authors establish the equivalence between the existence of an optimal feedback operator and the solvability of the corresponding operator-valued, backward stochastic Riccati equations. However, their result cannot cover some important stochastic partial differential equations, such as stochastic heat equations, stochastic stokes equations, etc. A key contribution of the current work is to relax the C0-group assumption of unbounded linear operator A in [22] and using the contraction semigroup assumption instead. Therefore, our result can be well applicable in the linear quadratic problem of stochastic parabolic equations. To this end, we introduce a suitable notion to the aforementioned Riccati equation, and some delicate techniques which are even new in the finite dimensional case.

Variations of $v$-change of time in an optimal control problem with state and mixed constraints

For a general optimal control problem with state and regular mixed constraints we propose a proof of the maximum principle based on the so-called $v$-change of time variable $t \mapsto \tau$, under which the original time becomes an additional state variable subject to the equation $dt/d\tau = v(\tau)$, while the additional control variable $v(\tau)\geqslant 0$ is piecewise constant, and its values become arguments of the new problem.

Optimal control problems with time inconsistency

<abstract><p>In the present study, the necessary and sufficient conditions of equilibrium control for general optimal control problems with time inconsistency are established in sense of open-loop. As an application, the linear quadratic optimal control problems with time inconsistency were also explored and an explicit equilibrium control is constructed.</p></abstract>

A Non-dimensional Input Excitation Optimization Approach for Battery Health Parameter Estimation

Model parameter estimation is an important subject in control engineering, including the field of battery management. Input excitation optimization has become an emerging topic lately to improve the accuracy of estimation. Traditional optimization approach suffers from a fundamental issue in parameter uncertainty, as the target parameters for estimation, often needed for computing the optimization objective and constraints, are intrinsically unknown. In this study, we introduce a non-dimensional approach to optimize excitations for estimating the health-related Li-ion battery electrochemical parameters. Guided by the Buckingham π theorem, we derived a control-oriented non-dimensional battery model, excluding uncertain target parameters from the problem formulation. As a result, the optimization problem can be solved without any prior knowledge of target parameters. The applicable control input sequence can be recovered by rescaling the obtained non-dimensional sequence with the best available knowledge of the parameters. Furthermore, the proposed method reveals the fundamental impact of the unknown parameters on the solution of input optimization. In light of this finding, we propose two iterative excitation optimization strategies, which both significantly improve the robustness and reduce the complexity of the optimization problem. The proposed method can be generalized to solve general optimal control problems for a broad class of systems.

Multi-objective low-thrust spacecraft trajectory design using reachability analysis

One of the fundamental problems in spacecraft trajectory design is finding the optimal transfer trajectory that minimizes the propellant consumption and transfer time simultaneously. We formulate this as a multi-objective optimal control (MOC) problem that involves optimizing over the initial or final state, subject to state constraints. Drawing on recent developments in reachability analysis subject to state constraints, we show that the proposed MOC problem can be stated as an optimization problem subject to a constraint that involves the sub-level set of the viscosity solution of a quasi-variational inequality. We then generalize this approach to account for more general optimal control problems in Bolza form. We relate these problems to the Pareto front of the developed multi-objective programs. The proposed approach is demonstrated on two low-thrust orbital transfer problems around a rotating asteroid.

Local Minimum Principle for an Optimal Control Problem with a Nonregular Mixed Constraint

We consider the simplest optimal control problem with one nonregular mixed constraint $G(x,u)\le0,$ i.e., such a constraint that the gradient $G_u(x, u)$ can vanish on the surface $G = 0.$ Using the Dubovitskii--Milyutin theorem on the approximate separation of convex cones, we prove a first order necessary condition for a weak minimum in the form of the so-called local minimum principle, which is formulated in terms of functions of bounded variation, integrable functions, and Lebesgue--Stieltjes measures and does not use functionals from $(L^\infty)^*$. Two illustrative examples are provided. The work is based on the book by Milyutin [Maximum Principle in the General Problem of Optimal Control, Fizmatlit, Moscow, 2001 (in Russian)].

On necessary optimality conditions and exact penalization for a constrained fractional optimal control problem

AbstractThe purpose of this article is twofold. We first develop the first‐order necessary optimality conditions for a general constrained fractional optimal control problem using calculus of variation. These conditions are of the form of fractional ordinary differential equations which reduce to the conventional Euler–Lagrange equations when the dynamical system becomes a first‐order one. Then, we prove, via the optimality conditions established, that a popular penalty method used for conventional constrained optimal control and optimization is an “exact” penalty method for the fractional control problem by showing that an optimal solution to the penalized problem satisfies the optimality conditions of the original one when the penalty parameter is sufficiently large. An example is also provided to verify the necessary optimality conditions.

POD-Galerkin model order reduction for parametrized nonlinear time-dependent optimal flow control: an application to shallow water equations

AbstractIn the present paper we propose reduced order methods as a reliable strategy to efficiently solve parametrized optimal control problems governed by shallow waters equations in a solution tracking setting. The physical parametrized model we deal with is nonlinear and time dependent: this leads to very time consuming simulations which can be unbearable, e.g., in a marine environmental monitoring plan application. Our aim is to show how reduced order modelling could help in studying different configurations and phenomena in a fast way. After building the optimality system, we rely on a POD-Galerkin reduction in order to solve the optimal control problem in a low dimensional reduced space. The presented theoretical framework is actually suited to general nonlinear time dependent optimal control problems. The proposed methodology is finally tested with a numerical experiment: the reduced optimal control problem governed by shallow waters equations reproduces the desired velocity and height profiles faster than the standard model, still remaining accurate.