Nonlinear Optimal Control Problems Research Articles

Pontryagin Neural Networks for the Class of Optimal Control Problems With Integral Quadratic Cost

This work introduces Pontryagin Neural Networks (PoNNs), a specialised subset of Physics-Informed Neural Networks (PINNs) that aim to learn optimal control actions for optimal control problems (OCPs) characterised by integral quadratic cost functions. PoNNs employ the Pontryagin Minimum Principle (PMP) to establish necessary conditions for optimality, resulting in a two-point boundary value problem (TPBVP) that involves both state and costate variables within a system of ordinary differential equations (ODEs). By modelling the unknown solutions of the TPBVP with neural networks, PoNNs effectively learn the optimal control strategies. We also derive upper bounds on the generalisation error of PoNNs in solving these OCPs, taking into account the selection and quantity of training points along with the training error. To validate our theoretical analysis, we perform numerical experiments on benchmark linear and nonlinear OCPs. The results indicate that PoNNs can successfully learn open-loop control actions for the considered class of OCPs, outperforming the commercial software GPOPS-II in terms of both accuracy and computational efficiency. The reduced computational time suggests that PoNNs hold promise for real-time applications.

Nonlinear Optimal Control Based on FBDEs and its Application to AGV.

This article focuses on solving a finite-horizon nonlinear optimal control problem by using the Pontryagin's maximum principle. In practical applications, linearization is a common approach for solving nonlinear dynamical systems. However, it is not universally applicable due to various reasons, such as instability and low accuracy. In contrast to linearization, the inherent challenge in directly solving the above nonlinear optimal control problem lies in addressing the highly coupled nonlinear forward and backward differential equations. In order to address this problem, an equivalent relationship is established between these equations and a new optimization problem. By exploiting the inherent relationship between supervised learning and an optimization problem from the view of a dynamical system, a deep neural network framework is constructed for describing the new optimization problem. Furthermore, a numerical algorithm for optimal control, which is very powerful for a large variety of nonlinear dynamical systems, is implemented by training a deep residual network. Finally, the effectiveness of the algorithm is demonstrated by solving a trajectory tracking control problem for automatic guided vehicle. The obtained results reveal that the proposed control scheme can achieve high-precision tracking.

Fast Nonlinear Model Predictive Control Combining Online Trajectory Optimization and Value Function Regression

ABSTRACTModel predictive control (MPC) is a well‐developed method capable of handling complex control tasks. Implementation of an MPC requires solution of a deterministic finite horizon nonlinear optimal control (OC) problem. OC problems can be solved globally, explicitly expressing the optimal policy (and value function) as a function of the present state, or, locally, using online trajectory optimization, generating a solution that is only valid for the present state. For nonlinear problems however, neither is possible analytically, and the latter, finding a local solution online, is usually preferred. The difficulty with online trajectory optimization is that the solution must be available within a single sampling period. The main parameter affecting the computational demand of MPC is the length of the prediction horizon. The goal in this work is to reduce the length of the prediction horizon of a model predictive controller (MPC) to reduce computation time whilst preserving optimality guarantees. To this end, we propose approximation of the trajectory optimization problem for MPC by learning a finite horizon value function. The approximated value function is inserted into a truncated trajectory optimization problem so that the MPC can be attained with a reduced prediction horizon, ergo reduced computational load. By sampling the value function in a goal oriented way, we show that an effective approximate value function can be found by including both the function value and the gradients of the value function. The result is an accurate approximate MPC which leverages learning methodologies to reduce computational cost while still accounting for constraints.

Heat-Equation-Based Smoothing Homotopy Method for Nonlinear Optimal Control Problems

This paper presents a new heat-equation-based smoothing homotopy method for solving nonlinear optimal control problems with the indirect method. The surrogates, derived from the heat equation solution, are first incorporated into the necessary conditions as replacement of the terminal state and costate variables. The homotopy process is then applied to the heat conduction time: a longer time results in a more uniform temperature distribution and greater stability against initial temperature variations, thereby making the corresponding homotopy problem much easier to solve; zero time implies no heat conduction and reverts to the original problem. Furthermore, capitalizing on the heat equation’s boundedness characteristic, an efficient approach for determining the hypersensitive parameters is proposed, thus obviating the necessity of manual tuning. Challenging numerical examples are provided to demonstrate the superior performance of the proposed method, indicating that its convergence and efficiency are significantly enhanced compared to the original smoothing homotopy methods.

A novel unified variational finite difference (VFD) solution method for optimal control problems

This article is aimed to propose a simple yet efficient unified numerical strategy for solving both linear and non-linear optimal control problems. To do so, the general form of quadratic performance index function and nonlinear state equations are considered first. Then, the idea of variational differential quadrature method is used to convert the integral/differential equations to the equivalent algebraic form. Since time is the only independent variable is this research, the finite difference method with an equally-spaced discretization scheme would be a more appropriate technique rather than the differential quadrature approach. So, the implemented numerical solution is called now as the variational finite difference method. The method of Lagrange multipliers is then utilized for minimization purpose and, as a result, the final set of nonlinear algebraic equations are obtained. Finally, the quadratic and triadic forms of non-linearity are considered and an explicit formulation is represented for the residual and Jacobian of the Newton-based iterative solution procedure. To demonstrate the accuracy and efficiency of the proposed approach in the quadratic optimal control area, several benchmark problems involving linear time-invariant, linear time-variant and nonlinear examples are successfully solved and the results are confirmed with those existed in the literature.

Mainardi smoothing homotopy method for solving nonlinear optimal control problems

This paper proposes a new type of smoothing homotopy method for solving general nonlinear optimal control problems via the indirect method, leveraging the Mainardi kernel as the smoothing kernel. The Mainardi kernel is firstly derived from the fundamental solution of the time-fractional diffusion-wave equation, representing a generalized form of the Gaussian kernel. By altering the fractional derivative order, the kernel can seamlessly switch between non-Gaussian and Gaussian forms. Then, parts of the two-point boundary value problems are convolved with the smoothing kernel, and the resulting surrogates are incorporated into the necessary conditions, replacing the terminal state and costate variables. The homotopy process is used for the smoothing parameter: increasing this parameter enhances the smoothing effect, simplifying the homotopy problems, while a parameter value of zero implies no smoothing, reverting to the original problems. Additionally, explicit expressions for the Mainardi kernel at specific derivative orders are derived, avoiding the inefficiencies associated with fractional derivatives. Simulation examples demonstrate that the proposed method offers significant advantages in flexibility and convergence compared to the traditional Gaussian smoothing homotopy method.

A novel successive convexification framework with symplectic pseudospectral solutions for nonlinear optimal control problems of constrained pantograph delay systems

This paper proposes a novel symplectic pseudospectral iteration framework for the nonlinear optimal control problem (OCP) of a pantograph delay system with inequality constraints. Traditional numerical algorithms for OCPs with pantograph delay systems have predominantly focused on unconstrained scenarios, neglecting constrained problems. Therefore, we propose a novel symplectic pseudospectral iteration framework for constrained problems. First, we employ the successive convexification (SCvx) method to transform the original problem into a series of linear quadratic (LQ) problems, addressing nonlinearity. Then, leveraging time-scale transformations and parametric variational principles, we derive the first-order necessary conditions (FONCs) for these convexified problems, including a Hamiltonian two-point boundary value problem (HTBVP) with stretching terms and a linear complementarity problem. To handle pantograph time delays effectively, we employ a local Legendre-Gauss-Lobatto (LGL) pseudospectral method with proportional grid discretization. Finally, employing a symplectic indirect algorithm, we develop a symplectic pseudospectral method (SPM) for solving the transformed LQ problems, converting them into a system of sparse linear algebraic equations and a linear complementarity problem. Validation of the framework is performed using diverse illustrative examples, demonstrating strong agreement between SCvx and SPM in computational efficiency. Numerical experiments confirm the sparsity of the core matrix and robustness to initial guesses.

A gradient-based optimization algorithm to solve optimal control problems with conformable fractional-order derivatives

In this paper, we solve numerically a class of fractional optimal control problems in the conformable sense. We first propose a nonlinear fractional optimal control problem subject to general equality and inequality constraints. Then, based on a novel numerical integration scheme, we present a discretization method for the governed fractional-order system as well as the cost and constraint functionals, which yields a discretized approximate problem. We further derive the gradients of the cost and constraint functionals in regard to the discretized controls. On the basis of this result, a numerical optimization approach to solve the approximate problem is developed. Finally, three non-trivial example problems are solved to illustrate the applicability and effectiveness of the developed approach.

A nonlinear optimal control approach for 3-DOF four-cable driven parallel robots

In this article, a nonlinear optimal control approach is proposed for the dynamic model of 3-DOF four-cable driven parallel robots (CDPR). To solve the associated nonlinear optimal control problem, the dynamic model of the 3-DOF cable-driven parallel robot undergoes approximate linearization around a temporary operating point that is recomputed at each time-step of the control method. The linearization relies on Taylor series expansion and on the associated Jacobian matrices. For the linearized state-space model of the 3-DOF cable-driven parallel robot a stabilizing optimal (H-infinity) feedback controller is designed. To compute the controller’s feedback gains an algebraic Riccati equation is repetitively solved at each iteration of the control algorithm. The stability properties of the control method are proven through Lyapunov analysis. The proposed nonlinear optimal control approach achieves fast and accurate tracking of reference setpoints under moderate variations of the control inputs and a minimum dispersion of energy.

Control strategy for trading off solar power and control input while rendezvous and docking

This paper presents a novel strategy for decoupling the attitude and orbit equations of a CubeSat for rendezvous and docking with an uncontrollable cooperative target. For computing a safe trajectory, the proposed control algorithm takes into account the solar energy received by the CubeSat. The CubeSat is equipped with a thruster for orbit maneuvers, magnetic coils for attitude control, and four fixed single-sided rectangular solar panels. The coupled dynamics arising from the dynamical model of the spacecraft is solved using a novel weighted vector-based approach. This paper presents a linearized nonlinear optimal control problem arising in small spacecrafts while trying to maximize solar energy input in the rendezvous and docking process. An intermediate orbit is defined and used to divide the problem into two different optimal control problems: rendezvous optimization and docking optimization problems. A geometrical approach based on the shape of the chaser and the target is contemplated for collision avoidance while docking. The proposed controller design is modified whenever the sunlight is obstructed by the Earth, and the maneuvering controls are redesigned accordingly for optimal rendezvous and docking. Numerical simulations have been carried out to show the efficacy of the proposed concept for steering the trade-off between solar energy and control input during the rendezvous and docking of CubeSat with a tumbling target.

Optimal control in models of virus propagation

Based on the SEIRD model, we consider that when multiple viruses of different virulence coexist, the more virulent virus will reinfect nodes already infected by the less virulent virus, which we call here Superexposed. Based on the state transitions, the corresponding differential equations and cost functions are established, then building the corresponding optimal control problem, where the vaccine efficiency and drug efficiency are controlled variables. This nonlinear optimal control problem is solved by Pontryagin’s maximum principle to finding the structure of the optimal control strategies. Based on the definition of the basic regeneration number, yielding the R0 value for the model, then discussed the final epidemic size. In the numerical analysis section, we validate the accuracy of the structure, fitting the behavior of each state and the effect of different parameter values.

Nonlinear optimal control for triangular tethered multi-satellite formations

Triangular satellite constellations are created by three satellites which are linked through tethers. Such formations serve distributed observation space missions or the creation of high-density satellite communication networks over high-frequency bands. In a tethered multi-satellite system consisting of three-satellites in triangular formation the associated dynamic model exhibits strong nonlinearities. Stabilization and precise positioning of the multi-satellite constellation is a nontrivial task and the solution of the associated nonlinear control problem is an elaborated procedure. In this article a novel nonlinear optimal control method is applied to the above-noted model the tethered multi-satellite system in triangular formation. First, the state-space model of the triangular tethered multi-satellite formation undergoes approximate linearization around a temporary operating point that is recomputed at each time-step of the control method. The linearization relies on Taylor series expansion and on the associated Jacobian matrices. For the linearized state-space model of the tethered satellites a stabilizing optimal (H-infinity) feedback controller is designed. This controller stands for the solution of the nonlinear optimal control problem under model uncertainty and external perturbations. To compute the controller’s feedback gains an algebraic Riccati equation is repetitively solved at each iteration of the control algorithm. The stability properties of the control method are proven through Lyapunov analysis. The proposed nonlinear optimal control approach achieves fast and accurate tracking of setpoints under moderate variations of the control inputs and a minimum dispersion of energy when changing the position of the satellites in their triangular tethered formation.

Various duality models associated with some constrained robust nonlinear optimal control problems

This paper deals with various duality results for a new class of constrained robust nonlinear control problems. Concretely, we introduce a minimisation problem associated with a vector of functionals of multiple integral type. In addition, the considered multi-objective controlled optimisation problem is governed by several mixed constraints generated by partial derivatives of second order and uncertain data. For this new class of control problems, we establish and characterise Wolfe, Mond-Weir and mixed type robust dual models. To this aim, by using the notion of convex multiple integral functional, generated by a controlled second-order Lagrangian with uncertain data, and the concept of robust weak efficient solution to the problem under study, we build a new mathematical context to state and prove the duality theorems.

Nonlinear optimal control for robotic exoskeletons with electropneumatic actuators

Nonlinear optimal control for robotic exoskeletons with electropneumatic actuators

Hermite kernel surrogates for the value function of high-dimensional nonlinear optimal control problems

Numerical methods for the optimal feedback control of high-dimensional dynamical systems typically suffer from the curse of dimensionality. In the current presentation, we devise a mesh-free data-based approximation method for the value function of optimal control problems, which partially mitigates the dimensionality problem. The method is based on a greedy Hermite kernel interpolation scheme and incorporates context knowledge by its structure. Especially, the value function surrogate is elegantly enforced to be 0 in the target state, non-negative and constructed as a correction of a linearized model. The algorithm allows formulation in a matrix-free way which ensures efficient offline and online evaluation of the surrogate, circumventing the large-matrix problem for multivariate Hermite interpolation. Additionally, an incremental Cholesky factorization is utilized in the offline generation of the surrogate. For finite time horizons, both convergence of the surrogate to the value function and for the surrogate vs. the optimal controlled dynamical system are proven. Experiments support the effectiveness of the scheme, using among others a new academic model with an explicitly given value function. It may also be useful for the community to validate other optimal control approaches.

State-dependent trust region for successive convex programming for autonomous spacecraft

Spacecraft trajectory optimization is essential for all the different phases of a space mission, from its launch to end-of-life disposal. Due to the increase in the number of satellites and future space missions beyond our planet, increasing the level of autonomy of spacecraft is a key technical challenge. In this context, traditional trajectory optimization methods, like direct and indirect methods are not suited for autonomous or on-board operations due to the lack of guaranteed convergence or the high demand for computational power. Heuristic control laws represent an alternative in terms of computational power and convergence but they usually result in sub-optimal solutions. Successive convex programming (SCVX) enables to extend the application of convex optimization to non-linear optimal control problems. The definition of a good value of the trust region size plays a key role in the convergence of SCVX algorithms, and there is no systematic procedure to define it. This work presents an improved trust region based on the information given by the nonlinearities of the constraints which is unique for each optimization variable. In addition, differential algebra is adopted to automatize the transcription process required for SCVX algorithms. This new technique is first tested on a simple 2D problem as a benchmark of its performance and then applied to solve complex astrodynamics problems while providing a comparison with indirect, direct, and standard SCVX solutions.

Error analysis of Haar wavelet-based Galerkin numerical method with application to various nonlinear optimal control problems

ABSTRACT First, this paper defines a general nonlinear optimal control problem with state/control constraints and its approximation problem as the Haar wavelet Galerkin optimal control problem (HWGOCP). Then, a Haar wavelet-based Galerkin numerical method has been developed, which converts it to a nonlinear optimization problem. We theoretically prove that a Haar wavelet feasible solution of HWGOCP will exist. We also show that the approximate solutions of HWGOCP are consistent and converge to the optimal solution of the problem. A variety of application problems have been considered, which include optimal control of tumour growth using Chemotherapy drugs, optimal control of infection via the SIS model using treatment, the Brachistochrone problem in mechanics, optimal control of mold using a fungicide, optimal control of pH value of a chemical reaction to determine the quality of a product, etc.

Optimal solution of nonlinear 2D variable-order fractional optimal control problems using generalized Bessel polynomials

This study aims to propose a new optimization method based on the generalized Bessel polynomials (GBPs) as a class of basis functions for a category of nonlinear two-dimensional variable-order fractional optimal control problems (N-2D-VOFOCPs) involved in fractional-order dynamical systems and Caputo derivatives. For the optimal solution of such problems, the optimization method is developed on the basis of operational matrices (OMs) scheme of derivatives, 2D Gauss–Legendre quadrature rule, and Lagrange multiplier technique. The state and control functions are expanded in terms of the GBPs to reduce the complexity of these problems. The proposed method focuses on a system of nonlinear algebraic equations in the process of finding solution to the problems. The convergence of the method based on GBPs is proved, and the accuracy of the method is analyzed by solving several examples.

An approximating state-dependent control method based on modified pattern search optimization for nonlinear optimal control problem

In this paper, an approximating state-dependent control (ASC) method with modified pattern search (MPS) optimization for nonlinear optimal control problem is proposed. First, by converting the nonlinear optimal control problem into a number of interrelated time-varying linear quadratic regulator subproblems, the ASC method can solve each subproblem iteratively until the approximate solution is obtained. Second, in each iterative control process, the MPS is used to solve the controllability optimization problem. The optimal state-dependent weighting coefficients are obtained during the MPS optimization. Moreover, the MPS uses simplex gradient to design the search direction, which makes the optimization process efficient and fast. The convergence of MPS optimization is also proved in this paper. Finally, two simulation examples are given to illustrate the effectiveness of ASC method using the MPS optimization. The result shows that the ASC method can reduce the iterations of the approximate solution, and the MPS optimization can optimize the control performance of the ASC method.

Iterative distributed model predictive control for nonlinear systems with coupled non‐convex constraints and costs

AbstractThis paper proposes a distributed model predictive control (DMPC) algorithm for dynamic decoupled discrete‐time nonlinear systems subject to nonlinear (maybe non‐convex) coupled constraints and costs. Solving the resulting nonlinear optimal control problem (OCP) using a DMPC algorithm that is fully distributed, termination‐flexible, and recursively feasible for nonlinear systems with coupled constraints and costs remains an open problem. To address this, we propose a fully distributed and globally convergence‐guaranteed framework called inexact distributed sequential quadratic programming (IDSQP) for solving the OCP at each time step. Specifically, the proposed IDSQP framework has the following advantages: (i) it uses a distributed dual fast gradient approach for solving inner quadratic programming problems, enabling fully distributed execution; (ii) it can handle the adverse effects of inexact (insufficient) calculation of each internal quadratic programming problem caused by early termination of iterations, thereby saving computational time; and (iii) it employs distributed globalization techniques to eliminate the need for an initial guess of the solution. Under reasonable assumptions, the proposed DMPC algorithm ensures the recursive feasibility and stability of the entire closed‐loop system. We conduct simulation experiments on multi‐agent formation control with non‐convex collision avoidance constraints and compare the results against several benchmarks to verify the performance of the proposed DMPC method.