Class Of Optimal Control Problems Research Articles

MODERNIZING CLASSICAL OPTIMAL CONTROL: HARNESSING DIRECT AND INDIRECT OPTIMIZATION

An introduction to optimal control, a fundamental concept in engineering and science disciplines, is a process of finding ways of controlling dynamic systems in such a way that they achieve certain goals while being subjected to given state limitations. The conventional approaches developed in the past have been integral to solving optimal control problems, including the maximum principle belonging to Pontryagin and the dynamic programming method. While relatively straightforward, these methods are not always amenable to higher-dimensional scenarios, interacting forces, or other non-trivial constraints. This paper presents a new methodology to extend classical optimal control, considering both direct and indirect optimization techniques. The direct methods, Euler and Runge-Kutta, Trapezoidal, and Hermite-Simpson, do not require the explicit derivation of the analytical control laws to execute control trajectories. Semi-analytical techniques, such as the shooting method, are based the control laws on proposed adjoint differential equations. This paper presents the basic outline of the classical optimal control problem and discusses the direct and indirect optimization methods. It is illustrated by referencing various examples, like the fixed-rate royalty payment approach. After outlining each framework, we identify the positive and negative aspects of the approach in questions of consistency and performance. Finally, the problems on the synergy of direct and indirect methods are considered further, and areas of further development of the presented methods and their integration with the more sophisticated tools, such as machine learning are identified. It can be concluded that the approach of using both direct and indirect optimization methods presents great potential in modernizing the classical optimal control, which challenges the conventional techniques and indicates potential for further development of the control system optimization.

Advanced Model with a Trajectory Tracking Stabilisation System and Feasible Solution of the Optimal Control Problem

In this study, we consider the extended optimal control problem and search for a control function in the class of feasible functions for a real control object. Unlike the classical optimal control problem, the control function should depend on the state, not time. Therefore, the control synthesis problem for the initial-state domain should be solved, instead of the optimal control problem with one initial state. Alternatively, an optimal trajectory motion stabilisation system may be constructed. Both approaches—control and trajectory motion stabilisation system syntheses—cannot be applied to real-time control, as the task is too complex. The minimum threshold of quality criteria is searched for in the space of mathematical expression codes. Among other problems, the search space is difficult to define and the gradient is hard to determine. Therefore, the advanced control object model is used to obtain a feasible control function. The advanced model is firstly obtained before solving the optimal control problem and it already includes a trajectory motion stabilisation system; in particular, this stabilisation system is synthesised in advance at the control system design stage. When the optimal control problem appears, it is solved in real time in the classical statement, and a control function is searched for as a function of time. The advanced control object model also uses the reference model to generate the optimal trajectory. The search for the optimal control function is performed in real time and considers the synthesised stabilisation system of motion along a determined trajectory. Machine learning control via symbolic regression, namely, the network operator method, is used to directly solve the control synthesis problem. An example solution of the optimal control problem, with an advanced model moving in the environment with obstacles for a group of two mobile robots, is presented. The obtained solution is a control function for a reference model that generates a trajectory from a class of trajectories stabilised with the object’s control system.

Computing optimal drug dosing regarding efficacy and safety: the enhanced OptiDose method in NONMEM.

Recently, an optimal dosing algorithm (OptiDose) was developed to compute the optimal drug doses for any pharmacometrics model for a given dosing scenario. In the present work, we enhance the OptiDose concept to compute optimal drug dosing with respect to both efficacy and safety targets. Usually, these are not of equal importance, but one is a top priority, that needs to be satisfied, whereas the other is a secondary target and should be achieved as good as possible without failing the top priority target. Mathematically, this leads to state-constrained optimal control problems. In this paper, we elaborate how to set up such problems and transform them into classical unconstrained optimal control problems which can be solved in NONMEM. Three different optimal dosing tasks illustrate the impact of the proposed enhanced OptiDose method.

A Second-Order Numerical Method for a Class of Optimal Control Problems

The numerical solution of optimal control problems through second-order methods is examined in this paper. Controlled processes are described by a system of nonlinear ordinary differential equations. There are two specific characteristics of the class of control actions used. The first one is that controls are searched for in a given class of functions, which depend on unknown parameters to be found by minimizing an objective functional. The parameter values, in general, may be different at different time intervals. The second feature of the considered problem is that the boundaries of time intervals are also optimized with fixed values of the parameters of the control actions in each of the intervals. The special cases of the problem under study are relay control problems with optimized switching moments. In this work, formulas for the gradient and the Hessian matrix of the objective functional with respect to the optimized parameters are obtained. For this, the technique of fast differentiation is used. A comparison of numerical experiment results obtained with the use of first- and second-order optimization methods is presented.

An accurate wavelets‐collocation technique for neutral delay distributed‐order fractional optimal control problems

AbstractIn this study, a new class of optimal control problems called neutral delay distributed‐order fractional optimal control problems is introduced, this problem is solved based on an efficient computational scheme. To solve the problem, we derive an exact formula for the Riemann–Liouville fractional integral operator of Genocchi wavelets based on beta functions for the first time. By taking into account this operator, collocation method, and Gauss–Legendre integration formula, the solution of fractional optimal control problems (FOCPs) under consideration is converted to a nonlinear programming one to which existing well‐developed algorithms may be applied. The mentioned scheme is applied to both FOCPs with or without delay. Error analysis associated with the proposed idea is also investigated under several mild conditions. The effectiveness of the strategy is showed by several illustrative examples, furthermore, a comparison with the previous methods highlights the preference of this scheme.

On the regularity of multipliers and second-order optimality conditions of KKT-type for semilinear parabolic control problems

A class of optimal control problems governed by semilinear parabolic equations with mixed constraints and a box constraint for control variable is considered. We show that if the so-called generalized separation condition is satisfied, then both optimality conditions of KKT-type and regularity of multipliers are fulfilled. Moreover, we show that if the initial value is good enough and boundary ∂Ω has a property of positive geometric density, then multipliers and optimal solutions are Hölder continuous.

Quaternary Continuous Classical Boundary Optimal Control Problem Dominating by Parabolic System

In this paper, our purpose is to study the quaternary continuous classical boundary optimal control vector problem dominated by a quaternary linear parabolic boundary value problem. Under suitable assumptions and with a given quaternary continuous classical boundary control vector, the existence theorem for a unique quaternary state vector solution to the weak form is stated and demonstrated via the method of Galerkin. Furthermore, the continuity of the Lipschitz operator between the state vector solution to the weak form for the dominating equations and the corresponding are proved. The existence of a quaternary continuous classical boundary optimal control vector is stated and demonstrated under suitable Assumptions. The mathematical formulation for the quaternary adjoint boundary value problem associated with each considered boundary value problem is obtained and the Frѐchet derivative for the objective function is derived. Finally, the necessary conditions for the optimality theorem of the problem are stated and demonstrated.

Bellman Neural Networks for the Class of Optimal Control Problems With Integral Quadratic Cost

This work introduces Bellman Neural Networks (BeNNs) and employs them to learn the optimal control actions for the class of optimal control problems (OCPs) with integral quadratic cost. BeNNs represent a particular family of Physics-Informed Neural Networks (PINNs) specifically designed and trained to tackle OCPs via applying the Bellman Principle of Optimality (BPO). The BPO provides necessary and sufficient optimality conditions, which result in a nonlinear partial differential equation known as the Hamilton-Jacobi-Bellman (HJB) equation. BeNNs learn the optimal control actions from the unknown solution of the arising HJB equation (i.e., the value function), where the unknown solution is modeled using a Neural Network. Additionally, the paper shows how to estimate the upper bounds on the generalization error of BeNNs while learning the solutions for the OCP class under consideration. The generalization error estimate is provided in terms of the choice and number of the training points as well as the training error. Numerical studies show that BeNNs can be successfully applied to learn the feedback control actions for the class of optimal control problems considered and, after the training is completed, deployed to control the system in a closed-loop fashion. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Impact Statement</i> —The proposed research improves our understanding of how to solve optimal control problems with closed-loop solutions and has potentially a countless number of applications in several different areas. The study is at the intersection between optimal control theory and artificial intelligence connected with mathematical tools for functional interpolation. This advances the ability to implement a higher level of autonomy in decision-making for practical applications with a beneficial impact on our society.

Cooperative Eco-Driving Control of Connected Multi-Vehicles With Spatio-Temporal Constraints

In this article, we propose a novel time-energy optimal control approach with applications in cooperative eco- driving of connected and automated vehicles (CAVs) in urban traffic networks. Safely approaching and departing signalized intersections requires the satisfaction of both spatial equality constraints determined by intersection locations and temporal inequality constraints in compliance with the green light phases. To generate time- and energy-optimal trajectories, the optimal crossing times at intersections are firstly treated as characteristic time constraints, which makes the problem tractable. Then the di- rect multiple shooting method and time transformation technique are applied to find a numerical solution. The contribution of this article is twofold. The first one is the development of a novel time- and energy-optimal control approach that ensures a trade- off between minimizing energy and time for a general class of optimal control problems with multiple characteristic times. The second contribution is the application of the proposed method to the challenging problem of multi-CAVs’ cooperative eco-driving control, in which multiple vehicles must simultaneously minimize travel time and energy consumption in the presence of spatio- temporal constraints. Simulation analysis over real-world urban route scenarios shows that the proposed eco-driving control strategy can save up to 8.2% of energy or reduce up to 6.7% of travel time compared to a baseline method. Furthermore, hardware-in-the-loop (HiL) experimental results indicate that the proposed strategy can be implemented in real-time.

Geometric preconditioner for indirect shooting and application to hybrid vehicle

In this article, we are interested in the hybrid electric vehicle torque split and gear shift problem, which can be formulated as a classical Lagrange optimal control problem with fixed initial condition. The Pontryagin maximum principle gives necessary optimality conditions adjoining to the state a covector called costate. Thus, the optimal state trajectory has to be found among the projections of the lifted trajectories, called Pontryagin extremals, given by the maximum principle. The indirect simple shooting method aims to compute Pontryagin extremals reducing the resolution to the research of the initial costate. Classically, a Newtonlike solver is used to compute zeros of the so-called shooting equations. The main drawback of this method is its sensitivity to the initial guess. Indeed, a good one needs to be given to make the Newton solver converge, which is not an easy task in practice. We propose a preconditioning method of the shooting function based on a geometrical interpretation of the costate, in relation with the reachable set of the extended system and the underlying symplectic structure. We numerically show that this method reduces the number of iterations of our solver. Remarkably, in our experiments, it is better to use the preconditioner than a clever initial guess for the Newton solver.

An improved slope-based adaptive control vector parameterization method for dynamic programming

Control vector parameterization method is the most commonly used numerical computation method in solving dynamic programming problems. However, to balance the expected trajectory and the computational cost, this method faces difficulties in dividing the optimal discretization time grid. In this paper, we propose an effective control approach for non-uniform adaptive grid division. By analyzing the slope change trend of the control parameter, time nodes are adaptively refined by merging time grids with gentle slopes to remove unnecessary time nodes and adding time nodes to time grids with steep slopes to improve function approximation accuracy. Eventually, we obtain an adaptive time grid division method under which the trajectory of discretization control parameter is more approximate to the optimal control trajectory. Under the condition of intuitive slope information, the proposed method can achieve better performance with fewer optimization parameters and shorter computation time. Finally, the proposed method is applied to solve a classic optimal control problem and the obtained results are compared with the traditional control vector parameterization method. By comparison through the example, our proposed method can overcome the contradiction between approximation accuracy and computational cost in control vector parameterization method and improve the optimization efficiency.

Relaxation approach for learning neural network regularizers for a class of identification problems

The present paper deals with the data-driven design of regularizers in the form of artificial neural networks, for solving certain inverse problems formulated as optimal control problems. These regularizers aim at improving accuracy, wellposedness or compensating uncertainties for a given class of optimal control problems (inner-problems). Parameterized as neural networks, their weights are chosen in order to reduce a misfit between data and observations of the state solution of the inner- optimal control problems. Learning these weights constitutes the outer-problem. Based on necessary first-order optimality conditions for the inner-problems, a relaxation approach is proposed in order to implement efficient solving of these inner-problems, namely the forward operator of the outer-problem. Optimality conditions are derived for the latter, and are implemented in numerical illustrations dealing with the inverse conductivity problem. The numerical tests show the feasibility of the relaxation approach, first for rediscovering standard L 2-regularizers, and next for designing regularizers that compensate unknown noise on the observed state of the inner-problem.

Optimal Control of Quasi-Stationary Equations of Complex Heat Transfer with Reflection and Refraction Conditions

The paper considers a class of optimal control problems for a nonlinear parabolic-elliptic system simulating radiative heat transfer with Fresnel matching conditions on surfaces of discontinuity of the refractive index. New estimates for the solution of the initial-boundary value problem are obtained, on the basis of which the solvability of optimal control problems is proved. Non-degenerate first-order optimality conditions are derived. The results are examplified by control problems with final, boundary, and distributed observations.

A Class of Optimal Control Problems of Forward–Backward Systems with Input Constraint

A Class of Optimal Control Problems of Forward–Backward Systems with Input Constraint

A shape parameter insensitive CRBFs‐collocation for solving nonlinear optimal control problems

AbstractIn this work, we introduce a Coupled Radial Basis Functions collocation (CRBFs‐Coll) approach for solving optimal control problems. CRBFs are real‐valued Radial Basis Functions (RBFs) augmented with a conical spline. They show insensitivity to the shape parameter resulting in robust behavior in function approximation. The method is applied to classical nonlinear optimal control problems: Zermelo's problem, a Duffing oscillator with various boundary conditions, and a nonlinear pendulum on a cart problem. The nonlinear optimal control problem is solved by deriving the necessary conditions for optimality followed by collocation using CRBFs as basis functions for approximating the two‐point boundary value problem (TPBVP). The resulting system of nonlinear algebraic equations (NAEs) is then solved using a standard solver. Unlike existing methods that rely on nodal distributions that are denser at the boundaries to obtain a solution, the present CRBFs‐Coll approach is capable of solving the optimal control problem on uniform nodes without the need for interpolation. The present CRBFs‐Coll approach is shown to be simple to implement and provides accurate results over an equidistant nodal distribution while maintaining a continuous representation of the control and states.

Stochastic maximum principle for recursive optimal control problems with varying terminal time

This paper introduces a new recursive stochastic optimal control problem driven by a forward-backward stochastic differential equations (FBSDEs), where the terminal time varies according to the constraints of the state of the forward equation. We can use this optimal control problem to describe the investment portfolio problems with the varying investment period. Based on novel ρ-moving variational and adjoint equations, we establish the stochastic maximum principle for this optimal control problem including the classical recursive optimal control problem as a particular case. Furthermore, we propose an example to verify the main results of this study.

Optimal control in linear-quadratic stochastic advertising models with memory

This paper deals with a class of optimal control problems which arises in advertising models with Volterra Ornstein-Uhlenbeck process representing the product goodwill. Such choice of the model can be regarded as a stochastic modification of the classical Nerlove-Arrow model that allows to incorporate both presence of uncertainty and empirically observed memory effects such as carryover or distributed forgetting. We present an approach to solve such optimal control problems based on an infinite dimensional lift which allows us to recover Markov properties by formulating an optimization problem equivalent to the original one in a Hilbert space. Such technique, however, requires the Volterra kernel from the forward equation to have a representation of a particular form that may be challenging to obtain in practice. We overcome this issue for Hölder continuous kernels by approximating them with Bernstein polynomials, which turn out to enjoy a simple representation of the required type. Then we solve the optimal control problem for the forward process with approximated kernel instead of the original one and study convergence. The approach is illustrated with simulations.

Optimal control problems in transport dynamics with additive noise

Motivated by the applications in leader-follower multi-agent dynamics, a class of optimal control problems is investigated, where the goal is to influence the behavior of a given population through another controlled one interacting with the first. The evolution of the systems obeys a non-linear Fokker–Planck-type equation as diffusive terms accounting for randomness in the evolution are taken into account. A well-posedness theory under very low regularity of the control vector fields is developed, as well as a rigorous mean-field limit of a stochastic particle approximation.

A collocation method using generalized Laguerre polynomials for solving nonlinear optimal control problems governed by integro-differential equations

The Laguerre polynomials represent a new set of basic functions adopted to solve a class of optimal control problems (OCP) specified by integro-differential equation namely IOCP. The corresponding operational matrices of derivatives are computed to extend the solution of the problem in terms of Laguerre polynomials. Using the results of the considered basis functions with a joint application of collocation method, the IOCP is reduced to solve a system of nonlinear algebraic system. Come acrossing an error bound and convergence analysis for the approximate optimal value of the performance index have been proved by the proposed method. Finally, some examples are given to bring about the validity and applicability of this technique.

A note on the minimum‐norm in dual space approach to some classical linear optimal control problems

AbstractSome classical optimal control problems of linear systems can be characterized as finding minimum‐norm vectors from within linear varieties in appropriate dual spaces. Then, the solution to such an optimal control problem can be derived from the alignment between the optimal vector in the dual space and the optimal vector in the primal space, and the dual maximization problem of the minimum‐norm problem. This note presents a detailed introduction to this minimum‐norm in dual space approach by examples of minimum‐supremum‐norm, minimum‐energy, and minimum‐time optimal control problems of linear systems. Connections and differences between these problems in light of the introduced approach are discussed.